Oct 30 18:31:04.918518 ip-10-0-134-46 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Oct 30 18:31:04.918530 ip-10-0-134-46 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Oct 30 18:31:04.918537 ip-10-0-134-46 systemd[1]: kubelet.service: Failed with result 'resources'.
Oct 30 18:31:04.918767 ip-10-0-134-46 systemd[1]: Failed to start Kubernetes Kubelet.
Oct 30 18:31:15.005557 ip-10-0-134-46 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Oct 30 18:31:15.005572 ip-10-0-134-46 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot d8d2fdad954d46d6a1038aed2454c617 --
Oct 30 18:33:12.191647 ip-10-0-134-46 systemd[1]: Starting Kubernetes Kubelet...
Oct 30 18:33:12.583624 ip-10-0-134-46 kubenswrapper[2574]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 18:33:12.583624 ip-10-0-134-46 kubenswrapper[2574]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Oct 30 18:33:12.583624 ip-10-0-134-46 kubenswrapper[2574]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 18:33:12.583624 ip-10-0-134-46 kubenswrapper[2574]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 18:33:12.587399 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.587309    2574 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 18:33:12.587399 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.587399    2574 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 18:33:12.587399 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.587402    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 18:33:12.587492 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.587406    2574 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 18:33:12.587492 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.587408    2574 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 18:33:12.587492 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.587411    2574 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 18:33:12.587492 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.587414    2574 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 18:33:12.587492 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.587418    2574 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 18:33:12.587492 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.587420    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 18:33:12.587492 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.587423    2574 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 18:33:12.587492 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.587425    2574 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 18:33:12.587492 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.587428    2574 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 18:33:12.587492 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.587431    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 18:33:12.587492 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.587434    2574 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 18:33:12.587492 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.587437    2574 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 18:33:12.587492 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.587440    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 18:33:12.587492 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.587442    2574 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 18:33:12.587492 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.587445    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 18:33:12.587492 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.587447    2574 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 18:33:12.587492 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.587450    2574 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 18:33:12.587492 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.587452    2574 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 18:33:12.587492 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.587455    2574 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 18:33:12.587492 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.587458    2574 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 18:33:12.587949 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.587460    2574 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 18:33:12.587949 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.587463    2574 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 18:33:12.587949 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.587465    2574 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 18:33:12.587949 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.587468    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 18:33:12.587949 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.587470    2574 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 18:33:12.587949 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.587472    2574 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 18:33:12.587949 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.587475    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 18:33:12.587949 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.587477    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 18:33:12.587949 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.587479    2574 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 18:33:12.587949 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.587482    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 18:33:12.587949 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.587484    2574 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 18:33:12.587949 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.587487    2574 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 18:33:12.587949 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.587490    2574 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 18:33:12.587949 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.587493    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 18:33:12.587949 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.587496    2574 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 18:33:12.587949 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.587499    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 18:33:12.587949 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.587502    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 18:33:12.587949 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.587505    2574 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 18:33:12.587949 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.587507    2574 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 18:33:12.588413 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.587510    2574 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 18:33:12.588413 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.587512    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 18:33:12.588413 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.587515    2574 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 18:33:12.588413 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.587517    2574 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 18:33:12.588413 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.587520    2574 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 18:33:12.588413 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.587522    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 18:33:12.588413 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.587525    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 18:33:12.588413 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.587527    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 18:33:12.588413 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.587530    2574 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 18:33:12.588413 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.587532    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 18:33:12.588413 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.587534    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 18:33:12.588413 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.587537    2574 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 18:33:12.588413 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.587539    2574 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 18:33:12.588413 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.587542    2574 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 18:33:12.588413 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.587544    2574 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 18:33:12.588413 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.587547    2574 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 18:33:12.588413 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.587549    2574 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 18:33:12.588413 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.587552    2574 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 18:33:12.588413 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.587556    2574 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 18:33:12.588882 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.587561    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 18:33:12.588882 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.587564    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 18:33:12.588882 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.587566    2574 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 18:33:12.588882 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.587569    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 18:33:12.588882 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.587573    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 18:33:12.588882 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.587575    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 18:33:12.588882 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.587578    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 18:33:12.588882 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.587581    2574 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 18:33:12.588882 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.587584    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 18:33:12.588882 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.587587    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 18:33:12.588882 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.587590    2574 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 18:33:12.588882 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.587593    2574 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 18:33:12.588882 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.587596    2574 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 18:33:12.588882 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.587598    2574 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 18:33:12.588882 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.587601    2574 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 18:33:12.588882 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.587603    2574 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 18:33:12.588882 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.587605    2574 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 18:33:12.588882 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.587608    2574 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 18:33:12.588882 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.587610    2574 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 18:33:12.588882 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.587613    2574 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 18:33:12.589342 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.587615    2574 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 18:33:12.589342 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.587618    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 18:33:12.589342 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.587620    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 18:33:12.589342 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.587623    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 18:33:12.589342 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.587625    2574 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 18:33:12.589342 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.587628    2574 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 18:33:12.589342 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.587630    2574 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 18:33:12.589342 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.587633    2574 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 18:33:12.589342 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.587635    2574 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 18:33:12.589342 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.587637    2574 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 18:33:12.589342 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.587640    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 18:33:12.589342 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.587650    2574 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 18:33:12.589342 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.587652    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 18:33:12.589342 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.587655    2574 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 18:33:12.589342 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.587658    2574 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 18:33:12.589342 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.587660    2574 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 18:33:12.589342 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.587663    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 18:33:12.589342 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.587665    2574 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 18:33:12.589342 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.587668    2574 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 18:33:12.589813 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.587671    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 18:33:12.589813 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.587674    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 18:33:12.589813 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.587677    2574 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 18:33:12.589813 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.587679    2574 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 18:33:12.589813 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.587684    2574 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 18:33:12.589813 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.587689    2574 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 18:33:12.589813 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.588443    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 18:33:12.589813 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.588451    2574 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 18:33:12.589813 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.588454    2574 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 18:33:12.589813 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.588458    2574 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 18:33:12.589813 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.588460    2574 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 18:33:12.589813 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.588463    2574 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 18:33:12.589813 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.588466    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 18:33:12.589813 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.588468    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 18:33:12.589813 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.588471    2574 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 18:33:12.589813 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.588474    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 18:33:12.589813 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.588476    2574 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 18:33:12.589813 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.588479    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 18:33:12.589813 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.588482    2574 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 18:33:12.590383 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.588484    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 18:33:12.590383 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.588487    2574 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 18:33:12.590383 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.588489    2574 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 18:33:12.590383 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.588492    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 18:33:12.590383 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.588495    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 18:33:12.590383 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.588498    2574 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 18:33:12.590383 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.588500    2574 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 18:33:12.590383 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.588503    2574 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 18:33:12.590383 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.588505    2574 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 18:33:12.590383 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.588508    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 18:33:12.590383 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.588510    2574 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 18:33:12.590383 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.588512    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 18:33:12.590383 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.588515    2574 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 18:33:12.590383 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.588517    2574 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 18:33:12.590383 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.588520    2574 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 18:33:12.590383 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.588522    2574 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 18:33:12.590383 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.588525    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 18:33:12.590383 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.588527    2574 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 18:33:12.590383 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.588530    2574 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 18:33:12.590383 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.588532    2574 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 18:33:12.590878 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.588535    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 18:33:12.590878 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.588538    2574 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 18:33:12.590878 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.588540    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 18:33:12.590878 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.588543    2574 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 18:33:12.590878 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.588545    2574 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 18:33:12.590878 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.588547    2574 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 18:33:12.590878 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.588550    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 18:33:12.590878 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.588552    2574 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 18:33:12.590878 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.588555    2574 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 18:33:12.590878 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.588559    2574 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 18:33:12.590878 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.588561    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 18:33:12.590878 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.588564    2574 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 18:33:12.590878 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.588567    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 18:33:12.590878 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.588570    2574 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 18:33:12.590878 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.588572    2574 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 18:33:12.590878 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.588574    2574 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 18:33:12.590878 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.588578    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 18:33:12.590878 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.588580    2574 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 18:33:12.590878 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.588583    2574 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 18:33:12.590878 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.588586    2574 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 18:33:12.591348 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.588588    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 18:33:12.591348 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.588590    2574 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 18:33:12.591348 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.588593    2574 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 18:33:12.591348 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.588595    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 18:33:12.591348 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.588598    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 18:33:12.591348 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.588600    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 18:33:12.591348 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.588603    2574 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 18:33:12.591348 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.588605    2574 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 18:33:12.591348 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.588607    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 18:33:12.591348 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.588610    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 18:33:12.591348 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.588612    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 18:33:12.591348 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.588615    2574 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 18:33:12.591348 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.588618    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 18:33:12.591348 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.588621    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 18:33:12.591348 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.588623    2574 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 18:33:12.591348 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.588626    2574 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 18:33:12.591348 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.588628    2574 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 18:33:12.591348 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.588630    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 18:33:12.591348 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.588633    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 18:33:12.591846 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.588636    2574 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 18:33:12.591846 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.588641    2574 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 18:33:12.591846 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.588644    2574 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 18:33:12.591846 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.588648    2574 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 18:33:12.591846 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.588650    2574 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 18:33:12.591846 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.588653    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 18:33:12.591846 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.588656    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 18:33:12.591846 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.588658    2574 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 18:33:12.591846 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.588660    2574 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 18:33:12.591846 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.588663    2574 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 18:33:12.591846 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.588666    2574 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 18:33:12.591846 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.588668    2574 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 18:33:12.591846 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.588671    2574 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 18:33:12.591846 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.588673    2574 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 18:33:12.591846 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.588676    2574 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 18:33:12.591846 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.588678    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 18:33:12.591846 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.588681    2574 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 18:33:12.591846 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.588683    2574 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 18:33:12.591846 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.588686    2574 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 18:33:12.592297 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.588688    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 18:33:12.592297 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.588691    2574 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 18:33:12.592297 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.588693    2574 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 18:33:12.592297 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.588697    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 18:33:12.592297 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.588699    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 18:33:12.592297 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.588702    2574 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 18:33:12.592297 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.588704    2574 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 18:33:12.592297 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.588707    2574 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 18:33:12.592297 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.588710    2574 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 18:33:12.592297 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.588713    2574 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 18:33:12.592297 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.588715    2574 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 18:33:12.592297 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.588721    2574 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 18:33:12.592297 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.588723    2574 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 18:33:12.592297 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.588725    2574 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 18:33:12.592297 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.588728    2574 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 18:33:12.592297 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.589196    2574 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 18:33:12.592297 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.589201    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 18:33:12.592297 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.589204    2574 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 18:33:12.592297 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.589207    2574 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 18:33:12.592297 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.589210    2574 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 18:33:12.592848 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.589212    2574 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 18:33:12.592848 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.589215    2574 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 18:33:12.592848 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.589218    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 18:33:12.592848 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.589221    2574 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 18:33:12.592848 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.589224    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 18:33:12.592848 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.589226    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 18:33:12.592848 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.589229    2574 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 18:33:12.592848 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.589231    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 18:33:12.592848 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.589234    2574 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 18:33:12.592848 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.589237    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 18:33:12.592848 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.589239    2574 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 18:33:12.592848 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.589242    2574 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 18:33:12.592848 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.589244    2574 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 18:33:12.592848 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.589247    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 18:33:12.592848 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.589249    2574 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 18:33:12.592848 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.589251    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 18:33:12.592848 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.589254    2574 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 18:33:12.592848 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.589256    2574 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 18:33:12.592848 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.589259    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 18:33:12.592848 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.589262    2574 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 18:33:12.593312 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.589265    2574 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 18:33:12.593312 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.589269    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 18:33:12.593312 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.589273    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 18:33:12.593312 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.589276    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 18:33:12.593312 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.589279    2574 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 18:33:12.593312 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.589281    2574 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 18:33:12.593312 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.589283    2574 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 18:33:12.593312 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.589286    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 18:33:12.593312 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.589288    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 18:33:12.593312 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.589292    2574 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 18:33:12.593312 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.589297    2574 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 18:33:12.593312 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.589300    2574 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 18:33:12.593312 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.589303    2574 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 18:33:12.593312 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.589306    2574 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 18:33:12.593312 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.589308    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 18:33:12.593312 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.589311    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 18:33:12.593312 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.589314    2574 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 18:33:12.593312 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.589316    2574 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 18:33:12.593312 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.589319    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 18:33:12.593776 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.589321    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 18:33:12.593776 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.589324    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 18:33:12.593776 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.589326    2574 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 18:33:12.593776 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.589328    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 18:33:12.593776 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.589331    2574 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 18:33:12.593776 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.589333    2574 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 18:33:12.593776 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.589335    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 18:33:12.593776 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.589338    2574 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 18:33:12.593776 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.589340    2574 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 18:33:12.593776 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.589343    2574 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 18:33:12.593776 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.589345    2574 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 18:33:12.593776 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.589348    2574 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 18:33:12.593776 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.589351    2574 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 18:33:12.593776 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.589354    2574 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 18:33:12.593776 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.589356    2574 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 18:33:12.593776 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.589373    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 18:33:12.593776 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.589376    2574 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 18:33:12.593776 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.589379    2574 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 18:33:12.593776 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.589381    2574 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 18:33:12.594257 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.589384    2574 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 18:33:12.594257 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.589386    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 18:33:12.594257 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.589389    2574 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 18:33:12.594257 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.589391    2574 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 18:33:12.594257 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.589394    2574 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 18:33:12.594257 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.589396    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 18:33:12.594257 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.589399    2574 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 18:33:12.594257 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.589403    2574 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 18:33:12.594257 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.589405    2574 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 18:33:12.594257 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.589407    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 18:33:12.594257 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.589410    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 18:33:12.594257 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.589413    2574 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 18:33:12.594257 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.589415    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 18:33:12.594257 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.589417    2574 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 18:33:12.594257 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.589420    2574 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 18:33:12.594257 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.589422    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 18:33:12.594257 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.589425    2574 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 18:33:12.594257 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.589427    2574 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 18:33:12.594257 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.589430    2574 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 18:33:12.594726 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.589432    2574 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 18:33:12.594726 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.589435    2574 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 18:33:12.594726 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.589437    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 18:33:12.594726 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.589439    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 18:33:12.594726 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.589442    2574 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 18:33:12.594726 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.589444    2574 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 18:33:12.594726 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.589447    2574 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 18:33:12.594726 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.589449    2574 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 18:33:12.594726 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.589452    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 18:33:12.594726 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.589455    2574 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 18:33:12.594726 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.589458    2574 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 18:33:12.594726 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.589461    2574 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 18:33:12.594726 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.589463    2574 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 18:33:12.594726 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.589465    2574 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 18:33:12.594726 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.589468    2574 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 18:33:12.594726 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.589470    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 18:33:12.594726 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.589473    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 18:33:12.594726 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.589475    2574 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 18:33:12.594726 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.589477    2574 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 18:33:12.594726 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.589480    2574 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 18:33:12.595192 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.589483    2574 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 18:33:12.595192 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.589487    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 18:33:12.595192 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.589490    2574 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 18:33:12.595192 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.589492    2574 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 18:33:12.595192 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.589580    2574 flags.go:64] FLAG: --address="0.0.0.0"
Oct 30 18:33:12.595192 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.589587    2574 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Oct 30 18:33:12.595192 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.589596    2574 flags.go:64] FLAG: --anonymous-auth="true"
Oct 30 18:33:12.595192 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.589602    2574 flags.go:64] FLAG: --application-metrics-count-limit="100"
Oct 30 18:33:12.595192 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.589609    2574 flags.go:64] FLAG: --authentication-token-webhook="false"
Oct 30 18:33:12.595192 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.589613    2574 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Oct 30 18:33:12.595192 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.589619    2574 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Oct 30 18:33:12.595192 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.589625    2574 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Oct 30 18:33:12.595192 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.589629    2574 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Oct 30 18:33:12.595192 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.589633    2574 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Oct 30 18:33:12.595192 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.589637    2574 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Oct 30 18:33:12.595192 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.589642    2574 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Oct 30 18:33:12.595192 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.589646    2574 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Oct 30 18:33:12.595192 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.589650    2574 flags.go:64] FLAG: --cgroup-root=""
Oct 30 18:33:12.595192 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.589654    2574 flags.go:64] FLAG: --cgroups-per-qos="true"
Oct 30 18:33:12.595192 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.589658    2574 flags.go:64] FLAG: --client-ca-file=""
Oct 30 18:33:12.595192 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.589662    2574 flags.go:64] FLAG: --cloud-provider="external"
Oct 30 18:33:12.595192 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.589667    2574 flags.go:64] FLAG: --cluster-dns="[]"
Oct 30 18:33:12.595192 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.589673    2574 flags.go:64] FLAG: --cluster-domain=""
Oct 30 18:33:12.595192 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.589677    2574 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Oct 30 18:33:12.595778 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.589682    2574 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Oct 30 18:33:12.595778 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.589687    2574 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Oct 30 18:33:12.595778 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.589692    2574 flags.go:64] FLAG: --container-log-max-files="5"
Oct 30 18:33:12.595778 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.589697    2574 flags.go:64] FLAG: --container-log-max-size="10Mi"
Oct 30 18:33:12.595778 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.589702    2574 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Oct 30 18:33:12.595778 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.589707    2574 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Oct 30 18:33:12.595778 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.589711    2574 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Oct 30 18:33:12.595778 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.589715    2574 flags.go:64] FLAG: --contention-profiling="false"
Oct 30 18:33:12.595778 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.589721    2574 flags.go:64] FLAG: --cpu-cfs-quota="true"
Oct 30 18:33:12.595778 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.589725    2574 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Oct 30 18:33:12.595778 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.589729    2574 flags.go:64] FLAG: --cpu-manager-policy="none"
Oct 30 18:33:12.595778 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.589733    2574 flags.go:64] FLAG: --cpu-manager-policy-options=""
Oct 30 18:33:12.595778 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.589739    2574 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Oct 30 18:33:12.595778 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.589743    2574 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Oct 30 18:33:12.595778 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.589747    2574 flags.go:64] FLAG: --enable-debugging-handlers="true"
Oct 30 18:33:12.595778 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.589752    2574 flags.go:64] FLAG: --enable-load-reader="false"
Oct 30 18:33:12.595778 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.589757    2574 flags.go:64] FLAG: --enable-server="true"
Oct 30 18:33:12.595778 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.589761    2574 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Oct 30 18:33:12.595778 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.589769    2574 flags.go:64] FLAG: --event-burst="100"
Oct 30 18:33:12.595778 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.589774    2574 flags.go:64] FLAG: --event-qps="50"
Oct 30 18:33:12.595778 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.589777    2574 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Oct 30 18:33:12.595778 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.589780    2574 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Oct 30 18:33:12.595778 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.589783    2574 flags.go:64] FLAG: --eviction-hard=""
Oct 30 18:33:12.595778 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.589787    2574 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Oct 30 18:33:12.596410 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.589790    2574 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Oct 30 18:33:12.596410 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.589793    2574 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Oct 30 18:33:12.596410 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.589796    2574 flags.go:64] FLAG: --eviction-soft=""
Oct 30 18:33:12.596410 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.589799    2574 flags.go:64] FLAG: --eviction-soft-grace-period=""
Oct 30 18:33:12.596410 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.589802    2574 flags.go:64] FLAG: --exit-on-lock-contention="false"
Oct 30 18:33:12.596410 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.589806    2574 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Oct 30 18:33:12.596410 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.589810    2574 flags.go:64] FLAG: --experimental-mounter-path=""
Oct 30 18:33:12.596410 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.589813    2574 flags.go:64] FLAG: --fail-cgroupv1="true"
Oct 30 18:33:12.596410 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.589816    2574 flags.go:64] FLAG: --fail-swap-on="true"
Oct 30 18:33:12.596410 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.589819    2574 flags.go:64] FLAG: --feature-gates=""
Oct 30 18:33:12.596410 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.589824    2574 flags.go:64] FLAG: --file-check-frequency="20s"
Oct 30 18:33:12.596410 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.589827    2574 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Oct 30 18:33:12.596410 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.589830    2574 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Oct 30 18:33:12.596410 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.589833    2574 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Oct 30 18:33:12.596410 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.589837    2574 flags.go:64] FLAG: --healthz-port="10248"
Oct 30 18:33:12.596410 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.589840    2574 flags.go:64] FLAG: --help="false"
Oct 30 18:33:12.596410 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.589844    2574 flags.go:64] FLAG: --hostname-override="ip-10-0-134-46.us-west-1.compute.internal"
Oct 30 18:33:12.596410 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.589847    2574 flags.go:64] FLAG: --housekeeping-interval="10s"
Oct 30 18:33:12.596410 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.589850    2574 flags.go:64] FLAG: --http-check-frequency="20s"
Oct 30 18:33:12.596410 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.589853    2574 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Oct 30 18:33:12.596410 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.589857    2574 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Oct 30 18:33:12.596410 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.589860    2574 flags.go:64] FLAG: --image-gc-high-threshold="85"
Oct 30 18:33:12.596410 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.589863    2574 flags.go:64] FLAG: --image-gc-low-threshold="80"
Oct 30 18:33:12.596953 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.589866    2574 flags.go:64] FLAG: --image-service-endpoint=""
Oct 30 18:33:12.596953 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.589868    2574 flags.go:64] FLAG: --kernel-memcg-notification="false"
Oct 30 18:33:12.596953 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.589871    2574 flags.go:64] FLAG: --kube-api-burst="100"
Oct 30 18:33:12.596953 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.589874    2574 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Oct 30 18:33:12.596953 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.589877    2574 flags.go:64] FLAG: --kube-api-qps="50"
Oct 30 18:33:12.596953 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.589880    2574 flags.go:64] FLAG: --kube-reserved=""
Oct 30 18:33:12.596953 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.589883    2574 flags.go:64] FLAG: --kube-reserved-cgroup=""
Oct 30 18:33:12.596953 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.589885    2574 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Oct 30 18:33:12.596953 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.589888    2574 flags.go:64] FLAG: --kubelet-cgroups=""
Oct 30 18:33:12.596953 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.589891    2574 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Oct 30 18:33:12.596953 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.589894    2574 flags.go:64] FLAG: --lock-file=""
Oct 30 18:33:12.596953 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.589897    2574 flags.go:64] FLAG: --log-cadvisor-usage="false"
Oct 30 18:33:12.596953 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.589899    2574 flags.go:64] FLAG: --log-flush-frequency="5s"
Oct 30 18:33:12.596953 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.589902    2574 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Oct 30 18:33:12.596953 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.589909    2574 flags.go:64] FLAG: --log-json-split-stream="false"
Oct 30 18:33:12.596953 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.589911    2574 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Oct 30 18:33:12.596953 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.589915    2574 flags.go:64] FLAG: --log-text-split-stream="false"
Oct 30 18:33:12.596953 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.589918    2574 flags.go:64] FLAG: --logging-format="text"
Oct 30 18:33:12.596953 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.589921    2574 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Oct 30 18:33:12.596953 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.589924    2574 flags.go:64] FLAG: --make-iptables-util-chains="true"
Oct 30 18:33:12.596953 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.589927    2574 flags.go:64] FLAG: --manifest-url=""
Oct 30 18:33:12.596953 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.589929    2574 flags.go:64] FLAG: --manifest-url-header=""
Oct 30 18:33:12.596953 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.589934    2574 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Oct 30 18:33:12.596953 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.589937    2574 flags.go:64] FLAG: --max-open-files="1000000"
Oct 30 18:33:12.596953 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.589941    2574 flags.go:64] FLAG: --max-pods="110"
Oct 30 18:33:12.597739 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.589945    2574 flags.go:64] FLAG: --maximum-dead-containers="-1"
Oct 30 18:33:12.597739 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.589948    2574 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Oct 30 18:33:12.597739 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.589951    2574 flags.go:64] FLAG: --memory-manager-policy="None"
Oct 30 18:33:12.597739 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.589954    2574 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Oct 30 18:33:12.597739 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.589956    2574 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Oct 30 18:33:12.597739 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.589959    2574 flags.go:64] FLAG: --node-ip="0.0.0.0"
Oct 30 18:33:12.597739 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.589962    2574 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Oct 30 18:33:12.597739 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.589969    2574 flags.go:64] FLAG: --node-status-max-images="50"
Oct 30 18:33:12.597739 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.589972    2574 flags.go:64] FLAG: --node-status-update-frequency="10s"
Oct 30 18:33:12.597739 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.589975    2574 flags.go:64] FLAG: --oom-score-adj="-999"
Oct 30 18:33:12.597739 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.589978    2574 flags.go:64] FLAG: --pod-cidr=""
Oct 30 18:33:12.597739 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.589981    2574 flags.go:64] FLAG: --pod-manifest-path=""
Oct 30 18:33:12.597739 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.589984    2574 flags.go:64] FLAG: --pod-max-pids="-1"
Oct 30 18:33:12.597739 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.589987    2574 flags.go:64] FLAG: --pods-per-core="0"
Oct 30 18:33:12.597739 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.589989    2574 flags.go:64] FLAG: --port="10250"
Oct 30 18:33:12.597739 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.589994    2574 flags.go:64] FLAG: --protect-kernel-defaults="false"
Oct 30 18:33:12.597739 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.589998    2574 flags.go:64] FLAG: --provider-id="aws:///us-west-1a/i-0459d68fab3e5d86f"
Oct 30 18:33:12.597739 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.590003    2574 flags.go:64] FLAG: --qos-reserved=""
Oct 30 18:33:12.597739 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.590007    2574 flags.go:64] FLAG: --read-only-port="10255"
Oct 30 18:33:12.597739 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.590011    2574 flags.go:64] FLAG: --register-node="true"
Oct 30 18:33:12.597739 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.590016    2574 flags.go:64] FLAG: --register-with-taints=""
Oct 30 18:33:12.597739 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.590023    2574 flags.go:64] FLAG: --registry-burst="10"
Oct 30 18:33:12.597739 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.590028    2574 flags.go:64] FLAG: --registry-qps="5"
Oct 30 18:33:12.597739 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.590032    2574 flags.go:64] FLAG: --reserved-cpus=""
Oct 30 18:33:12.597739 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.590036    2574 flags.go:64] FLAG: --reserved-memory=""
Oct 30 18:33:12.597739 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.590043    2574 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Oct 30 18:33:12.598764 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.590047    2574 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Oct 30 18:33:12.598764 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.590051    2574 flags.go:64] FLAG: --rotate-certificates="false"
Oct 30 18:33:12.598764 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.590055    2574 flags.go:64] FLAG: --rotate-server-certificates="false"
Oct 30 18:33:12.598764 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.590059    2574 flags.go:64] FLAG: --runonce="false"
Oct 30 18:33:12.598764 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.590063    2574 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Oct 30 18:33:12.598764 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.590067    2574 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Oct 30 18:33:12.598764 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.590073    2574 flags.go:64] FLAG: --seccomp-default="false"
Oct 30 18:33:12.598764 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.590077    2574 flags.go:64] FLAG: --serialize-image-pulls="true"
Oct 30 18:33:12.598764 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.590081    2574 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Oct 30 18:33:12.598764 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.590086    2574 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Oct 30 18:33:12.598764 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.590090    2574 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Oct 30 18:33:12.598764 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.590094    2574 flags.go:64] FLAG: --storage-driver-password="root"
Oct 30 18:33:12.598764 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.590098    2574 flags.go:64] FLAG: --storage-driver-secure="false"
Oct 30 18:33:12.598764 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.590102    2574 flags.go:64] FLAG: --storage-driver-table="stats"
Oct 30 18:33:12.598764 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.590106    2574 flags.go:64] FLAG: --storage-driver-user="root"
Oct 30 18:33:12.598764 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.590110    2574 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Oct 30 18:33:12.598764 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.590114    2574 flags.go:64] FLAG: --sync-frequency="1m0s"
Oct 30 18:33:12.598764 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.590119    2574 flags.go:64] FLAG: --system-cgroups=""
Oct 30 18:33:12.598764 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.590122    2574 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Oct 30 18:33:12.598764 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.590131    2574 flags.go:64] FLAG: --system-reserved-cgroup=""
Oct 30 18:33:12.598764 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.590136    2574 flags.go:64] FLAG: --tls-cert-file=""
Oct 30 18:33:12.598764 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.590140    2574 flags.go:64] FLAG: --tls-cipher-suites="[]"
Oct 30 18:33:12.598764 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.590146    2574 flags.go:64] FLAG: --tls-min-version=""
Oct 30 18:33:12.598764 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.590151    2574 flags.go:64] FLAG: --tls-private-key-file=""
Oct 30 18:33:12.598764 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.590154    2574 flags.go:64] FLAG: --topology-manager-policy="none"
Oct 30 18:33:12.599828 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.590157    2574 flags.go:64] FLAG: --topology-manager-policy-options=""
Oct 30 18:33:12.599828 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.590160    2574 flags.go:64] FLAG: --topology-manager-scope="container"
Oct 30 18:33:12.599828 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.590165    2574 flags.go:64] FLAG: --v="2"
Oct 30 18:33:12.599828 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.590170    2574 flags.go:64] FLAG: --version="false"
Oct 30 18:33:12.599828 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.590174    2574 flags.go:64] FLAG: --vmodule=""
Oct 30 18:33:12.599828 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.590178    2574 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Oct 30 18:33:12.599828 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.590181    2574 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Oct 30 18:33:12.599828 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590344    2574 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 18:33:12.599828 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590351    2574 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 18:33:12.599828 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590354    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 18:33:12.599828 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590357    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 18:33:12.599828 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590379    2574 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 18:33:12.599828 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590382    2574 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 18:33:12.599828 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590387    2574 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 18:33:12.599828 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590389    2574 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 18:33:12.599828 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590392    2574 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 18:33:12.599828 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590395    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 18:33:12.599828 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590397    2574 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 18:33:12.599828 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590400    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 18:33:12.599828 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590403    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 18:33:12.599828 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590405    2574 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 18:33:12.600797 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590408    2574 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 18:33:12.600797 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590410    2574 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 18:33:12.600797 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590413    2574 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 18:33:12.600797 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590415    2574 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 18:33:12.600797 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590418    2574 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 18:33:12.600797 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590420    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 18:33:12.600797 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590423    2574 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 18:33:12.600797 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590426    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 18:33:12.600797 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590429    2574 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 18:33:12.600797 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590431    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 18:33:12.600797 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590434    2574 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 18:33:12.600797 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590436    2574 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 18:33:12.600797 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590439    2574 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 18:33:12.600797 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590442    2574 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 18:33:12.600797 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590445    2574 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 18:33:12.600797 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590448    2574 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 18:33:12.600797 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590450    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 18:33:12.600797 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590453    2574 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 18:33:12.600797 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590456    2574 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 18:33:12.600797 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590458    2574 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 18:33:12.601692 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590461    2574 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 18:33:12.601692 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590463    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 18:33:12.601692 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590467    2574 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 18:33:12.601692 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590472    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 18:33:12.601692 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590475    2574 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 18:33:12.601692 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590478    2574 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 18:33:12.601692 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590481    2574 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 18:33:12.601692 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590485    2574 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 18:33:12.601692 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590487    2574 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 18:33:12.601692 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590490    2574 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 18:33:12.601692 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590502    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 18:33:12.601692 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590505    2574 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 18:33:12.601692 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590508    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 18:33:12.601692 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590510    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 18:33:12.601692 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590513    2574 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 18:33:12.601692 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590516    2574 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 18:33:12.601692 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590518    2574 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 18:33:12.601692 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590521    2574 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 18:33:12.601692 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590523    2574 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 18:33:12.602398 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590525    2574 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 18:33:12.602398 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590528    2574 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 18:33:12.602398 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590530    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 18:33:12.602398 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590533    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 18:33:12.602398 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590535    2574 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 18:33:12.602398 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590538    2574 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 18:33:12.602398 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590541    2574 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 18:33:12.602398 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590544    2574 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 18:33:12.602398 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590546    2574 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 18:33:12.602398 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590549    2574 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 18:33:12.602398 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590552    2574 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 18:33:12.602398 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590554    2574 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 18:33:12.602398 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590558    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 18:33:12.602398 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590561    2574 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 18:33:12.602398 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590563    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 18:33:12.602398 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590566    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 18:33:12.602398 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590568    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 18:33:12.602398 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590571    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 18:33:12.602398 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590574    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 18:33:12.602398 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590576    2574 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 18:33:12.603207 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590579    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 18:33:12.603207 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590582    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 18:33:12.603207 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590584    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 18:33:12.603207 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590588    2574 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 18:33:12.603207 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590591    2574 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 18:33:12.603207 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590594    2574 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 18:33:12.603207 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590596    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 18:33:12.603207 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590599    2574 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 18:33:12.603207 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590602    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 18:33:12.603207 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590604    2574 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 18:33:12.603207 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590607    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 18:33:12.603207 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590609    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 18:33:12.603207 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590612    2574 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 18:33:12.603207 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590614    2574 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 18:33:12.603207 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590617    2574 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 18:33:12.603207 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590619    2574 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 18:33:12.603207 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590622    2574 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 18:33:12.603207 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590624    2574 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 18:33:12.603207 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590629    2574 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 18:33:12.604006 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590631    2574 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 18:33:12.604006 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590634    2574 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 18:33:12.604006 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590636    2574 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 18:33:12.604006 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590639    2574 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 18:33:12.604006 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590641    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 18:33:12.604006 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590644    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 18:33:12.604006 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590647    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 18:33:12.604006 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590649    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 18:33:12.604006 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590652    2574 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 18:33:12.604006 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590654    2574 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 18:33:12.604006 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590657    2574 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 18:33:12.604006 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590659    2574 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 18:33:12.604006 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590662    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 18:33:12.604006 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590664    2574 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 18:33:12.604006 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.590694    2574 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 18:33:12.604006 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590736    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 18:33:12.604616 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590740    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 18:33:12.604616 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590744    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 18:33:12.604616 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590746    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 18:33:12.604616 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590749    2574 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 18:33:12.604616 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590751    2574 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 18:33:12.604616 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590754    2574 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 18:33:12.604616 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590756    2574 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 18:33:12.604616 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590759    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 18:33:12.604616 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590761    2574 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 18:33:12.604616 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590764    2574 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 18:33:12.604616 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590766    2574 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 18:33:12.604616 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590769    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 18:33:12.604616 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590772    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 18:33:12.604616 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590774    2574 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 18:33:12.604616 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590776    2574 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 18:33:12.604616 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590779    2574 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 18:33:12.604616 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590782    2574 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 18:33:12.604616 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590785    2574 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 18:33:12.604616 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590787    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 18:33:12.604616 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590790    2574 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 18:33:12.605175 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590793    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 18:33:12.605175 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590795    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 18:33:12.605175 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590798    2574 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 18:33:12.605175 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590801    2574 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 18:33:12.605175 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590804    2574 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 18:33:12.605175 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590806    2574 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 18:33:12.605175 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590809    2574 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 18:33:12.605175 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590812    2574 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 18:33:12.605175 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590814    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 18:33:12.605175 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590817    2574 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 18:33:12.605175 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590819    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 18:33:12.605175 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590822    2574 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 18:33:12.605175 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590825    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 18:33:12.605175 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590827    2574 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 18:33:12.605175 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590830    2574 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 18:33:12.605175 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590832    2574 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 18:33:12.605175 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590835    2574 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 18:33:12.605175 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590837    2574 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 18:33:12.605175 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590840    2574 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 18:33:12.605175 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590842    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 18:33:12.605722 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590844    2574 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 18:33:12.605722 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590847    2574 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 18:33:12.605722 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590850    2574 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 18:33:12.605722 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590852    2574 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 18:33:12.605722 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590855    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 18:33:12.605722 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590858    2574 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 18:33:12.605722 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590862    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 18:33:12.605722 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590865    2574 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 18:33:12.605722 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590869    2574 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 18:33:12.605722 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590872    2574 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 18:33:12.605722 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590876    2574 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 18:33:12.605722 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590879    2574 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 18:33:12.605722 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590881    2574 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 18:33:12.605722 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590885    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 18:33:12.605722 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590888    2574 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 18:33:12.605722 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590891    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 18:33:12.605722 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590894    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 18:33:12.605722 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590896    2574 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 18:33:12.605722 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590899    2574 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 18:33:12.606178 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590901    2574 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 18:33:12.606178 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590903    2574 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 18:33:12.606178 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590906    2574 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 18:33:12.606178 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590908    2574 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 18:33:12.606178 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590911    2574 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 18:33:12.606178 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590913    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 18:33:12.606178 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590916    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 18:33:12.606178 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590918    2574 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 18:33:12.606178 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590921    2574 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 18:33:12.606178 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590923    2574 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 18:33:12.606178 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590926    2574 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 18:33:12.606178 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590928    2574 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 18:33:12.606178 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590930    2574 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 18:33:12.606178 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590933    2574 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 18:33:12.606178 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590935    2574 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 18:33:12.606178 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590938    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 18:33:12.606178 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590941    2574 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 18:33:12.606178 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590943    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 18:33:12.606178 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590946    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 18:33:12.606178 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590948    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 18:33:12.606744 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590951    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 18:33:12.606744 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590954    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 18:33:12.606744 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590956    2574 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 18:33:12.606744 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590960    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 18:33:12.606744 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590963    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 18:33:12.606744 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590965    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 18:33:12.606744 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590969    2574 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 18:33:12.606744 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590972    2574 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 18:33:12.606744 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590975    2574 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 18:33:12.606744 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590977    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 18:33:12.606744 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590980    2574 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 18:33:12.606744 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590983    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 18:33:12.606744 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590985    2574 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 18:33:12.606744 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590988    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 18:33:12.606744 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590990    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 18:33:12.606744 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590993    2574 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 18:33:12.606744 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590995    2574 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 18:33:12.606744 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.590997    2574 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 18:33:12.606744 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.591000    2574 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 18:33:12.607270 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.591002    2574 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 18:33:12.607270 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.591006    2574 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 18:33:12.607270 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.591008    2574 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 18:33:12.607270 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.591010    2574 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 18:33:12.607270 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.591013    2574 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 18:33:12.607270 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.591015    2574 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 18:33:12.607270 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.591018    2574 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 18:33:12.607270 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.599968    2574 server.go:525] "Kubelet version" kubeletVersion="v1.35.9"
Oct 30 18:33:12.607270 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.599991    2574 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Oct 30 18:33:12.607270 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.600020    2574 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Oct 30 18:33:12.607270 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.600028    2574 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Oct 30 18:33:12.607270 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.600115    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 18:33:12.607270 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.600124    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 18:33:12.607270 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.600129    2574 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 18:33:12.607270 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.600134    2574 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 18:33:12.607270 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.600138    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 18:33:12.607270 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.600142    2574 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 18:33:12.607270 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.600147    2574 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 18:33:12.607270 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.600151    2574 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 18:33:12.608051 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.600155    2574 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 18:33:12.608051 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.600160    2574 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 18:33:12.608051 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.600164    2574 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 18:33:12.608051 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.600169    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 18:33:12.608051 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.600173    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 18:33:12.608051 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.600177    2574 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 18:33:12.608051 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.600182    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 18:33:12.608051 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.600186    2574 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 18:33:12.608051 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.600191    2574 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 18:33:12.608051 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.600195    2574 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 18:33:12.608051 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.600199    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 18:33:12.608051 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.600203    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 18:33:12.608051 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.600209    2574 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 18:33:12.608051 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.600213    2574 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 18:33:12.608051 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.600217    2574 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 18:33:12.608051 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.600222    2574 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 18:33:12.608051 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.600226    2574 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 18:33:12.608051 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.600230    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 18:33:12.608051 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.600234    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 18:33:12.608772 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.600239    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 18:33:12.608772 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.600245    2574 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 18:33:12.608772 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.600249    2574 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 18:33:12.608772 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.600256    2574 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 18:33:12.608772 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.600262    2574 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 18:33:12.608772 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.600267    2574 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 18:33:12.608772 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.600272    2574 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 18:33:12.608772 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.600276    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 18:33:12.608772 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.600280    2574 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 18:33:12.608772 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.600284    2574 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 18:33:12.608772 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.600289    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 18:33:12.608772 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.600294    2574 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 18:33:12.608772 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.600298    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 18:33:12.608772 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.600303    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 18:33:12.608772 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.600307    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 18:33:12.608772 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.600311    2574 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 18:33:12.608772 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.600315    2574 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 18:33:12.608772 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.600319    2574 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 18:33:12.608772 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.600324    2574 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 18:33:12.608772 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.600328    2574 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 18:33:12.609256 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.600333    2574 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 18:33:12.609256 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.600338    2574 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 18:33:12.609256 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.600342    2574 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 18:33:12.609256 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.600346    2574 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 18:33:12.609256 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.600351    2574 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 18:33:12.609256 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.600355    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 18:33:12.609256 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.600383    2574 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 18:33:12.609256 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.600388    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 18:33:12.609256 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.600393    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 18:33:12.609256 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.600397    2574 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 18:33:12.609256 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.600402    2574 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 18:33:12.609256 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.600406    2574 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 18:33:12.609256 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.600410    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 18:33:12.609256 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.600415    2574 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 18:33:12.609256 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.600420    2574 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 18:33:12.609256 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.600424    2574 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 18:33:12.609256 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.600429    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 18:33:12.609256 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.600433    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 18:33:12.609256 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.600437    2574 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 18:33:12.609766 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.600441    2574 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 18:33:12.609766 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.600445    2574 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 18:33:12.609766 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.600450    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 18:33:12.609766 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.600454    2574 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 18:33:12.609766 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.600458    2574 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 18:33:12.609766 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.600463    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 18:33:12.609766 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.600468    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 18:33:12.609766 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.600472    2574 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 18:33:12.609766 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.600477    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 18:33:12.609766 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.600481    2574 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 18:33:12.609766 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.600485    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 18:33:12.609766 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.600490    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 18:33:12.609766 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.600494    2574 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 18:33:12.609766 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.600499    2574 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 18:33:12.609766 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.600504    2574 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 18:33:12.609766 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.600508    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 18:33:12.609766 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.600512    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 18:33:12.609766 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.600516    2574 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 18:33:12.609766 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.600521    2574 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 18:33:12.610240 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.600527    2574 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 18:33:12.610240 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.600533    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 18:33:12.610240 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.600538    2574 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 18:33:12.610240 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.600543    2574 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 18:33:12.610240 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.600547    2574 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 18:33:12.610240 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.600552    2574 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 18:33:12.610240 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.600556    2574 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 18:33:12.610240 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.600560    2574 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 18:33:12.610240 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.600564    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 18:33:12.610240 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.600569    2574 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 18:33:12.610240 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.600573    2574 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 18:33:12.610240 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.600577    2574 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 18:33:12.610240 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.600582    2574 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 18:33:12.610240 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.600585    2574 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 18:33:12.610240 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.600590    2574 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 18:33:12.610240 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.600594    2574 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 18:33:12.610240 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.600598    2574 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 18:33:12.610240 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.600602    2574 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 18:33:12.610240 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.600607    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 18:33:12.610717 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.600612    2574 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 18:33:12.610717 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.600616    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 18:33:12.610717 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.600657    2574 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 18:33:12.610717 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.600905    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 18:33:12.610717 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.600915    2574 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 18:33:12.610717 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.600920    2574 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 18:33:12.610717 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.600926    2574 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 18:33:12.610717 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.600932    2574 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 18:33:12.610717 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.600938    2574 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 18:33:12.610717 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.600942    2574 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 18:33:12.610717 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.600947    2574 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 18:33:12.610717 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.600950    2574 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 18:33:12.610717 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.600954    2574 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 18:33:12.610717 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.600959    2574 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 18:33:12.610717 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.600963    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 18:33:12.610717 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.600968    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 18:33:12.611288 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.600972    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 18:33:12.611288 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.600976    2574 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 18:33:12.611288 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.600980    2574 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 18:33:12.611288 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.600985    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 18:33:12.611288 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.600989    2574 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 18:33:12.611288 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.600993    2574 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 18:33:12.611288 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.600997    2574 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 18:33:12.611288 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601002    2574 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 18:33:12.611288 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601006    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 18:33:12.611288 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601010    2574 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 18:33:12.611288 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601017    2574 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 18:33:12.611288 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601023    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 18:33:12.611288 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601028    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 18:33:12.611288 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601032    2574 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 18:33:12.611288 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601036    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 18:33:12.611288 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601041    2574 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 18:33:12.611288 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601045    2574 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 18:33:12.611288 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601050    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 18:33:12.611288 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601054    2574 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 18:33:12.611880 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601058    2574 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 18:33:12.611880 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601062    2574 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 18:33:12.611880 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601066    2574 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 18:33:12.611880 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601070    2574 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 18:33:12.611880 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601074    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 18:33:12.611880 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601079    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 18:33:12.611880 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601084    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 18:33:12.611880 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601089    2574 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 18:33:12.611880 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601093    2574 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 18:33:12.611880 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601097    2574 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 18:33:12.611880 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601101    2574 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 18:33:12.611880 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601105    2574 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 18:33:12.611880 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601110    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 18:33:12.611880 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601114    2574 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 18:33:12.611880 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601118    2574 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 18:33:12.611880 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601122    2574 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 18:33:12.611880 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601126    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 18:33:12.611880 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601131    2574 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 18:33:12.611880 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601135    2574 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 18:33:12.611880 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601139    2574 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 18:33:12.612520 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601143    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 18:33:12.612520 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601147    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 18:33:12.612520 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601151    2574 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 18:33:12.612520 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601155    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 18:33:12.612520 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601159    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 18:33:12.612520 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601164    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 18:33:12.612520 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601168    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 18:33:12.612520 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601173    2574 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 18:33:12.612520 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601177    2574 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 18:33:12.612520 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601181    2574 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 18:33:12.612520 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601185    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 18:33:12.612520 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601189    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 18:33:12.612520 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601193    2574 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 18:33:12.612520 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601198    2574 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 18:33:12.612520 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601201    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 18:33:12.612520 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601206    2574 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 18:33:12.612520 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601211    2574 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 18:33:12.612520 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601215    2574 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 18:33:12.612520 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601219    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 18:33:12.612975 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601224    2574 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 18:33:12.612975 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601228    2574 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 18:33:12.612975 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601232    2574 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 18:33:12.612975 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601236    2574 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 18:33:12.612975 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601240    2574 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 18:33:12.612975 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601245    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 18:33:12.612975 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601249    2574 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 18:33:12.612975 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601253    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 18:33:12.612975 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601257    2574 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 18:33:12.612975 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601261    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 18:33:12.612975 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601265    2574 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 18:33:12.612975 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601270    2574 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 18:33:12.612975 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601274    2574 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 18:33:12.612975 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601278    2574 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 18:33:12.612975 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601282    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 18:33:12.612975 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601286    2574 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 18:33:12.612975 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601290    2574 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 18:33:12.612975 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601294    2574 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 18:33:12.612975 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601299    2574 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 18:33:12.613460 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601303    2574 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 18:33:12.613460 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601307    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 18:33:12.613460 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601311    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 18:33:12.613460 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601316    2574 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 18:33:12.613460 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601320    2574 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 18:33:12.613460 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601324    2574 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 18:33:12.613460 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601328    2574 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 18:33:12.613460 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601332    2574 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 18:33:12.613460 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601337    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 18:33:12.613460 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601341    2574 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 18:33:12.613460 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601345    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 18:33:12.613460 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601349    2574 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 18:33:12.613460 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601353    2574 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 18:33:12.613460 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601378    2574 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 18:33:12.613460 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601386    2574 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 18:33:12.613460 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601392    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 18:33:12.613833 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.601434    2574 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 18:33:12.613833 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601493    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 18:33:12.613833 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601499    2574 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 18:33:12.613833 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601503    2574 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 18:33:12.613833 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601507    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 18:33:12.613833 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601512    2574 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 18:33:12.613833 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601516    2574 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 18:33:12.613833 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601520    2574 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 18:33:12.613833 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601524    2574 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 18:33:12.613833 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601529    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 18:33:12.613833 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601533    2574 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 18:33:12.613833 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601538    2574 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 18:33:12.613833 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601544    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 18:33:12.613833 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601548    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 18:33:12.613833 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601552    2574 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 18:33:12.613833 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601556    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 18:33:12.614205 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601560    2574 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 18:33:12.614205 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601565    2574 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 18:33:12.614205 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601569    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 18:33:12.614205 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601573    2574 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 18:33:12.614205 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601577    2574 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 18:33:12.614205 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601581    2574 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 18:33:12.614205 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601585    2574 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 18:33:12.614205 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601589    2574 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 18:33:12.614205 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601594    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 18:33:12.614205 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601598    2574 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 18:33:12.614205 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601603    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 18:33:12.614205 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601607    2574 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 18:33:12.614205 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601611    2574 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 18:33:12.614205 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601616    2574 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 18:33:12.614205 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601620    2574 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 18:33:12.614205 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601625    2574 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 18:33:12.614205 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601629    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 18:33:12.614205 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601633    2574 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 18:33:12.614205 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601637    2574 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 18:33:12.615116 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601642    2574 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 18:33:12.615116 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601646    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 18:33:12.615116 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601650    2574 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 18:33:12.615116 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601655    2574 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 18:33:12.615116 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601659    2574 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 18:33:12.615116 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601663    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 18:33:12.615116 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601667    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 18:33:12.615116 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601672    2574 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 18:33:12.615116 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601676    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 18:33:12.615116 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601680    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 18:33:12.615116 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601684    2574 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 18:33:12.615116 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601688    2574 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 18:33:12.615116 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601692    2574 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 18:33:12.615116 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601696    2574 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 18:33:12.615116 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601700    2574 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 18:33:12.615116 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601704    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 18:33:12.615116 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601708    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 18:33:12.615116 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601712    2574 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 18:33:12.615116 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601719    2574 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 18:33:12.615575 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601723    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 18:33:12.615575 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601727    2574 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 18:33:12.615575 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601732    2574 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 18:33:12.615575 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601736    2574 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 18:33:12.615575 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601740    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 18:33:12.615575 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601744    2574 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 18:33:12.615575 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601748    2574 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 18:33:12.615575 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601752    2574 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 18:33:12.615575 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601756    2574 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 18:33:12.615575 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601761    2574 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 18:33:12.615575 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601766    2574 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 18:33:12.615575 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601771    2574 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 18:33:12.615575 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601775    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 18:33:12.615575 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601780    2574 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 18:33:12.615575 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601785    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 18:33:12.615575 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601789    2574 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 18:33:12.615575 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601793    2574 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 18:33:12.615575 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601797    2574 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 18:33:12.615575 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601801    2574 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 18:33:12.616096 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601805    2574 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 18:33:12.616096 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601810    2574 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 18:33:12.616096 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601814    2574 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 18:33:12.616096 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601819    2574 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 18:33:12.616096 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601823    2574 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 18:33:12.616096 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601827    2574 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 18:33:12.616096 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601831    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 18:33:12.616096 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601835    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 18:33:12.616096 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601839    2574 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 18:33:12.616096 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601843    2574 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 18:33:12.616096 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601847    2574 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 18:33:12.616096 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601852    2574 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 18:33:12.616096 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601858    2574 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 18:33:12.616096 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601862    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 18:33:12.616096 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601866    2574 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 18:33:12.616096 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601870    2574 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 18:33:12.616096 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601874    2574 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 18:33:12.616096 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601878    2574 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 18:33:12.616096 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601885    2574 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 18:33:12.616566 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601891    2574 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 18:33:12.616566 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601896    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 18:33:12.616566 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601901    2574 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 18:33:12.616566 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601905    2574 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 18:33:12.616566 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601910    2574 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 18:33:12.616566 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601915    2574 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 18:33:12.616566 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601919    2574 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 18:33:12.616566 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601924    2574 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 18:33:12.616566 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601928    2574 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 18:33:12.616566 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601934    2574 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 18:33:12.616566 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601938    2574 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 18:33:12.616566 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601942    2574 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 18:33:12.616566 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601947    2574 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 18:33:12.616566 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601951    2574 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 18:33:12.616566 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:12.601955    2574 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 18:33:12.616566 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.602037    2574 server.go:957] "Client rotation is on, will bootstrap in background"
Oct 30 18:33:12.616566 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.604576    2574 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Oct 30 18:33:12.616566 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.605506    2574 server.go:1014] "Starting client certificate rotation"
Oct 30 18:33:12.616566 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.605598    2574 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Oct 30 18:33:12.617017 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.606337    2574 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Oct 30 18:33:12.617017 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.607741    2574 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 18:33:12.617017 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.612394    2574 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 18:33:12.625199 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.625184    2574 log.go:25] "Validated CRI v1 runtime API"
Oct 30 18:33:12.628659 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.628637    2574 log.go:25] "Validated CRI v1 image API"
Oct 30 18:33:12.629456 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.629433    2574 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Oct 30 18:33:12.631928 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.631905    2574 fs.go:135] Filesystem UUIDs: map[2c0c17c3-049f-444e-8104-bc73b9e8fb70:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 9f3017e7-a4ab-4bd0-9554-505960a37d8e:/dev/nvme0n1p3]
Oct 30 18:33:12.631999 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.631928    2574 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0}]
Oct 30 18:33:12.638540 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.638431    2574 manager.go:217] Machine: {Timestamp:2026-10-30 18:33:12.636924373 +0000 UTC m=+0.324278991 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3106872 MemoryCapacity:32812011520 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec209f7cf0c716387e6716ee829baada SystemUUID:ec209f7c-f0c7-1638-7e67-16ee829baada BootID:d8d2fdad-954d-46d6-a103-8aed2454c617 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16406003712 Type:vfs Inodes:4005372 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6562402304 Type:vfs Inodes:819200 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16406007808 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:2f:7f:1b:a1:51 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:2f:7f:1b:a1:51 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:ce:cf:b9:73:19:f0 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812011520 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Oct 30 18:33:12.639054 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.639044    2574 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Oct 30 18:33:12.639138 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.639127    2574 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Oct 30 18:33:12.641201 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.640639    2574 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Oct 30 18:33:12.641340 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.641205    2574 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-134-46.us-west-1.compute.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Oct 30 18:33:12.641412 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.641347    2574 topology_manager.go:143] "Creating topology manager with none policy"
Oct 30 18:33:12.641412 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.641359    2574 container_manager_linux.go:311] "Creating device plugin manager"
Oct 30 18:33:12.641412 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.641388    2574 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 30 18:33:12.641412 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.641399    2574 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 30 18:33:12.642188 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.642177    2574 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager"
Oct 30 18:33:12.645814 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.645802    2574 state_mem.go:41] "Initialized" logger="CPUManager state memory"
Oct 30 18:33:12.645938 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.645929    2574 server.go:1235] "Using root directory" path="/var/lib/kubelet"
Oct 30 18:33:12.646626 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.646615    2574 kubelet.go:493] "Attempting to sync node with API server"
Oct 30 18:33:12.646657 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.646632    2574 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests"
Oct 30 18:33:12.646657 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.646644    2574 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Oct 30 18:33:12.646657 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.646652    2574 kubelet.go:405] "Adding apiserver pod source"
Oct 30 18:33:12.646756 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.646660    2574 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Oct 30 18:33:12.647010 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.646999    2574 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 30 18:33:12.649132 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.649115    2574 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1"
Oct 30 18:33:12.653829 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.653816    2574 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 30 18:33:12.654575 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.654562    2574 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Oct 30 18:33:12.654612 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.654593    2574 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Oct 30 18:33:12.654751 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.654741    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Oct 30 18:33:12.654796 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.654755    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Oct 30 18:33:12.654796 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.654762    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Oct 30 18:33:12.654796 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.654768    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Oct 30 18:33:12.654796 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.654774    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Oct 30 18:33:12.654796 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.654779    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Oct 30 18:33:12.655374 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.655353    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Oct 30 18:33:12.655403 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.655383    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Oct 30 18:33:12.655403 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.655390    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Oct 30 18:33:12.655403 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.655396    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Oct 30 18:33:12.655484 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.655404    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Oct 30 18:33:12.655484 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.655413    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Oct 30 18:33:12.656392 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.656383    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Oct 30 18:33:12.656392 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.656392    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Oct 30 18:33:12.659885 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.659872    2574 server.go:1263] "Started kubelet"
Oct 30 18:33:12.660045 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.659956    2574 server.go:182] "Starting to listen" address="0.0.0.0" port=10250
Oct 30 18:33:12.660138 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.660054    2574 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Oct 30 18:33:12.660183 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.660154    2574 server_v1.go:49] "podresources" method="list" useActivePods=true
Oct 30 18:33:12.661196 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.661163    2574 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Oct 30 18:33:12.661556 ip-10-0-134-46 systemd[1]: Started Kubernetes Kubelet.
Oct 30 18:33:12.662639 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.662624    2574 server.go:317] "Adding debug handlers to kubelet server"
Oct 30 18:33:12.665846 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:12.665800    2574 kubelet.go:1679] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Oct 30 18:33:12.666300 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.666287    2574 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Oct 30 18:33:12.667330 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.667316    2574 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer"
Oct 30 18:33:12.667442 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:12.667427    2574 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-134-46.us-west-1.compute.internal\" not found"
Oct 30 18:33:12.667971 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.667956    2574 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Oct 30 18:33:12.668051 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.667958    2574 volume_manager.go:309] "The desired_state_of_world populator starts"
Oct 30 18:33:12.668051 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.668036    2574 volume_manager.go:311] "Starting Kubelet Volume Manager"
Oct 30 18:33:12.668336 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.668167    2574 reconstruct.go:97] "Volume reconstruction finished"
Oct 30 18:33:12.668336 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.668176    2574 reconciler.go:29] "Reconciler: start to sync state"
Oct 30 18:33:12.668336 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.668321    2574 factory.go:55] Registering systemd factory
Oct 30 18:33:12.668490 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.668341    2574 factory.go:223] Registration of the systemd container factory successfully
Oct 30 18:33:12.668623 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.668609    2574 factory.go:153] Registering CRI-O factory
Oct 30 18:33:12.668623 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.668623    2574 factory.go:223] Registration of the crio container factory successfully
Oct 30 18:33:12.668724 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.668661    2574 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Oct 30 18:33:12.668724 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.668673    2574 factory.go:103] Registering Raw factory
Oct 30 18:33:12.668724 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.668686    2574 manager.go:1199] Started watching for new ooms in manager
Oct 30 18:33:12.669620 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.669607    2574 manager.go:319] Starting recovery of all containers
Oct 30 18:33:12.681739 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.681719    2574 manager.go:324] Recovery completed
Oct 30 18:33:12.685506 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.685492    2574 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 18:33:12.687038 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.687011    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-46.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Oct 30 18:33:12.687112 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.687052    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-46.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 18:33:12.687112 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.687062    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-46.us-west-1.compute.internal" event="NodeHasSufficientPID"
Oct 30 18:33:12.687563 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.687551    2574 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 18:33:12.687614 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.687557    2574 cpu_manager.go:226] "Starting" policy="none"
Oct 30 18:33:12.687663 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.687621    2574 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s"
Oct 30 18:33:12.687663 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.687641    2574 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory"
Oct 30 18:33:12.688222 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.688207    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-46.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Oct 30 18:33:12.688327 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.688231    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-46.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 18:33:12.688327 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.688242    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-46.us-west-1.compute.internal" event="NodeHasSufficientPID"
Oct 30 18:33:12.689782 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.689770    2574 policy_none.go:50] "Start"
Oct 30 18:33:12.690199 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.690191    2574 memory_manager.go:187] "Starting memorymanager" policy="None"
Oct 30 18:33:12.690228 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.690204    2574 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Oct 30 18:33:12.692289 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.692279    2574 policy_none.go:44] "Start"
Oct 30 18:33:12.753141 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.732434    2574 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"}
Oct 30 18:33:12.753141 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.740610    2574 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Oct 30 18:33:12.753141 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.741915    2574 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Oct 30 18:33:12.753141 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.742334    2574 status_manager.go:249] "Starting to sync pod status with apiserver"
Oct 30 18:33:12.753141 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.742375    2574 kubelet.go:2521] "Starting kubelet main sync loop"
Oct 30 18:33:12.753141 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:12.742411    2574 kubelet.go:2545] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]"
Oct 30 18:33:12.753141 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.745716    2574 manager.go:341] "Starting Device Plugin manager"
Oct 30 18:33:12.753141 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:12.746139    2574 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Oct 30 18:33:12.753141 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.746150    2574 server.go:87] "Starting device plugin registration server"
Oct 30 18:33:12.753141 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.746334    2574 eviction_manager.go:194] "Eviction manager: starting control loop"
Oct 30 18:33:12.753141 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.746345    2574 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Oct 30 18:33:12.753141 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.746442    2574 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Oct 30 18:33:12.753141 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.746519    2574 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts"
Oct 30 18:33:12.753141 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.746527    2574 plugin_manager.go:121] "Starting Kubelet Plugin Manager"
Oct 30 18:33:12.753141 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:12.746956    2574 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Oct 30 18:33:12.753141 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:12.747002    2574 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-134-46.us-west-1.compute.internal\" not found"
Oct 30 18:33:12.807845 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:12.807822    2574 controller.go:201] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-134-46.us-west-1.compute.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Oct 30 18:33:12.808801 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:12.807814    2574 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-134-46.us-west-1.compute.internal.18e36254b9ef7a19  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-46.us-west-1.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-134-46.us-west-1.compute.internal,},FirstTimestamp:2026-10-30 18:33:12.659843609 +0000 UTC m=+0.347198226,LastTimestamp:2026-10-30 18:33:12.659843609 +0000 UTC m=+0.347198226,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-46.us-west-1.compute.internal,}"
Oct 30 18:33:12.843469 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.843419    2574 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-134-46.us-west-1.compute.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-46.us-west-1.compute.internal"]
Oct 30 18:33:12.843557 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.843470    2574 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 18:33:12.844211 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.844199    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-46.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Oct 30 18:33:12.844256 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.844223    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-46.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 18:33:12.844256 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.844233    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-46.us-west-1.compute.internal" event="NodeHasSufficientPID"
Oct 30 18:33:12.846084 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.846068    2574 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 18:33:12.846166 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.846150    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-46.us-west-1.compute.internal"
Oct 30 18:33:12.846220 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.846182    2574 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 18:33:12.846581 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.846565    2574 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 18:33:12.846932 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.846916    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-46.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Oct 30 18:33:12.847000 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.846942    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-46.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 18:33:12.847000 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.846952    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-46.us-west-1.compute.internal" event="NodeHasSufficientPID"
Oct 30 18:33:12.847000 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.846967    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-46.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Oct 30 18:33:12.847000 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.846986    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-46.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 18:33:12.847000 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.846998    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-46.us-west-1.compute.internal" event="NodeHasSufficientPID"
Oct 30 18:33:12.847212 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.847198    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-46.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Oct 30 18:33:12.847249 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.847228    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-46.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 18:33:12.847249 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.847240    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-46.us-west-1.compute.internal" event="NodeHasSufficientPID"
Oct 30 18:33:12.847314 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.847265    2574 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-134-46.us-west-1.compute.internal"
Oct 30 18:33:12.848771 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.848756    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-46.us-west-1.compute.internal"
Oct 30 18:33:12.848828 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.848783    2574 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 18:33:12.849406 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.849393    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-46.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Oct 30 18:33:12.849491 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.849419    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-46.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 18:33:12.849491 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.849431    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-46.us-west-1.compute.internal" event="NodeHasSufficientPID"
Oct 30 18:33:12.856148 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.856132    2574 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-134-46.us-west-1.compute.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Oct 30 18:33:12.878701 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:12.878685    2574 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-134-46.us-west-1.compute.internal\" not found" node="ip-10-0-134-46.us-west-1.compute.internal"
Oct 30 18:33:12.885005 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:12.884989    2574 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-134-46.us-west-1.compute.internal\" not found" node="ip-10-0-134-46.us-west-1.compute.internal"
Oct 30 18:33:12.894464 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.894446    2574 reflector.go:446] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Oct 30 18:33:12.894540 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.894483    2574 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-s5f5k"
Oct 30 18:33:12.903637 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:12.903613    2574 reflector.go:204] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Service"
Oct 30 18:33:12.903637 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:12.903622    2574 reflector.go:204] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-134-46.us-west-1.compute.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Node"
Oct 30 18:33:12.903793 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:12.903655    2574 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-134-46.us-west-1.compute.internal.18e36254bb8e6ed1  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-46.us-west-1.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-134-46.us-west-1.compute.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-134-46.us-west-1.compute.internal,},FirstTimestamp:2026-10-30 18:33:12.687038161 +0000 UTC m=+0.374392778,LastTimestamp:2026-10-30 18:33:12.687038161 +0000 UTC m=+0.374392778,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-46.us-west-1.compute.internal,}"
Oct 30 18:33:12.966428 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.966412    2574 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 18:33:12.966519 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.966444    2574 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 18:33:12.966765 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:12.966753    2574 kubelet_node_status.go:115] "Unable to register node with API server, error getting existing node" err="nodes \"ip-10-0-134-46.us-west-1.compute.internal\" not found" node="ip-10-0-134-46.us-west-1.compute.internal"
Oct 30 18:33:12.969754 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.969731    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6f66fae2621414751921d90f1913a614-config\") pod \"kube-apiserver-proxy-ip-10-0-134-46.us-west-1.compute.internal\" (UID: \"6f66fae2621414751921d90f1913a614\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-46.us-west-1.compute.internal"
Oct 30 18:33:12.969798 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.969763    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/563184d807a5bf00cb82141d4921b071-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-46.us-west-1.compute.internal\" (UID: \"563184d807a5bf00cb82141d4921b071\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-46.us-west-1.compute.internal"
Oct 30 18:33:12.969798 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:12.969780    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/563184d807a5bf00cb82141d4921b071-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-46.us-west-1.compute.internal\" (UID: \"563184d807a5bf00cb82141d4921b071\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-46.us-west-1.compute.internal"
Oct 30 18:33:13.069998 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:13.069968    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6f66fae2621414751921d90f1913a614-config\") pod \"kube-apiserver-proxy-ip-10-0-134-46.us-west-1.compute.internal\" (UID: \"6f66fae2621414751921d90f1913a614\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-46.us-west-1.compute.internal"
Oct 30 18:33:13.070103 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:13.070003    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/563184d807a5bf00cb82141d4921b071-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-46.us-west-1.compute.internal\" (UID: \"563184d807a5bf00cb82141d4921b071\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-46.us-west-1.compute.internal"
Oct 30 18:33:13.070103 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:13.070028    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/563184d807a5bf00cb82141d4921b071-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-46.us-west-1.compute.internal\" (UID: \"563184d807a5bf00cb82141d4921b071\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-46.us-west-1.compute.internal"
Oct 30 18:33:13.070103 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:13.070081    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/563184d807a5bf00cb82141d4921b071-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-46.us-west-1.compute.internal\" (UID: \"563184d807a5bf00cb82141d4921b071\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-46.us-west-1.compute.internal"
Oct 30 18:33:13.070207 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:13.070081    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/563184d807a5bf00cb82141d4921b071-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-46.us-west-1.compute.internal\" (UID: \"563184d807a5bf00cb82141d4921b071\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-46.us-west-1.compute.internal"
Oct 30 18:33:13.070207 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:13.070081    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6f66fae2621414751921d90f1913a614-config\") pod \"kube-apiserver-proxy-ip-10-0-134-46.us-west-1.compute.internal\" (UID: \"6f66fae2621414751921d90f1913a614\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-46.us-west-1.compute.internal"
Oct 30 18:33:13.112254 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:13.112214    2574 nodelease.go:50] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-134-46.us-west-1.compute.internal\" not found" node="ip-10-0-134-46.us-west-1.compute.internal"
Oct 30 18:33:13.167122 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:13.167105    2574 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 18:33:13.167907 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:13.167893    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-46.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Oct 30 18:33:13.167973 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:13.167919    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-46.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 18:33:13.167973 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:13.167930    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-46.us-west-1.compute.internal" event="NodeHasSufficientPID"
Oct 30 18:33:13.167973 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:13.167951    2574 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-134-46.us-west-1.compute.internal"
Oct 30 18:33:13.181297 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:13.181275    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-46.us-west-1.compute.internal"
Oct 30 18:33:13.188693 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:13.188678    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-46.us-west-1.compute.internal"
Oct 30 18:33:13.223763 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:13.223742    2574 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-134-46.us-west-1.compute.internal"
Oct 30 18:33:13.223763 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:13.223762    2574 kubelet_node_status.go:524] "Error updating node status, will retry" err="error getting node \"ip-10-0-134-46.us-west-1.compute.internal\": node \"ip-10-0-134-46.us-west-1.compute.internal\" not found"
Oct 30 18:33:13.333775 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:13.333749    2574 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-134-46.us-west-1.compute.internal\" not found"
Oct 30 18:33:13.434548 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:13.434457    2574 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-134-46.us-west-1.compute.internal\" not found"
Oct 30 18:33:13.534686 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:13.534631    2574 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-134-46.us-west-1.compute.internal\" not found"
Oct 30 18:33:13.605894 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:13.605866    2574 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Oct 30 18:33:13.606334 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:13.606043    2574 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received"
Oct 30 18:33:13.606334 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:13.606055    2574 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received"
Oct 30 18:33:13.606334 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:13.606022    2574 event.go:368] "Unable to write event (may retry after sleeping)" err="Patch \"https://a4d303792d30e478da0cee90b09d474b-fa61d523e62daa65.elb.us-east-2.amazonaws.com:6443/api/v1/namespaces/default/events/ip-10-0-134-46.us-west-1.compute.internal.18e36254bb8eb714\": read tcp 10.0.134.46:40444->77.113.110.115:6443: use of closed network connection" event="&Event{ObjectMeta:{ip-10-0-134-46.us-west-1.compute.internal.18e36254bb8eb714  default   4684 0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-46.us-west-1.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-134-46.us-west-1.compute.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-134-46.us-west-1.compute.internal,},FirstTimestamp:2026-10-30 18:33:12 +0000 UTC,LastTimestamp:2026-10-30 18:33:12.846946736 +0000 UTC m=+0.534301354,Count:4,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-46.us-west-1.compute.internal,}"
Oct 30 18:33:13.635178 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:13.635150    2574 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-134-46.us-west-1.compute.internal\" not found"
Oct 30 18:33:13.735881 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:13.735825    2574 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-134-46.us-west-1.compute.internal\" not found"
Oct 30 18:33:13.836404 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:13.836376    2574 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-134-46.us-west-1.compute.internal\" not found"
Oct 30 18:33:13.899074 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:13.899028    2574 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-10-29 18:28:14 +0000 UTC" deadline="2028-06-23 05:28:49.07840773 +0000 UTC"
Oct 30 18:33:13.899074 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:13.899067    2574 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14434h55m35.179343417s"
Oct 30 18:33:13.937125 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:13.937108    2574 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-134-46.us-west-1.compute.internal\" not found"
Oct 30 18:33:14.006454 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:14.006380    2574 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 18:33:14.007398 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:14.007348    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-46.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Oct 30 18:33:14.007494 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:14.007407    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-46.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 18:33:14.007494 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:14.007418    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-46.us-west-1.compute.internal" event="NodeHasSufficientPID"
Oct 30 18:33:14.038083 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:14.038060    2574 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-134-46.us-west-1.compute.internal\" not found"
Oct 30 18:33:14.123110 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:14.123090    2574 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 18:33:14.138272 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:14.138254    2574 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-134-46.us-west-1.compute.internal\" not found"
Oct 30 18:33:14.167485 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:14.167467    2574 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 18:33:14.168446 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:14.168430    2574 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-46.us-west-1.compute.internal"
Oct 30 18:33:14.226037 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:14.226015    2574 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 18:33:14.227186 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:14.227166    2574 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-46.us-west-1.compute.internal"
Oct 30 18:33:14.289129 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:14.289111    2574 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 18:33:14.289281 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:14.289255    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod563184d807a5bf00cb82141d4921b071.slice/crio-94eb3352db48f6acf5fa74ee0bcc8ac26482c3360cb4db2fabe3ff668bd86c13 WatchSource:0}: Error finding container 94eb3352db48f6acf5fa74ee0bcc8ac26482c3360cb4db2fabe3ff668bd86c13: Status 404 returned error can't find the container with id 94eb3352db48f6acf5fa74ee0bcc8ac26482c3360cb4db2fabe3ff668bd86c13
Oct 30 18:33:14.294607 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:14.294592    2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 18:33:14.397502 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:14.397482    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f66fae2621414751921d90f1913a614.slice/crio-ecaf79502997d6f1f864f0f6e6427641cd2d8cc464f7f8ec8279c05e8bbc14b3 WatchSource:0}: Error finding container ecaf79502997d6f1f864f0f6e6427641cd2d8cc464f7f8ec8279c05e8bbc14b3: Status 404 returned error can't find the container with id ecaf79502997d6f1f864f0f6e6427641cd2d8cc464f7f8ec8279c05e8bbc14b3
Oct 30 18:33:14.549093 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:14.549017    2574 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 18:33:15.843424 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:15.843403    2574 apiserver.go:52] "Watching apiserver"
Oct 30 18:33:15.862567 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:15.862549    2574 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Oct 30 18:33:15.943508 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:15.943458    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-46.us-west-1.compute.internal" event={"ID":"6f66fae2621414751921d90f1913a614","Type":"ContainerStarted","Data":"ecaf79502997d6f1f864f0f6e6427641cd2d8cc464f7f8ec8279c05e8bbc14b3"}
Oct 30 18:33:15.944470 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:15.944450    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-46.us-west-1.compute.internal" event={"ID":"563184d807a5bf00cb82141d4921b071","Type":"ContainerStarted","Data":"94eb3352db48f6acf5fa74ee0bcc8ac26482c3360cb4db2fabe3ff668bd86c13"}
Oct 30 18:33:15.945646 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:15.945628    2574 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 18:33:15.945739 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:15.945725    2574 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Oct 30 18:33:15.946006 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:15.945991    2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-dns/node-resolver-76f9s","openshift-multus/multus-additional-cni-plugins-rzrjz","openshift-multus/network-metrics-daemon-v5phn","openshift-network-diagnostics/network-check-target-8xskh","openshift-ovn-kubernetes/ovnkube-node-xfcdl","kube-system/global-pull-secret-syncer-g2svp","kube-system/kube-apiserver-proxy-ip-10-0-134-46.us-west-1.compute.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7j9jw","openshift-cluster-node-tuning-operator/tuned-z9jmw","openshift-image-registry/node-ca-g78lj","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-46.us-west-1.compute.internal","openshift-multus/multus-9ssmv","openshift-network-operator/iptables-alerter-rcjmv","kube-system/konnectivity-agent-lmdpp"]
Oct 30 18:33:15.948558 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:15.948544    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-g78lj"
Oct 30 18:33:15.950549 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:15.950528    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-rcjmv"
Oct 30 18:33:15.952722 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:15.952706    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v5phn"
Oct 30 18:33:15.952792 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:15.952779    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v5phn" podUID="a0823e8f-bbdd-477b-95ac-dad05d23cefa"
Oct 30 18:33:15.954884 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:15.954865    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8xskh"
Oct 30 18:33:15.954970 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:15.954953    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8xskh" podUID="613020c0-f3e4-4650-896f-748a40f0ab8b"
Oct 30 18:33:15.957121 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:15.957104    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-xfcdl"
Oct 30 18:33:15.959602 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:15.959586    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g2svp"
Oct 30 18:33:15.959695 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:15.959628    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g2svp" podUID="46d9ac39-ee45-4e43-bda7-3931c3e599cf"
Oct 30 18:33:15.961932 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:15.961917    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-z9jmw"
Oct 30 18:33:15.964050 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:15.964033    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-lmdpp"
Oct 30 18:33:15.966361 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:15.966341    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-rzrjz"
Oct 30 18:33:15.968662 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:15.968648    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-76f9s"
Oct 30 18:33:15.971369 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:15.971355    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7j9jw"
Oct 30 18:33:15.975165 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:15.975144    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-9ssmv"
Oct 30 18:33:15.975257 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:15.975170    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2e49faaa-a718-44c8-9237-5a41169faee7-cni-binary-copy\") pod \"multus-additional-cni-plugins-rzrjz\" (UID: \"2e49faaa-a718-44c8-9237-5a41169faee7\") " pod="openshift-multus/multus-additional-cni-plugins-rzrjz"
Oct 30 18:33:15.975257 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:15.975194    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2e49faaa-a718-44c8-9237-5a41169faee7-tuning-conf-dir\") pod \"multus-additional-cni-plugins-rzrjz\" (UID: \"2e49faaa-a718-44c8-9237-5a41169faee7\") " pod="openshift-multus/multus-additional-cni-plugins-rzrjz"
Oct 30 18:33:15.975363 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:15.975260    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f2a80d89-61d1-4694-8734-004ce5cfb22b-host\") pod \"tuned-z9jmw\" (UID: \"f2a80d89-61d1-4694-8734-004ce5cfb22b\") " pod="openshift-cluster-node-tuning-operator/tuned-z9jmw"
Oct 30 18:33:15.975363 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:15.975344    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/46d9ac39-ee45-4e43-bda7-3931c3e599cf-dbus\") pod \"global-pull-secret-syncer-g2svp\" (UID: \"46d9ac39-ee45-4e43-bda7-3931c3e599cf\") " pod="kube-system/global-pull-secret-syncer-g2svp"
Oct 30 18:33:15.975426 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:15.975397    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/8ecf2b62-8053-4bba-a852-c6f6119d3288-agent-certs\") pod \"konnectivity-agent-lmdpp\" (UID: \"8ecf2b62-8053-4bba-a852-c6f6119d3288\") " pod="kube-system/konnectivity-agent-lmdpp"
Oct 30 18:33:15.975460 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:15.975422    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/8ecf2b62-8053-4bba-a852-c6f6119d3288-konnectivity-ca\") pod \"konnectivity-agent-lmdpp\" (UID: \"8ecf2b62-8053-4bba-a852-c6f6119d3288\") " pod="kube-system/konnectivity-agent-lmdpp"
Oct 30 18:33:15.975460 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:15.975445    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/3f3fe764-2652-46c9-8e54-7882c0e6bcf1-host-kubelet\") pod \"ovnkube-node-xfcdl\" (UID: \"3f3fe764-2652-46c9-8e54-7882c0e6bcf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xfcdl"
Oct 30 18:33:15.975526 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:15.975468    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/3f3fe764-2652-46c9-8e54-7882c0e6bcf1-host-cni-netd\") pod \"ovnkube-node-xfcdl\" (UID: \"3f3fe764-2652-46c9-8e54-7882c0e6bcf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xfcdl"
Oct 30 18:33:15.975526 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:15.975492    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/3f3fe764-2652-46c9-8e54-7882c0e6bcf1-ovnkube-config\") pod \"ovnkube-node-xfcdl\" (UID: \"3f3fe764-2652-46c9-8e54-7882c0e6bcf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xfcdl"
Oct 30 18:33:15.975641 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:15.975530    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f2a80d89-61d1-4694-8734-004ce5cfb22b-etc-systemd\") pod \"tuned-z9jmw\" (UID: \"f2a80d89-61d1-4694-8734-004ce5cfb22b\") " pod="openshift-cluster-node-tuning-operator/tuned-z9jmw"
Oct 30 18:33:15.975641 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:15.975582    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6llht\" (UniqueName: \"kubernetes.io/projected/2e49faaa-a718-44c8-9237-5a41169faee7-kube-api-access-6llht\") pod \"multus-additional-cni-plugins-rzrjz\" (UID: \"2e49faaa-a718-44c8-9237-5a41169faee7\") " pod="openshift-multus/multus-additional-cni-plugins-rzrjz"
Oct 30 18:33:15.975641 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:15.975617    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/46a80637-6ccf-4a56-9904-f5e2fe5784ad-serviceca\") pod \"node-ca-g78lj\" (UID: \"46a80637-6ccf-4a56-9904-f5e2fe5784ad\") " pod="openshift-image-registry/node-ca-g78lj"
Oct 30 18:33:15.975641 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:15.975637    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/3f3fe764-2652-46c9-8e54-7882c0e6bcf1-env-overrides\") pod \"ovnkube-node-xfcdl\" (UID: \"3f3fe764-2652-46c9-8e54-7882c0e6bcf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xfcdl"
Oct 30 18:33:15.975798 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:15.975651    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f2a80d89-61d1-4694-8734-004ce5cfb22b-var-lib-kubelet\") pod \"tuned-z9jmw\" (UID: \"f2a80d89-61d1-4694-8734-004ce5cfb22b\") " pod="openshift-cluster-node-tuning-operator/tuned-z9jmw"
Oct 30 18:33:15.975798 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:15.975666    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f2a80d89-61d1-4694-8734-004ce5cfb22b-tmp\") pod \"tuned-z9jmw\" (UID: \"f2a80d89-61d1-4694-8734-004ce5cfb22b\") " pod="openshift-cluster-node-tuning-operator/tuned-z9jmw"
Oct 30 18:33:15.975798 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:15.975679    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3f3fe764-2652-46c9-8e54-7882c0e6bcf1-host-run-netns\") pod \"ovnkube-node-xfcdl\" (UID: \"3f3fe764-2652-46c9-8e54-7882c0e6bcf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xfcdl"
Oct 30 18:33:15.975798 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:15.975701    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3f3fe764-2652-46c9-8e54-7882c0e6bcf1-etc-openvswitch\") pod \"ovnkube-node-xfcdl\" (UID: \"3f3fe764-2652-46c9-8e54-7882c0e6bcf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xfcdl"
Oct 30 18:33:15.975798 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:15.975736    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3f3fe764-2652-46c9-8e54-7882c0e6bcf1-host-run-ovn-kubernetes\") pod \"ovnkube-node-xfcdl\" (UID: \"3f3fe764-2652-46c9-8e54-7882c0e6bcf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xfcdl"
Oct 30 18:33:15.975798 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:15.975761    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f2a80d89-61d1-4694-8734-004ce5cfb22b-etc-sysctl-d\") pod \"tuned-z9jmw\" (UID: \"f2a80d89-61d1-4694-8734-004ce5cfb22b\") " pod="openshift-cluster-node-tuning-operator/tuned-z9jmw"
Oct 30 18:33:15.976080 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:15.975798    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a0823e8f-bbdd-477b-95ac-dad05d23cefa-metrics-certs\") pod \"network-metrics-daemon-v5phn\" (UID: \"a0823e8f-bbdd-477b-95ac-dad05d23cefa\") " pod="openshift-multus/network-metrics-daemon-v5phn"
Oct 30 18:33:15.976080 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:15.975859    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wzxwr\" (UniqueName: \"kubernetes.io/projected/3f3fe764-2652-46c9-8e54-7882c0e6bcf1-kube-api-access-wzxwr\") pod \"ovnkube-node-xfcdl\" (UID: \"3f3fe764-2652-46c9-8e54-7882c0e6bcf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xfcdl"
Oct 30 18:33:15.976080 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:15.975913    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/3f3fe764-2652-46c9-8e54-7882c0e6bcf1-systemd-units\") pod \"ovnkube-node-xfcdl\" (UID: \"3f3fe764-2652-46c9-8e54-7882c0e6bcf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xfcdl"
Oct 30 18:33:15.976080 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:15.975938    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/3f3fe764-2652-46c9-8e54-7882c0e6bcf1-run-systemd\") pod \"ovnkube-node-xfcdl\" (UID: \"3f3fe764-2652-46c9-8e54-7882c0e6bcf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xfcdl"
Oct 30 18:33:15.976080 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:15.975961    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/46d9ac39-ee45-4e43-bda7-3931c3e599cf-original-pull-secret\") pod \"global-pull-secret-syncer-g2svp\" (UID: \"46d9ac39-ee45-4e43-bda7-3931c3e599cf\") " pod="kube-system/global-pull-secret-syncer-g2svp"
Oct 30 18:33:15.976080 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:15.975986    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2e49faaa-a718-44c8-9237-5a41169faee7-system-cni-dir\") pod \"multus-additional-cni-plugins-rzrjz\" (UID: \"2e49faaa-a718-44c8-9237-5a41169faee7\") " pod="openshift-multus/multus-additional-cni-plugins-rzrjz"
Oct 30 18:33:15.976080 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:15.976025    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2e49faaa-a718-44c8-9237-5a41169faee7-cnibin\") pod \"multus-additional-cni-plugins-rzrjz\" (UID: \"2e49faaa-a718-44c8-9237-5a41169faee7\") " pod="openshift-multus/multus-additional-cni-plugins-rzrjz"
Oct 30 18:33:15.976080 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:15.976056    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/2e49faaa-a718-44c8-9237-5a41169faee7-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-rzrjz\" (UID: \"2e49faaa-a718-44c8-9237-5a41169faee7\") " pod="openshift-multus/multus-additional-cni-plugins-rzrjz"
Oct 30 18:33:15.976398 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:15.976081    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bgxbg\" (UniqueName: \"kubernetes.io/projected/63090f48-9143-4118-9906-d759c0aff52c-kube-api-access-bgxbg\") pod \"iptables-alerter-rcjmv\" (UID: \"63090f48-9143-4118-9906-d759c0aff52c\") " pod="openshift-network-operator/iptables-alerter-rcjmv"
Oct 30 18:33:15.976398 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:15.976110    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3f3fe764-2652-46c9-8e54-7882c0e6bcf1-var-lib-openvswitch\") pod \"ovnkube-node-xfcdl\" (UID: \"3f3fe764-2652-46c9-8e54-7882c0e6bcf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xfcdl"
Oct 30 18:33:15.976398 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:15.976132    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f2a80d89-61d1-4694-8734-004ce5cfb22b-run\") pod \"tuned-z9jmw\" (UID: \"f2a80d89-61d1-4694-8734-004ce5cfb22b\") " pod="openshift-cluster-node-tuning-operator/tuned-z9jmw"
Oct 30 18:33:15.976398 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:15.976180    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3f3fe764-2652-46c9-8e54-7882c0e6bcf1-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-xfcdl\" (UID: \"3f3fe764-2652-46c9-8e54-7882c0e6bcf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xfcdl"
Oct 30 18:33:15.976398 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:15.976199    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/3f3fe764-2652-46c9-8e54-7882c0e6bcf1-ovn-node-metrics-cert\") pod \"ovnkube-node-xfcdl\" (UID: \"3f3fe764-2652-46c9-8e54-7882c0e6bcf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xfcdl"
Oct 30 18:33:15.976398 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:15.976213    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/63090f48-9143-4118-9906-d759c0aff52c-iptables-alerter-script\") pod \"iptables-alerter-rcjmv\" (UID: \"63090f48-9143-4118-9906-d759c0aff52c\") " pod="openshift-network-operator/iptables-alerter-rcjmv"
Oct 30 18:33:15.976398 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:15.976228    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/3f3fe764-2652-46c9-8e54-7882c0e6bcf1-run-ovn\") pod \"ovnkube-node-xfcdl\" (UID: \"3f3fe764-2652-46c9-8e54-7882c0e6bcf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xfcdl"
Oct 30 18:33:15.976398 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:15.976248    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f2a80d89-61d1-4694-8734-004ce5cfb22b-etc-sysconfig\") pod \"tuned-z9jmw\" (UID: \"f2a80d89-61d1-4694-8734-004ce5cfb22b\") " pod="openshift-cluster-node-tuning-operator/tuned-z9jmw"
Oct 30 18:33:15.976398 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:15.976273    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3f3fe764-2652-46c9-8e54-7882c0e6bcf1-host-cni-bin\") pod \"ovnkube-node-xfcdl\" (UID: \"3f3fe764-2652-46c9-8e54-7882c0e6bcf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xfcdl"
Oct 30 18:33:15.976398 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:15.976323    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/3f3fe764-2652-46c9-8e54-7882c0e6bcf1-ovnkube-script-lib\") pod \"ovnkube-node-xfcdl\" (UID: \"3f3fe764-2652-46c9-8e54-7882c0e6bcf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xfcdl"
Oct 30 18:33:15.976398 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:15.976345    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f2a80d89-61d1-4694-8734-004ce5cfb22b-etc-tuned\") pod \"tuned-z9jmw\" (UID: \"f2a80d89-61d1-4694-8734-004ce5cfb22b\") " pod="openshift-cluster-node-tuning-operator/tuned-z9jmw"
Oct 30 18:33:15.976398 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:15.976364    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x2978\" (UniqueName: \"kubernetes.io/projected/46a80637-6ccf-4a56-9904-f5e2fe5784ad-kube-api-access-x2978\") pod \"node-ca-g78lj\" (UID: \"46a80637-6ccf-4a56-9904-f5e2fe5784ad\") " pod="openshift-image-registry/node-ca-g78lj"
Oct 30 18:33:15.976398 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:15.976377    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f2a80d89-61d1-4694-8734-004ce5cfb22b-etc-sysctl-conf\") pod \"tuned-z9jmw\" (UID: \"f2a80d89-61d1-4694-8734-004ce5cfb22b\") " pod="openshift-cluster-node-tuning-operator/tuned-z9jmw"
Oct 30 18:33:15.976398 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:15.976392    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/46d9ac39-ee45-4e43-bda7-3931c3e599cf-kubelet-config\") pod \"global-pull-secret-syncer-g2svp\" (UID: \"46d9ac39-ee45-4e43-bda7-3931c3e599cf\") " pod="kube-system/global-pull-secret-syncer-g2svp"
Oct 30 18:33:15.976930 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:15.976422    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/2e49faaa-a718-44c8-9237-5a41169faee7-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-rzrjz\" (UID: \"2e49faaa-a718-44c8-9237-5a41169faee7\") " pod="openshift-multus/multus-additional-cni-plugins-rzrjz"
Oct 30 18:33:15.976930 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:15.976440    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c45qr\" (UniqueName: \"kubernetes.io/projected/613020c0-f3e4-4650-896f-748a40f0ab8b-kube-api-access-c45qr\") pod \"network-check-target-8xskh\" (UID: \"613020c0-f3e4-4650-896f-748a40f0ab8b\") " pod="openshift-network-diagnostics/network-check-target-8xskh"
Oct 30 18:33:15.976930 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:15.976453    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f2a80d89-61d1-4694-8734-004ce5cfb22b-etc-modprobe-d\") pod \"tuned-z9jmw\" (UID: \"f2a80d89-61d1-4694-8734-004ce5cfb22b\") " pod="openshift-cluster-node-tuning-operator/tuned-z9jmw"
Oct 30 18:33:15.976930 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:15.976466    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f2a80d89-61d1-4694-8734-004ce5cfb22b-etc-kubernetes\") pod \"tuned-z9jmw\" (UID: \"f2a80d89-61d1-4694-8734-004ce5cfb22b\") " pod="openshift-cluster-node-tuning-operator/tuned-z9jmw"
Oct 30 18:33:15.976930 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:15.976486    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/46a80637-6ccf-4a56-9904-f5e2fe5784ad-host\") pod \"node-ca-g78lj\" (UID: \"46a80637-6ccf-4a56-9904-f5e2fe5784ad\") " pod="openshift-image-registry/node-ca-g78lj"
Oct 30 18:33:15.976930 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:15.976505    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/63090f48-9143-4118-9906-d759c0aff52c-host-slash\") pod \"iptables-alerter-rcjmv\" (UID: \"63090f48-9143-4118-9906-d759c0aff52c\") " pod="openshift-network-operator/iptables-alerter-rcjmv"
Oct 30 18:33:15.976930 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:15.976526    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3f3fe764-2652-46c9-8e54-7882c0e6bcf1-host-slash\") pod \"ovnkube-node-xfcdl\" (UID: \"3f3fe764-2652-46c9-8e54-7882c0e6bcf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xfcdl"
Oct 30 18:33:15.976930 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:15.976573    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/3f3fe764-2652-46c9-8e54-7882c0e6bcf1-node-log\") pod \"ovnkube-node-xfcdl\" (UID: \"3f3fe764-2652-46c9-8e54-7882c0e6bcf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xfcdl"
Oct 30 18:33:15.976930 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:15.976614    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f2a80d89-61d1-4694-8734-004ce5cfb22b-lib-modules\") pod \"tuned-z9jmw\" (UID: \"f2a80d89-61d1-4694-8734-004ce5cfb22b\") " pod="openshift-cluster-node-tuning-operator/tuned-z9jmw"
Oct 30 18:33:15.976930 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:15.976641    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t56q5\" (UniqueName: \"kubernetes.io/projected/a0823e8f-bbdd-477b-95ac-dad05d23cefa-kube-api-access-t56q5\") pod \"network-metrics-daemon-v5phn\" (UID: \"a0823e8f-bbdd-477b-95ac-dad05d23cefa\") " pod="openshift-multus/network-metrics-daemon-v5phn"
Oct 30 18:33:15.976930 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:15.976668    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3f3fe764-2652-46c9-8e54-7882c0e6bcf1-run-openvswitch\") pod \"ovnkube-node-xfcdl\" (UID: \"3f3fe764-2652-46c9-8e54-7882c0e6bcf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xfcdl"
Oct 30 18:33:15.976930 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:15.976691    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/3f3fe764-2652-46c9-8e54-7882c0e6bcf1-log-socket\") pod \"ovnkube-node-xfcdl\" (UID: \"3f3fe764-2652-46c9-8e54-7882c0e6bcf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xfcdl"
Oct 30 18:33:15.976930 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:15.976711    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f2a80d89-61d1-4694-8734-004ce5cfb22b-sys\") pod \"tuned-z9jmw\" (UID: \"f2a80d89-61d1-4694-8734-004ce5cfb22b\") " pod="openshift-cluster-node-tuning-operator/tuned-z9jmw"
Oct 30 18:33:15.976930 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:15.976748    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8wvhh\" (UniqueName: \"kubernetes.io/projected/f2a80d89-61d1-4694-8734-004ce5cfb22b-kube-api-access-8wvhh\") pod \"tuned-z9jmw\" (UID: \"f2a80d89-61d1-4694-8734-004ce5cfb22b\") " pod="openshift-cluster-node-tuning-operator/tuned-z9jmw"
Oct 30 18:33:16.000816 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.000795    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Oct 30 18:33:16.000900 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.000890    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-qb49q\""
Oct 30 18:33:16.000953 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.000888    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Oct 30 18:33:16.000953 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.000884    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Oct 30 18:33:16.002220 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.002201    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-nh9hp\""
Oct 30 18:33:16.002343 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.002245    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Oct 30 18:33:16.002343 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.002270    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Oct 30 18:33:16.002343 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.002312    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Oct 30 18:33:16.008923 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.008907    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Oct 30 18:33:16.009077 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.009066    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Oct 30 18:33:16.009194 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.009179    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-jkxcr\""
Oct 30 18:33:16.009256 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.009230    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Oct 30 18:33:16.009256 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.009233    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Oct 30 18:33:16.009383 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.009343    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Oct 30 18:33:16.009383 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.009360    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Oct 30 18:33:16.013438 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.013423    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Oct 30 18:33:16.013571 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.013558    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Oct 30 18:33:16.013630 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.013615    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-5qqcs\""
Oct 30 18:33:16.015543 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.015525    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Oct 30 18:33:16.015689 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.015676    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Oct 30 18:33:16.015689 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.015684    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-g6mvb\""
Oct 30 18:33:16.021413 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.021400    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Oct 30 18:33:16.021460 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.021453    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Oct 30 18:33:16.048948 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.048930    2574 reflector.go:446] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Oct 30 18:33:16.049056 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.048975    2574 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-qwgtf"
Oct 30 18:33:16.064744 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.064719    2574 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Oct 30 18:33:16.076863 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.076840    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/aa3f3b62-a1ab-4113-bef3-ed1b290e47c1-host-var-lib-cni-bin\") pod \"multus-9ssmv\" (UID: \"aa3f3b62-a1ab-4113-bef3-ed1b290e47c1\") " pod="openshift-multus/multus-9ssmv"
Oct 30 18:33:16.076949 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.076869    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/63090f48-9143-4118-9906-d759c0aff52c-iptables-alerter-script\") pod \"iptables-alerter-rcjmv\" (UID: \"63090f48-9143-4118-9906-d759c0aff52c\") " pod="openshift-network-operator/iptables-alerter-rcjmv"
Oct 30 18:33:16.076949 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.076887    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/3f3fe764-2652-46c9-8e54-7882c0e6bcf1-run-ovn\") pod \"ovnkube-node-xfcdl\" (UID: \"3f3fe764-2652-46c9-8e54-7882c0e6bcf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xfcdl"
Oct 30 18:33:16.076949 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.076912    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f2a80d89-61d1-4694-8734-004ce5cfb22b-etc-sysconfig\") pod \"tuned-z9jmw\" (UID: \"f2a80d89-61d1-4694-8734-004ce5cfb22b\") " pod="openshift-cluster-node-tuning-operator/tuned-z9jmw"
Oct 30 18:33:16.077080 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.076968    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ad78ad05-b0af-4adf-a6bc-0f993b8a6288-socket-dir\") pod \"aws-ebs-csi-driver-node-7j9jw\" (UID: \"ad78ad05-b0af-4adf-a6bc-0f993b8a6288\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7j9jw"
Oct 30 18:33:16.077080 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.076987    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f2a80d89-61d1-4694-8734-004ce5cfb22b-etc-sysconfig\") pod \"tuned-z9jmw\" (UID: \"f2a80d89-61d1-4694-8734-004ce5cfb22b\") " pod="openshift-cluster-node-tuning-operator/tuned-z9jmw"
Oct 30 18:33:16.077080 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.076996    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/aa3f3b62-a1ab-4113-bef3-ed1b290e47c1-multus-conf-dir\") pod \"multus-9ssmv\" (UID: \"aa3f3b62-a1ab-4113-bef3-ed1b290e47c1\") " pod="openshift-multus/multus-9ssmv"
Oct 30 18:33:16.077080 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.077045    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3f3fe764-2652-46c9-8e54-7882c0e6bcf1-host-cni-bin\") pod \"ovnkube-node-xfcdl\" (UID: \"3f3fe764-2652-46c9-8e54-7882c0e6bcf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xfcdl"
Oct 30 18:33:16.077080 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.077065    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/3f3fe764-2652-46c9-8e54-7882c0e6bcf1-run-ovn\") pod \"ovnkube-node-xfcdl\" (UID: \"3f3fe764-2652-46c9-8e54-7882c0e6bcf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xfcdl"
Oct 30 18:33:16.077080 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.077071    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/3f3fe764-2652-46c9-8e54-7882c0e6bcf1-ovnkube-script-lib\") pod \"ovnkube-node-xfcdl\" (UID: \"3f3fe764-2652-46c9-8e54-7882c0e6bcf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xfcdl"
Oct 30 18:33:16.077383 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.077070    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3f3fe764-2652-46c9-8e54-7882c0e6bcf1-host-cni-bin\") pod \"ovnkube-node-xfcdl\" (UID: \"3f3fe764-2652-46c9-8e54-7882c0e6bcf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xfcdl"
Oct 30 18:33:16.077383 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.077117    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f2a80d89-61d1-4694-8734-004ce5cfb22b-etc-tuned\") pod \"tuned-z9jmw\" (UID: \"f2a80d89-61d1-4694-8734-004ce5cfb22b\") " pod="openshift-cluster-node-tuning-operator/tuned-z9jmw"
Oct 30 18:33:16.077383 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.077146    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x2978\" (UniqueName: \"kubernetes.io/projected/46a80637-6ccf-4a56-9904-f5e2fe5784ad-kube-api-access-x2978\") pod \"node-ca-g78lj\" (UID: \"46a80637-6ccf-4a56-9904-f5e2fe5784ad\") " pod="openshift-image-registry/node-ca-g78lj"
Oct 30 18:33:16.077383 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.077175    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f2a80d89-61d1-4694-8734-004ce5cfb22b-etc-sysctl-conf\") pod \"tuned-z9jmw\" (UID: \"f2a80d89-61d1-4694-8734-004ce5cfb22b\") " pod="openshift-cluster-node-tuning-operator/tuned-z9jmw"
Oct 30 18:33:16.077383 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.077203    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5f2ls\" (UniqueName: \"kubernetes.io/projected/ad78ad05-b0af-4adf-a6bc-0f993b8a6288-kube-api-access-5f2ls\") pod \"aws-ebs-csi-driver-node-7j9jw\" (UID: \"ad78ad05-b0af-4adf-a6bc-0f993b8a6288\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7j9jw"
Oct 30 18:33:16.077383 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.077255    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/aa3f3b62-a1ab-4113-bef3-ed1b290e47c1-hostroot\") pod \"multus-9ssmv\" (UID: \"aa3f3b62-a1ab-4113-bef3-ed1b290e47c1\") " pod="openshift-multus/multus-9ssmv"
Oct 30 18:33:16.077383 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.077299    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/46d9ac39-ee45-4e43-bda7-3931c3e599cf-kubelet-config\") pod \"global-pull-secret-syncer-g2svp\" (UID: \"46d9ac39-ee45-4e43-bda7-3931c3e599cf\") " pod="kube-system/global-pull-secret-syncer-g2svp"
Oct 30 18:33:16.077383 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.077327    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/2e49faaa-a718-44c8-9237-5a41169faee7-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-rzrjz\" (UID: \"2e49faaa-a718-44c8-9237-5a41169faee7\") " pod="openshift-multus/multus-additional-cni-plugins-rzrjz"
Oct 30 18:33:16.077383 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.077359    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c45qr\" (UniqueName: \"kubernetes.io/projected/613020c0-f3e4-4650-896f-748a40f0ab8b-kube-api-access-c45qr\") pod \"network-check-target-8xskh\" (UID: \"613020c0-f3e4-4650-896f-748a40f0ab8b\") " pod="openshift-network-diagnostics/network-check-target-8xskh"
Oct 30 18:33:16.077383 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.077384    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f2a80d89-61d1-4694-8734-004ce5cfb22b-etc-modprobe-d\") pod \"tuned-z9jmw\" (UID: \"f2a80d89-61d1-4694-8734-004ce5cfb22b\") " pod="openshift-cluster-node-tuning-operator/tuned-z9jmw"
Oct 30 18:33:16.077844 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.077388    2574 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Oct 30 18:33:16.077844 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.077397    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f2a80d89-61d1-4694-8734-004ce5cfb22b-etc-sysctl-conf\") pod \"tuned-z9jmw\" (UID: \"f2a80d89-61d1-4694-8734-004ce5cfb22b\") " pod="openshift-cluster-node-tuning-operator/tuned-z9jmw"
Oct 30 18:33:16.077844 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.077407    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f2a80d89-61d1-4694-8734-004ce5cfb22b-etc-kubernetes\") pod \"tuned-z9jmw\" (UID: \"f2a80d89-61d1-4694-8734-004ce5cfb22b\") " pod="openshift-cluster-node-tuning-operator/tuned-z9jmw"
Oct 30 18:33:16.077844 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.077433    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ad78ad05-b0af-4adf-a6bc-0f993b8a6288-sys-fs\") pod \"aws-ebs-csi-driver-node-7j9jw\" (UID: \"ad78ad05-b0af-4adf-a6bc-0f993b8a6288\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7j9jw"
Oct 30 18:33:16.077844 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.077440    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/46d9ac39-ee45-4e43-bda7-3931c3e599cf-kubelet-config\") pod \"global-pull-secret-syncer-g2svp\" (UID: \"46d9ac39-ee45-4e43-bda7-3931c3e599cf\") " pod="kube-system/global-pull-secret-syncer-g2svp"
Oct 30 18:33:16.077844 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.077456    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/46a80637-6ccf-4a56-9904-f5e2fe5784ad-host\") pod \"node-ca-g78lj\" (UID: \"46a80637-6ccf-4a56-9904-f5e2fe5784ad\") " pod="openshift-image-registry/node-ca-g78lj"
Oct 30 18:33:16.077844 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.077489    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f2a80d89-61d1-4694-8734-004ce5cfb22b-etc-modprobe-d\") pod \"tuned-z9jmw\" (UID: \"f2a80d89-61d1-4694-8734-004ce5cfb22b\") " pod="openshift-cluster-node-tuning-operator/tuned-z9jmw"
Oct 30 18:33:16.077844 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.077497    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/46a80637-6ccf-4a56-9904-f5e2fe5784ad-host\") pod \"node-ca-g78lj\" (UID: \"46a80637-6ccf-4a56-9904-f5e2fe5784ad\") " pod="openshift-image-registry/node-ca-g78lj"
Oct 30 18:33:16.077844 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.077506    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/63090f48-9143-4118-9906-d759c0aff52c-host-slash\") pod \"iptables-alerter-rcjmv\" (UID: \"63090f48-9143-4118-9906-d759c0aff52c\") " pod="openshift-network-operator/iptables-alerter-rcjmv"
Oct 30 18:33:16.077844 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.077520    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/63090f48-9143-4118-9906-d759c0aff52c-iptables-alerter-script\") pod \"iptables-alerter-rcjmv\" (UID: \"63090f48-9143-4118-9906-d759c0aff52c\") " pod="openshift-network-operator/iptables-alerter-rcjmv"
Oct 30 18:33:16.077844 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.077578    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3f3fe764-2652-46c9-8e54-7882c0e6bcf1-host-slash\") pod \"ovnkube-node-xfcdl\" (UID: \"3f3fe764-2652-46c9-8e54-7882c0e6bcf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xfcdl"
Oct 30 18:33:16.077844 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.077550    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3f3fe764-2652-46c9-8e54-7882c0e6bcf1-host-slash\") pod \"ovnkube-node-xfcdl\" (UID: \"3f3fe764-2652-46c9-8e54-7882c0e6bcf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xfcdl"
Oct 30 18:33:16.077844 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.077610    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/3f3fe764-2652-46c9-8e54-7882c0e6bcf1-node-log\") pod \"ovnkube-node-xfcdl\" (UID: \"3f3fe764-2652-46c9-8e54-7882c0e6bcf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xfcdl"
Oct 30 18:33:16.077844 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.077545    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f2a80d89-61d1-4694-8734-004ce5cfb22b-etc-kubernetes\") pod \"tuned-z9jmw\" (UID: \"f2a80d89-61d1-4694-8734-004ce5cfb22b\") " pod="openshift-cluster-node-tuning-operator/tuned-z9jmw"
Oct 30 18:33:16.077844 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.077579    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/63090f48-9143-4118-9906-d759c0aff52c-host-slash\") pod \"iptables-alerter-rcjmv\" (UID: \"63090f48-9143-4118-9906-d759c0aff52c\") " pod="openshift-network-operator/iptables-alerter-rcjmv"
Oct 30 18:33:16.077844 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.077633    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f2a80d89-61d1-4694-8734-004ce5cfb22b-lib-modules\") pod \"tuned-z9jmw\" (UID: \"f2a80d89-61d1-4694-8734-004ce5cfb22b\") " pod="openshift-cluster-node-tuning-operator/tuned-z9jmw"
Oct 30 18:33:16.077844 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.077637    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/3f3fe764-2652-46c9-8e54-7882c0e6bcf1-node-log\") pod \"ovnkube-node-xfcdl\" (UID: \"3f3fe764-2652-46c9-8e54-7882c0e6bcf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xfcdl"
Oct 30 18:33:16.077844 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.077645    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/3f3fe764-2652-46c9-8e54-7882c0e6bcf1-ovnkube-script-lib\") pod \"ovnkube-node-xfcdl\" (UID: \"3f3fe764-2652-46c9-8e54-7882c0e6bcf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xfcdl"
Oct 30 18:33:16.078669 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.077659    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ad78ad05-b0af-4adf-a6bc-0f993b8a6288-kubelet-dir\") pod \"aws-ebs-csi-driver-node-7j9jw\" (UID: \"ad78ad05-b0af-4adf-a6bc-0f993b8a6288\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7j9jw"
Oct 30 18:33:16.078669 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.077685    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t56q5\" (UniqueName: \"kubernetes.io/projected/a0823e8f-bbdd-477b-95ac-dad05d23cefa-kube-api-access-t56q5\") pod \"network-metrics-daemon-v5phn\" (UID: \"a0823e8f-bbdd-477b-95ac-dad05d23cefa\") " pod="openshift-multus/network-metrics-daemon-v5phn"
Oct 30 18:33:16.078669 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.077709    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3f3fe764-2652-46c9-8e54-7882c0e6bcf1-run-openvswitch\") pod \"ovnkube-node-xfcdl\" (UID: \"3f3fe764-2652-46c9-8e54-7882c0e6bcf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xfcdl"
Oct 30 18:33:16.078669 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.077745    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/3f3fe764-2652-46c9-8e54-7882c0e6bcf1-log-socket\") pod \"ovnkube-node-xfcdl\" (UID: \"3f3fe764-2652-46c9-8e54-7882c0e6bcf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xfcdl"
Oct 30 18:33:16.078669 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.077755    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3f3fe764-2652-46c9-8e54-7882c0e6bcf1-run-openvswitch\") pod \"ovnkube-node-xfcdl\" (UID: \"3f3fe764-2652-46c9-8e54-7882c0e6bcf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xfcdl"
Oct 30 18:33:16.078669 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.077764    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f2a80d89-61d1-4694-8734-004ce5cfb22b-lib-modules\") pod \"tuned-z9jmw\" (UID: \"f2a80d89-61d1-4694-8734-004ce5cfb22b\") " pod="openshift-cluster-node-tuning-operator/tuned-z9jmw"
Oct 30 18:33:16.078669 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.077766    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f2a80d89-61d1-4694-8734-004ce5cfb22b-sys\") pod \"tuned-z9jmw\" (UID: \"f2a80d89-61d1-4694-8734-004ce5cfb22b\") " pod="openshift-cluster-node-tuning-operator/tuned-z9jmw"
Oct 30 18:33:16.078669 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.077788    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/3f3fe764-2652-46c9-8e54-7882c0e6bcf1-log-socket\") pod \"ovnkube-node-xfcdl\" (UID: \"3f3fe764-2652-46c9-8e54-7882c0e6bcf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xfcdl"
Oct 30 18:33:16.078669 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.077799    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8wvhh\" (UniqueName: \"kubernetes.io/projected/f2a80d89-61d1-4694-8734-004ce5cfb22b-kube-api-access-8wvhh\") pod \"tuned-z9jmw\" (UID: \"f2a80d89-61d1-4694-8734-004ce5cfb22b\") " pod="openshift-cluster-node-tuning-operator/tuned-z9jmw"
Oct 30 18:33:16.078669 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.077824    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gzhnk\" (UniqueName: \"kubernetes.io/projected/d81cef55-16ee-4677-ae12-86de3c9eb216-kube-api-access-gzhnk\") pod \"node-resolver-76f9s\" (UID: \"d81cef55-16ee-4677-ae12-86de3c9eb216\") " pod="openshift-dns/node-resolver-76f9s"
Oct 30 18:33:16.078669 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.077845    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/aa3f3b62-a1ab-4113-bef3-ed1b290e47c1-multus-cni-dir\") pod \"multus-9ssmv\" (UID: \"aa3f3b62-a1ab-4113-bef3-ed1b290e47c1\") " pod="openshift-multus/multus-9ssmv"
Oct 30 18:33:16.078669 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.077858    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f2a80d89-61d1-4694-8734-004ce5cfb22b-sys\") pod \"tuned-z9jmw\" (UID: \"f2a80d89-61d1-4694-8734-004ce5cfb22b\") " pod="openshift-cluster-node-tuning-operator/tuned-z9jmw"
Oct 30 18:33:16.078669 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.077862    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2e49faaa-a718-44c8-9237-5a41169faee7-cni-binary-copy\") pod \"multus-additional-cni-plugins-rzrjz\" (UID: \"2e49faaa-a718-44c8-9237-5a41169faee7\") " pod="openshift-multus/multus-additional-cni-plugins-rzrjz"
Oct 30 18:33:16.078669 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.077888    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ad78ad05-b0af-4adf-a6bc-0f993b8a6288-registration-dir\") pod \"aws-ebs-csi-driver-node-7j9jw\" (UID: \"ad78ad05-b0af-4adf-a6bc-0f993b8a6288\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7j9jw"
Oct 30 18:33:16.078669 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.077908    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/aa3f3b62-a1ab-4113-bef3-ed1b290e47c1-cnibin\") pod \"multus-9ssmv\" (UID: \"aa3f3b62-a1ab-4113-bef3-ed1b290e47c1\") " pod="openshift-multus/multus-9ssmv"
Oct 30 18:33:16.078669 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.077957    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/aa3f3b62-a1ab-4113-bef3-ed1b290e47c1-host-var-lib-cni-multus\") pod \"multus-9ssmv\" (UID: \"aa3f3b62-a1ab-4113-bef3-ed1b290e47c1\") " pod="openshift-multus/multus-9ssmv"
Oct 30 18:33:16.078669 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.077993    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/aa3f3b62-a1ab-4113-bef3-ed1b290e47c1-etc-kubernetes\") pod \"multus-9ssmv\" (UID: \"aa3f3b62-a1ab-4113-bef3-ed1b290e47c1\") " pod="openshift-multus/multus-9ssmv"
Oct 30 18:33:16.079644 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.078027    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2e49faaa-a718-44c8-9237-5a41169faee7-tuning-conf-dir\") pod \"multus-additional-cni-plugins-rzrjz\" (UID: \"2e49faaa-a718-44c8-9237-5a41169faee7\") " pod="openshift-multus/multus-additional-cni-plugins-rzrjz"
Oct 30 18:33:16.079644 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.078052    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f2a80d89-61d1-4694-8734-004ce5cfb22b-host\") pod \"tuned-z9jmw\" (UID: \"f2a80d89-61d1-4694-8734-004ce5cfb22b\") " pod="openshift-cluster-node-tuning-operator/tuned-z9jmw"
Oct 30 18:33:16.079644 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.078067    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d81cef55-16ee-4677-ae12-86de3c9eb216-hosts-file\") pod \"node-resolver-76f9s\" (UID: \"d81cef55-16ee-4677-ae12-86de3c9eb216\") " pod="openshift-dns/node-resolver-76f9s"
Oct 30 18:33:16.079644 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.078081    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d81cef55-16ee-4677-ae12-86de3c9eb216-tmp-dir\") pod \"node-resolver-76f9s\" (UID: \"d81cef55-16ee-4677-ae12-86de3c9eb216\") " pod="openshift-dns/node-resolver-76f9s"
Oct 30 18:33:16.079644 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.078094    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/aa3f3b62-a1ab-4113-bef3-ed1b290e47c1-cni-binary-copy\") pod \"multus-9ssmv\" (UID: \"aa3f3b62-a1ab-4113-bef3-ed1b290e47c1\") " pod="openshift-multus/multus-9ssmv"
Oct 30 18:33:16.079644 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.078108    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/46d9ac39-ee45-4e43-bda7-3931c3e599cf-dbus\") pod \"global-pull-secret-syncer-g2svp\" (UID: \"46d9ac39-ee45-4e43-bda7-3931c3e599cf\") " pod="kube-system/global-pull-secret-syncer-g2svp"
Oct 30 18:33:16.079644 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.078125    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f2a80d89-61d1-4694-8734-004ce5cfb22b-host\") pod \"tuned-z9jmw\" (UID: \"f2a80d89-61d1-4694-8734-004ce5cfb22b\") " pod="openshift-cluster-node-tuning-operator/tuned-z9jmw"
Oct 30 18:33:16.079644 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.078163    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/8ecf2b62-8053-4bba-a852-c6f6119d3288-agent-certs\") pod \"konnectivity-agent-lmdpp\" (UID: \"8ecf2b62-8053-4bba-a852-c6f6119d3288\") " pod="kube-system/konnectivity-agent-lmdpp"
Oct 30 18:33:16.079644 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.078220    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/8ecf2b62-8053-4bba-a852-c6f6119d3288-konnectivity-ca\") pod \"konnectivity-agent-lmdpp\" (UID: \"8ecf2b62-8053-4bba-a852-c6f6119d3288\") " pod="kube-system/konnectivity-agent-lmdpp"
Oct 30 18:33:16.079644 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.078245    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2e49faaa-a718-44c8-9237-5a41169faee7-tuning-conf-dir\") pod \"multus-additional-cni-plugins-rzrjz\" (UID: \"2e49faaa-a718-44c8-9237-5a41169faee7\") " pod="openshift-multus/multus-additional-cni-plugins-rzrjz"
Oct 30 18:33:16.079644 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.078300    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2e49faaa-a718-44c8-9237-5a41169faee7-cni-binary-copy\") pod \"multus-additional-cni-plugins-rzrjz\" (UID: \"2e49faaa-a718-44c8-9237-5a41169faee7\") " pod="openshift-multus/multus-additional-cni-plugins-rzrjz"
Oct 30 18:33:16.079644 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.078319    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/3f3fe764-2652-46c9-8e54-7882c0e6bcf1-host-kubelet\") pod \"ovnkube-node-xfcdl\" (UID: \"3f3fe764-2652-46c9-8e54-7882c0e6bcf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xfcdl"
Oct 30 18:33:16.079644 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.078320    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/46d9ac39-ee45-4e43-bda7-3931c3e599cf-dbus\") pod \"global-pull-secret-syncer-g2svp\" (UID: \"46d9ac39-ee45-4e43-bda7-3931c3e599cf\") " pod="kube-system/global-pull-secret-syncer-g2svp"
Oct 30 18:33:16.079644 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.078420    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/3f3fe764-2652-46c9-8e54-7882c0e6bcf1-host-kubelet\") pod \"ovnkube-node-xfcdl\" (UID: \"3f3fe764-2652-46c9-8e54-7882c0e6bcf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xfcdl"
Oct 30 18:33:16.079644 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.078427    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/3f3fe764-2652-46c9-8e54-7882c0e6bcf1-host-cni-netd\") pod \"ovnkube-node-xfcdl\" (UID: \"3f3fe764-2652-46c9-8e54-7882c0e6bcf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xfcdl"
Oct 30 18:33:16.079644 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.078472    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/3f3fe764-2652-46c9-8e54-7882c0e6bcf1-ovnkube-config\") pod \"ovnkube-node-xfcdl\" (UID: \"3f3fe764-2652-46c9-8e54-7882c0e6bcf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xfcdl"
Oct 30 18:33:16.079644 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.078503    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/3f3fe764-2652-46c9-8e54-7882c0e6bcf1-host-cni-netd\") pod \"ovnkube-node-xfcdl\" (UID: \"3f3fe764-2652-46c9-8e54-7882c0e6bcf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xfcdl"
Oct 30 18:33:16.080476 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.078559    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f2a80d89-61d1-4694-8734-004ce5cfb22b-etc-systemd\") pod \"tuned-z9jmw\" (UID: \"f2a80d89-61d1-4694-8734-004ce5cfb22b\") " pod="openshift-cluster-node-tuning-operator/tuned-z9jmw"
Oct 30 18:33:16.080476 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.078595    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/aa3f3b62-a1ab-4113-bef3-ed1b290e47c1-multus-socket-dir-parent\") pod \"multus-9ssmv\" (UID: \"aa3f3b62-a1ab-4113-bef3-ed1b290e47c1\") " pod="openshift-multus/multus-9ssmv"
Oct 30 18:33:16.080476 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.078611    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f2a80d89-61d1-4694-8734-004ce5cfb22b-etc-systemd\") pod \"tuned-z9jmw\" (UID: \"f2a80d89-61d1-4694-8734-004ce5cfb22b\") " pod="openshift-cluster-node-tuning-operator/tuned-z9jmw"
Oct 30 18:33:16.080476 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.078626    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6llht\" (UniqueName: \"kubernetes.io/projected/2e49faaa-a718-44c8-9237-5a41169faee7-kube-api-access-6llht\") pod \"multus-additional-cni-plugins-rzrjz\" (UID: \"2e49faaa-a718-44c8-9237-5a41169faee7\") " pod="openshift-multus/multus-additional-cni-plugins-rzrjz"
Oct 30 18:33:16.080476 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.078653    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/46a80637-6ccf-4a56-9904-f5e2fe5784ad-serviceca\") pod \"node-ca-g78lj\" (UID: \"46a80637-6ccf-4a56-9904-f5e2fe5784ad\") " pod="openshift-image-registry/node-ca-g78lj"
Oct 30 18:33:16.080476 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.078676    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/3f3fe764-2652-46c9-8e54-7882c0e6bcf1-env-overrides\") pod \"ovnkube-node-xfcdl\" (UID: \"3f3fe764-2652-46c9-8e54-7882c0e6bcf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xfcdl"
Oct 30 18:33:16.080476 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.078699    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f2a80d89-61d1-4694-8734-004ce5cfb22b-var-lib-kubelet\") pod \"tuned-z9jmw\" (UID: \"f2a80d89-61d1-4694-8734-004ce5cfb22b\") " pod="openshift-cluster-node-tuning-operator/tuned-z9jmw"
Oct 30 18:33:16.080476 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.078724    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f2a80d89-61d1-4694-8734-004ce5cfb22b-tmp\") pod \"tuned-z9jmw\" (UID: \"f2a80d89-61d1-4694-8734-004ce5cfb22b\") " pod="openshift-cluster-node-tuning-operator/tuned-z9jmw"
Oct 30 18:33:16.080476 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.078750    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/aa3f3b62-a1ab-4113-bef3-ed1b290e47c1-host-run-netns\") pod \"multus-9ssmv\" (UID: \"aa3f3b62-a1ab-4113-bef3-ed1b290e47c1\") " pod="openshift-multus/multus-9ssmv"
Oct 30 18:33:16.080476 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.078827    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/8ecf2b62-8053-4bba-a852-c6f6119d3288-konnectivity-ca\") pod \"konnectivity-agent-lmdpp\" (UID: \"8ecf2b62-8053-4bba-a852-c6f6119d3288\") " pod="kube-system/konnectivity-agent-lmdpp"
Oct 30 18:33:16.080476 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.078870    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/aa3f3b62-a1ab-4113-bef3-ed1b290e47c1-multus-daemon-config\") pod \"multus-9ssmv\" (UID: \"aa3f3b62-a1ab-4113-bef3-ed1b290e47c1\") " pod="openshift-multus/multus-9ssmv"
Oct 30 18:33:16.080476 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.078960    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3f3fe764-2652-46c9-8e54-7882c0e6bcf1-host-run-netns\") pod \"ovnkube-node-xfcdl\" (UID: \"3f3fe764-2652-46c9-8e54-7882c0e6bcf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xfcdl"
Oct 30 18:33:16.080476 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.078985    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3f3fe764-2652-46c9-8e54-7882c0e6bcf1-etc-openvswitch\") pod \"ovnkube-node-xfcdl\" (UID: \"3f3fe764-2652-46c9-8e54-7882c0e6bcf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xfcdl"
Oct 30 18:33:16.080476 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.078984    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/3f3fe764-2652-46c9-8e54-7882c0e6bcf1-ovnkube-config\") pod \"ovnkube-node-xfcdl\" (UID: \"3f3fe764-2652-46c9-8e54-7882c0e6bcf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xfcdl"
Oct 30 18:33:16.080476 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.079015    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3f3fe764-2652-46c9-8e54-7882c0e6bcf1-host-run-ovn-kubernetes\") pod \"ovnkube-node-xfcdl\" (UID: \"3f3fe764-2652-46c9-8e54-7882c0e6bcf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xfcdl"
Oct 30 18:33:16.080476 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.079036    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f2a80d89-61d1-4694-8734-004ce5cfb22b-etc-sysctl-d\") pod \"tuned-z9jmw\" (UID: \"f2a80d89-61d1-4694-8734-004ce5cfb22b\") " pod="openshift-cluster-node-tuning-operator/tuned-z9jmw"
Oct 30 18:33:16.080476 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.079073    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/aa3f3b62-a1ab-4113-bef3-ed1b290e47c1-host-run-k8s-cni-cncf-io\") pod \"multus-9ssmv\" (UID: \"aa3f3b62-a1ab-4113-bef3-ed1b290e47c1\") " pod="openshift-multus/multus-9ssmv"
Oct 30 18:33:16.081249 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.079097    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/aa3f3b62-a1ab-4113-bef3-ed1b290e47c1-host-run-multus-certs\") pod \"multus-9ssmv\" (UID: \"aa3f3b62-a1ab-4113-bef3-ed1b290e47c1\") " pod="openshift-multus/multus-9ssmv"
Oct 30 18:33:16.081249 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.079121    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a0823e8f-bbdd-477b-95ac-dad05d23cefa-metrics-certs\") pod \"network-metrics-daemon-v5phn\" (UID: \"a0823e8f-bbdd-477b-95ac-dad05d23cefa\") " pod="openshift-multus/network-metrics-daemon-v5phn"
Oct 30 18:33:16.081249 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.079149    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/3f3fe764-2652-46c9-8e54-7882c0e6bcf1-env-overrides\") pod \"ovnkube-node-xfcdl\" (UID: \"3f3fe764-2652-46c9-8e54-7882c0e6bcf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xfcdl"
Oct 30 18:33:16.081249 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.079198    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f2a80d89-61d1-4694-8734-004ce5cfb22b-var-lib-kubelet\") pod \"tuned-z9jmw\" (UID: \"f2a80d89-61d1-4694-8734-004ce5cfb22b\") " pod="openshift-cluster-node-tuning-operator/tuned-z9jmw"
Oct 30 18:33:16.081249 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.079207    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3f3fe764-2652-46c9-8e54-7882c0e6bcf1-etc-openvswitch\") pod \"ovnkube-node-xfcdl\" (UID: \"3f3fe764-2652-46c9-8e54-7882c0e6bcf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xfcdl"
Oct 30 18:33:16.081249 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:16.079214    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 18:33:16.081249 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.079238    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wzxwr\" (UniqueName: \"kubernetes.io/projected/3f3fe764-2652-46c9-8e54-7882c0e6bcf1-kube-api-access-wzxwr\") pod \"ovnkube-node-xfcdl\" (UID: \"3f3fe764-2652-46c9-8e54-7882c0e6bcf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xfcdl"
Oct 30 18:33:16.081249 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:16.079312    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a0823e8f-bbdd-477b-95ac-dad05d23cefa-metrics-certs podName:a0823e8f-bbdd-477b-95ac-dad05d23cefa nodeName:}" failed. No retries permitted until 2026-10-30 18:33:16.579255964 +0000 UTC m=+3.070690428 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a0823e8f-bbdd-477b-95ac-dad05d23cefa-metrics-certs") pod "network-metrics-daemon-v5phn" (UID: "a0823e8f-bbdd-477b-95ac-dad05d23cefa") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 18:33:16.081249 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.079327    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3f3fe764-2652-46c9-8e54-7882c0e6bcf1-host-run-netns\") pod \"ovnkube-node-xfcdl\" (UID: \"3f3fe764-2652-46c9-8e54-7882c0e6bcf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xfcdl"
Oct 30 18:33:16.081249 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.079333    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ad78ad05-b0af-4adf-a6bc-0f993b8a6288-device-dir\") pod \"aws-ebs-csi-driver-node-7j9jw\" (UID: \"ad78ad05-b0af-4adf-a6bc-0f993b8a6288\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7j9jw"
Oct 30 18:33:16.081249 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.079364    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/aa3f3b62-a1ab-4113-bef3-ed1b290e47c1-system-cni-dir\") pod \"multus-9ssmv\" (UID: \"aa3f3b62-a1ab-4113-bef3-ed1b290e47c1\") " pod="openshift-multus/multus-9ssmv"
Oct 30 18:33:16.081249 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.079365    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f2a80d89-61d1-4694-8734-004ce5cfb22b-etc-sysctl-d\") pod \"tuned-z9jmw\" (UID: \"f2a80d89-61d1-4694-8734-004ce5cfb22b\") " pod="openshift-cluster-node-tuning-operator/tuned-z9jmw"
Oct 30 18:33:16.081249 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.079395    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/46a80637-6ccf-4a56-9904-f5e2fe5784ad-serviceca\") pod \"node-ca-g78lj\" (UID: \"46a80637-6ccf-4a56-9904-f5e2fe5784ad\") " pod="openshift-image-registry/node-ca-g78lj"
Oct 30 18:33:16.081249 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.079396    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/aa3f3b62-a1ab-4113-bef3-ed1b290e47c1-host-var-lib-kubelet\") pod \"multus-9ssmv\" (UID: \"aa3f3b62-a1ab-4113-bef3-ed1b290e47c1\") " pod="openshift-multus/multus-9ssmv"
Oct 30 18:33:16.081249 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.079416    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3f3fe764-2652-46c9-8e54-7882c0e6bcf1-host-run-ovn-kubernetes\") pod \"ovnkube-node-xfcdl\" (UID: \"3f3fe764-2652-46c9-8e54-7882c0e6bcf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xfcdl"
Oct 30 18:33:16.081249 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.079471    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rvrxn\" (UniqueName: \"kubernetes.io/projected/aa3f3b62-a1ab-4113-bef3-ed1b290e47c1-kube-api-access-rvrxn\") pod \"multus-9ssmv\" (UID: \"aa3f3b62-a1ab-4113-bef3-ed1b290e47c1\") " pod="openshift-multus/multus-9ssmv"
Oct 30 18:33:16.081249 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.079509    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/3f3fe764-2652-46c9-8e54-7882c0e6bcf1-systemd-units\") pod \"ovnkube-node-xfcdl\" (UID: \"3f3fe764-2652-46c9-8e54-7882c0e6bcf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xfcdl"
Oct 30 18:33:16.081727 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.079537    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/3f3fe764-2652-46c9-8e54-7882c0e6bcf1-run-systemd\") pod \"ovnkube-node-xfcdl\" (UID: \"3f3fe764-2652-46c9-8e54-7882c0e6bcf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xfcdl"
Oct 30 18:33:16.081727 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.079552    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/3f3fe764-2652-46c9-8e54-7882c0e6bcf1-systemd-units\") pod \"ovnkube-node-xfcdl\" (UID: \"3f3fe764-2652-46c9-8e54-7882c0e6bcf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xfcdl"
Oct 30 18:33:16.081727 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.079565    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ad78ad05-b0af-4adf-a6bc-0f993b8a6288-etc-selinux\") pod \"aws-ebs-csi-driver-node-7j9jw\" (UID: \"ad78ad05-b0af-4adf-a6bc-0f993b8a6288\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7j9jw"
Oct 30 18:33:16.081727 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.079583    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/3f3fe764-2652-46c9-8e54-7882c0e6bcf1-run-systemd\") pod \"ovnkube-node-xfcdl\" (UID: \"3f3fe764-2652-46c9-8e54-7882c0e6bcf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xfcdl"
Oct 30 18:33:16.081727 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.079592    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/46d9ac39-ee45-4e43-bda7-3931c3e599cf-original-pull-secret\") pod \"global-pull-secret-syncer-g2svp\" (UID: \"46d9ac39-ee45-4e43-bda7-3931c3e599cf\") " pod="kube-system/global-pull-secret-syncer-g2svp"
Oct 30 18:33:16.081727 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.079626    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2e49faaa-a718-44c8-9237-5a41169faee7-system-cni-dir\") pod \"multus-additional-cni-plugins-rzrjz\" (UID: \"2e49faaa-a718-44c8-9237-5a41169faee7\") " pod="openshift-multus/multus-additional-cni-plugins-rzrjz"
Oct 30 18:33:16.081727 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.079659    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2e49faaa-a718-44c8-9237-5a41169faee7-system-cni-dir\") pod \"multus-additional-cni-plugins-rzrjz\" (UID: \"2e49faaa-a718-44c8-9237-5a41169faee7\") " pod="openshift-multus/multus-additional-cni-plugins-rzrjz"
Oct 30 18:33:16.081727 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.079686    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2e49faaa-a718-44c8-9237-5a41169faee7-cnibin\") pod \"multus-additional-cni-plugins-rzrjz\" (UID: \"2e49faaa-a718-44c8-9237-5a41169faee7\") " pod="openshift-multus/multus-additional-cni-plugins-rzrjz"
Oct 30 18:33:16.081727 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.079709    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/2e49faaa-a718-44c8-9237-5a41169faee7-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-rzrjz\" (UID: \"2e49faaa-a718-44c8-9237-5a41169faee7\") " pod="openshift-multus/multus-additional-cni-plugins-rzrjz"
Oct 30 18:33:16.081727 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:16.079715    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 18:33:16.081727 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.079724    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bgxbg\" (UniqueName: \"kubernetes.io/projected/63090f48-9143-4118-9906-d759c0aff52c-kube-api-access-bgxbg\") pod \"iptables-alerter-rcjmv\" (UID: \"63090f48-9143-4118-9906-d759c0aff52c\") " pod="openshift-network-operator/iptables-alerter-rcjmv"
Oct 30 18:33:16.081727 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.079716    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2e49faaa-a718-44c8-9237-5a41169faee7-cnibin\") pod \"multus-additional-cni-plugins-rzrjz\" (UID: \"2e49faaa-a718-44c8-9237-5a41169faee7\") " pod="openshift-multus/multus-additional-cni-plugins-rzrjz"
Oct 30 18:33:16.081727 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.079746    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3f3fe764-2652-46c9-8e54-7882c0e6bcf1-var-lib-openvswitch\") pod \"ovnkube-node-xfcdl\" (UID: \"3f3fe764-2652-46c9-8e54-7882c0e6bcf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xfcdl"
Oct 30 18:33:16.081727 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:16.079762    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46d9ac39-ee45-4e43-bda7-3931c3e599cf-original-pull-secret podName:46d9ac39-ee45-4e43-bda7-3931c3e599cf nodeName:}" failed. No retries permitted until 2026-10-30 18:33:16.579746991 +0000 UTC m=+3.071181433 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/46d9ac39-ee45-4e43-bda7-3931c3e599cf-original-pull-secret") pod "global-pull-secret-syncer-g2svp" (UID: "46d9ac39-ee45-4e43-bda7-3931c3e599cf") : object "kube-system"/"original-pull-secret" not registered
Oct 30 18:33:16.081727 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.079771    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3f3fe764-2652-46c9-8e54-7882c0e6bcf1-var-lib-openvswitch\") pod \"ovnkube-node-xfcdl\" (UID: \"3f3fe764-2652-46c9-8e54-7882c0e6bcf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xfcdl"
Oct 30 18:33:16.081727 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.079789    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f2a80d89-61d1-4694-8734-004ce5cfb22b-run\") pod \"tuned-z9jmw\" (UID: \"f2a80d89-61d1-4694-8734-004ce5cfb22b\") " pod="openshift-cluster-node-tuning-operator/tuned-z9jmw"
Oct 30 18:33:16.082138 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.079818    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3f3fe764-2652-46c9-8e54-7882c0e6bcf1-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-xfcdl\" (UID: \"3f3fe764-2652-46c9-8e54-7882c0e6bcf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xfcdl"
Oct 30 18:33:16.082138 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.079842    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/3f3fe764-2652-46c9-8e54-7882c0e6bcf1-ovn-node-metrics-cert\") pod \"ovnkube-node-xfcdl\" (UID: \"3f3fe764-2652-46c9-8e54-7882c0e6bcf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xfcdl"
Oct 30 18:33:16.082138 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.079837    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f2a80d89-61d1-4694-8734-004ce5cfb22b-run\") pod \"tuned-z9jmw\" (UID: \"f2a80d89-61d1-4694-8734-004ce5cfb22b\") " pod="openshift-cluster-node-tuning-operator/tuned-z9jmw"
Oct 30 18:33:16.082138 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.079869    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3f3fe764-2652-46c9-8e54-7882c0e6bcf1-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-xfcdl\" (UID: \"3f3fe764-2652-46c9-8e54-7882c0e6bcf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xfcdl"
Oct 30 18:33:16.082138 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.081086    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f2a80d89-61d1-4694-8734-004ce5cfb22b-etc-tuned\") pod \"tuned-z9jmw\" (UID: \"f2a80d89-61d1-4694-8734-004ce5cfb22b\") " pod="openshift-cluster-node-tuning-operator/tuned-z9jmw"
Oct 30 18:33:16.082138 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.081204    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/8ecf2b62-8053-4bba-a852-c6f6119d3288-agent-certs\") pod \"konnectivity-agent-lmdpp\" (UID: \"8ecf2b62-8053-4bba-a852-c6f6119d3288\") " pod="kube-system/konnectivity-agent-lmdpp"
Oct 30 18:33:16.082138 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.081260    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f2a80d89-61d1-4694-8734-004ce5cfb22b-tmp\") pod \"tuned-z9jmw\" (UID: \"f2a80d89-61d1-4694-8734-004ce5cfb22b\") " pod="openshift-cluster-node-tuning-operator/tuned-z9jmw"
Oct 30 18:33:16.082138 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.081917    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/3f3fe764-2652-46c9-8e54-7882c0e6bcf1-ovn-node-metrics-cert\") pod \"ovnkube-node-xfcdl\" (UID: \"3f3fe764-2652-46c9-8e54-7882c0e6bcf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xfcdl"
Oct 30 18:33:16.101732 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.101699    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Oct 30 18:33:16.101732 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.101714    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Oct 30 18:33:16.101856 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.101735    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Oct 30 18:33:16.102072 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.102053    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Oct 30 18:33:16.102072 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.102063    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Oct 30 18:33:16.102072 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.102063    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-cr7s6\""
Oct 30 18:33:16.102072 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.102102    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Oct 30 18:33:16.102072 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.102114    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Oct 30 18:33:16.102072 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.102136    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Oct 30 18:33:16.102072 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.102158    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-cgz7p\""
Oct 30 18:33:16.102419 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.102158    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-4bpx7\""
Oct 30 18:33:16.102419 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.102193    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Oct 30 18:33:16.102419 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.102233    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-b7gnv\""
Oct 30 18:33:16.108708 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.108686    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/2e49faaa-a718-44c8-9237-5a41169faee7-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-rzrjz\" (UID: \"2e49faaa-a718-44c8-9237-5a41169faee7\") " pod="openshift-multus/multus-additional-cni-plugins-rzrjz"
Oct 30 18:33:16.110940 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.110917    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/2e49faaa-a718-44c8-9237-5a41169faee7-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-rzrjz\" (UID: \"2e49faaa-a718-44c8-9237-5a41169faee7\") " pod="openshift-multus/multus-additional-cni-plugins-rzrjz"
Oct 30 18:33:16.136440 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:16.136417    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 18:33:16.136440 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:16.136434    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 18:33:16.136440 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:16.136443    2574 projected.go:196] Error preparing data for projected volume kube-api-access-c45qr for pod openshift-network-diagnostics/network-check-target-8xskh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 18:33:16.136641 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:16.136519    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/613020c0-f3e4-4650-896f-748a40f0ab8b-kube-api-access-c45qr podName:613020c0-f3e4-4650-896f-748a40f0ab8b nodeName:}" failed. No retries permitted until 2026-10-30 18:33:16.63650742 +0000 UTC m=+3.127941889 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-c45qr" (UniqueName: "kubernetes.io/projected/613020c0-f3e4-4650-896f-748a40f0ab8b-kube-api-access-c45qr") pod "network-check-target-8xskh" (UID: "613020c0-f3e4-4650-896f-748a40f0ab8b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 18:33:16.138191 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.138167    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t56q5\" (UniqueName: \"kubernetes.io/projected/a0823e8f-bbdd-477b-95ac-dad05d23cefa-kube-api-access-t56q5\") pod \"network-metrics-daemon-v5phn\" (UID: \"a0823e8f-bbdd-477b-95ac-dad05d23cefa\") " pod="openshift-multus/network-metrics-daemon-v5phn"
Oct 30 18:33:16.138546 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.138522    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x2978\" (UniqueName: \"kubernetes.io/projected/46a80637-6ccf-4a56-9904-f5e2fe5784ad-kube-api-access-x2978\") pod \"node-ca-g78lj\" (UID: \"46a80637-6ccf-4a56-9904-f5e2fe5784ad\") " pod="openshift-image-registry/node-ca-g78lj"
Oct 30 18:33:16.139159 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.139132    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6llht\" (UniqueName: \"kubernetes.io/projected/2e49faaa-a718-44c8-9237-5a41169faee7-kube-api-access-6llht\") pod \"multus-additional-cni-plugins-rzrjz\" (UID: \"2e49faaa-a718-44c8-9237-5a41169faee7\") " pod="openshift-multus/multus-additional-cni-plugins-rzrjz"
Oct 30 18:33:16.139247 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.139224    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8wvhh\" (UniqueName: \"kubernetes.io/projected/f2a80d89-61d1-4694-8734-004ce5cfb22b-kube-api-access-8wvhh\") pod \"tuned-z9jmw\" (UID: \"f2a80d89-61d1-4694-8734-004ce5cfb22b\") " pod="openshift-cluster-node-tuning-operator/tuned-z9jmw"
Oct 30 18:33:16.139537 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.139522    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bgxbg\" (UniqueName: \"kubernetes.io/projected/63090f48-9143-4118-9906-d759c0aff52c-kube-api-access-bgxbg\") pod \"iptables-alerter-rcjmv\" (UID: \"63090f48-9143-4118-9906-d759c0aff52c\") " pod="openshift-network-operator/iptables-alerter-rcjmv"
Oct 30 18:33:16.139775 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.139760    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wzxwr\" (UniqueName: \"kubernetes.io/projected/3f3fe764-2652-46c9-8e54-7882c0e6bcf1-kube-api-access-wzxwr\") pod \"ovnkube-node-xfcdl\" (UID: \"3f3fe764-2652-46c9-8e54-7882c0e6bcf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xfcdl"
Oct 30 18:33:16.180185 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.180158    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/aa3f3b62-a1ab-4113-bef3-ed1b290e47c1-host-run-k8s-cni-cncf-io\") pod \"multus-9ssmv\" (UID: \"aa3f3b62-a1ab-4113-bef3-ed1b290e47c1\") " pod="openshift-multus/multus-9ssmv"
Oct 30 18:33:16.180298 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.180188    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/aa3f3b62-a1ab-4113-bef3-ed1b290e47c1-host-run-multus-certs\") pod \"multus-9ssmv\" (UID: \"aa3f3b62-a1ab-4113-bef3-ed1b290e47c1\") " pod="openshift-multus/multus-9ssmv"
Oct 30 18:33:16.180298 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.180227    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ad78ad05-b0af-4adf-a6bc-0f993b8a6288-device-dir\") pod \"aws-ebs-csi-driver-node-7j9jw\" (UID: \"ad78ad05-b0af-4adf-a6bc-0f993b8a6288\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7j9jw"
Oct 30 18:33:16.180298 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.180267    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/aa3f3b62-a1ab-4113-bef3-ed1b290e47c1-host-run-multus-certs\") pod \"multus-9ssmv\" (UID: \"aa3f3b62-a1ab-4113-bef3-ed1b290e47c1\") " pod="openshift-multus/multus-9ssmv"
Oct 30 18:33:16.180424 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.180267    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/aa3f3b62-a1ab-4113-bef3-ed1b290e47c1-host-run-k8s-cni-cncf-io\") pod \"multus-9ssmv\" (UID: \"aa3f3b62-a1ab-4113-bef3-ed1b290e47c1\") " pod="openshift-multus/multus-9ssmv"
Oct 30 18:33:16.180424 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.180272    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/aa3f3b62-a1ab-4113-bef3-ed1b290e47c1-system-cni-dir\") pod \"multus-9ssmv\" (UID: \"aa3f3b62-a1ab-4113-bef3-ed1b290e47c1\") " pod="openshift-multus/multus-9ssmv"
Oct 30 18:33:16.180424 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.180343    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ad78ad05-b0af-4adf-a6bc-0f993b8a6288-device-dir\") pod \"aws-ebs-csi-driver-node-7j9jw\" (UID: \"ad78ad05-b0af-4adf-a6bc-0f993b8a6288\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7j9jw"
Oct 30 18:33:16.180424 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.180349    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/aa3f3b62-a1ab-4113-bef3-ed1b290e47c1-system-cni-dir\") pod \"multus-9ssmv\" (UID: \"aa3f3b62-a1ab-4113-bef3-ed1b290e47c1\") " pod="openshift-multus/multus-9ssmv"
Oct 30 18:33:16.180424 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.180354    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/aa3f3b62-a1ab-4113-bef3-ed1b290e47c1-host-var-lib-kubelet\") pod \"multus-9ssmv\" (UID: \"aa3f3b62-a1ab-4113-bef3-ed1b290e47c1\") " pod="openshift-multus/multus-9ssmv"
Oct 30 18:33:16.180424 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.180371    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rvrxn\" (UniqueName: \"kubernetes.io/projected/aa3f3b62-a1ab-4113-bef3-ed1b290e47c1-kube-api-access-rvrxn\") pod \"multus-9ssmv\" (UID: \"aa3f3b62-a1ab-4113-bef3-ed1b290e47c1\") " pod="openshift-multus/multus-9ssmv"
Oct 30 18:33:16.180424 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.180389    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ad78ad05-b0af-4adf-a6bc-0f993b8a6288-etc-selinux\") pod \"aws-ebs-csi-driver-node-7j9jw\" (UID: \"ad78ad05-b0af-4adf-a6bc-0f993b8a6288\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7j9jw"
Oct 30 18:33:16.180721 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.180430    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/aa3f3b62-a1ab-4113-bef3-ed1b290e47c1-host-var-lib-cni-bin\") pod \"multus-9ssmv\" (UID: \"aa3f3b62-a1ab-4113-bef3-ed1b290e47c1\") " pod="openshift-multus/multus-9ssmv"
Oct 30 18:33:16.180721 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.180446    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/aa3f3b62-a1ab-4113-bef3-ed1b290e47c1-host-var-lib-kubelet\") pod \"multus-9ssmv\" (UID: \"aa3f3b62-a1ab-4113-bef3-ed1b290e47c1\") " pod="openshift-multus/multus-9ssmv"
Oct 30 18:33:16.180721 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.180456    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ad78ad05-b0af-4adf-a6bc-0f993b8a6288-socket-dir\") pod \"aws-ebs-csi-driver-node-7j9jw\" (UID: \"ad78ad05-b0af-4adf-a6bc-0f993b8a6288\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7j9jw"
Oct 30 18:33:16.180721 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.180493    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/aa3f3b62-a1ab-4113-bef3-ed1b290e47c1-host-var-lib-cni-bin\") pod \"multus-9ssmv\" (UID: \"aa3f3b62-a1ab-4113-bef3-ed1b290e47c1\") " pod="openshift-multus/multus-9ssmv"
Oct 30 18:33:16.180721 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.180494    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ad78ad05-b0af-4adf-a6bc-0f993b8a6288-etc-selinux\") pod \"aws-ebs-csi-driver-node-7j9jw\" (UID: \"ad78ad05-b0af-4adf-a6bc-0f993b8a6288\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7j9jw"
Oct 30 18:33:16.180721 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.180521    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/aa3f3b62-a1ab-4113-bef3-ed1b290e47c1-multus-conf-dir\") pod \"multus-9ssmv\" (UID: \"aa3f3b62-a1ab-4113-bef3-ed1b290e47c1\") " pod="openshift-multus/multus-9ssmv"
Oct 30 18:33:16.180721 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.180562    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5f2ls\" (UniqueName: \"kubernetes.io/projected/ad78ad05-b0af-4adf-a6bc-0f993b8a6288-kube-api-access-5f2ls\") pod \"aws-ebs-csi-driver-node-7j9jw\" (UID: \"ad78ad05-b0af-4adf-a6bc-0f993b8a6288\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7j9jw"
Oct 30 18:33:16.180721 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.180566    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ad78ad05-b0af-4adf-a6bc-0f993b8a6288-socket-dir\") pod \"aws-ebs-csi-driver-node-7j9jw\" (UID: \"ad78ad05-b0af-4adf-a6bc-0f993b8a6288\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7j9jw"
Oct 30 18:33:16.180721 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.180588    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/aa3f3b62-a1ab-4113-bef3-ed1b290e47c1-hostroot\") pod \"multus-9ssmv\" (UID: \"aa3f3b62-a1ab-4113-bef3-ed1b290e47c1\") " pod="openshift-multus/multus-9ssmv"
Oct 30 18:33:16.180721 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.180598    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/aa3f3b62-a1ab-4113-bef3-ed1b290e47c1-multus-conf-dir\") pod \"multus-9ssmv\" (UID: \"aa3f3b62-a1ab-4113-bef3-ed1b290e47c1\") " pod="openshift-multus/multus-9ssmv"
Oct 30 18:33:16.180721 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.180643    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ad78ad05-b0af-4adf-a6bc-0f993b8a6288-sys-fs\") pod \"aws-ebs-csi-driver-node-7j9jw\" (UID: \"ad78ad05-b0af-4adf-a6bc-0f993b8a6288\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7j9jw"
Oct 30 18:33:16.180721 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.180646    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/aa3f3b62-a1ab-4113-bef3-ed1b290e47c1-hostroot\") pod \"multus-9ssmv\" (UID: \"aa3f3b62-a1ab-4113-bef3-ed1b290e47c1\") " pod="openshift-multus/multus-9ssmv"
Oct 30 18:33:16.180721 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.180665    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ad78ad05-b0af-4adf-a6bc-0f993b8a6288-kubelet-dir\") pod \"aws-ebs-csi-driver-node-7j9jw\" (UID: \"ad78ad05-b0af-4adf-a6bc-0f993b8a6288\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7j9jw"
Oct 30 18:33:16.180721 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.180685    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gzhnk\" (UniqueName: \"kubernetes.io/projected/d81cef55-16ee-4677-ae12-86de3c9eb216-kube-api-access-gzhnk\") pod \"node-resolver-76f9s\" (UID: \"d81cef55-16ee-4677-ae12-86de3c9eb216\") " pod="openshift-dns/node-resolver-76f9s"
Oct 30 18:33:16.180721 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.180698    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ad78ad05-b0af-4adf-a6bc-0f993b8a6288-sys-fs\") pod \"aws-ebs-csi-driver-node-7j9jw\" (UID: \"ad78ad05-b0af-4adf-a6bc-0f993b8a6288\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7j9jw"
Oct 30 18:33:16.180721 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.180700    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ad78ad05-b0af-4adf-a6bc-0f993b8a6288-kubelet-dir\") pod \"aws-ebs-csi-driver-node-7j9jw\" (UID: \"ad78ad05-b0af-4adf-a6bc-0f993b8a6288\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7j9jw"
Oct 30 18:33:16.180721 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.180704    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/aa3f3b62-a1ab-4113-bef3-ed1b290e47c1-multus-cni-dir\") pod \"multus-9ssmv\" (UID: \"aa3f3b62-a1ab-4113-bef3-ed1b290e47c1\") " pod="openshift-multus/multus-9ssmv"
Oct 30 18:33:16.181372 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.180732    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/aa3f3b62-a1ab-4113-bef3-ed1b290e47c1-multus-cni-dir\") pod \"multus-9ssmv\" (UID: \"aa3f3b62-a1ab-4113-bef3-ed1b290e47c1\") " pod="openshift-multus/multus-9ssmv"
Oct 30 18:33:16.181372 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.180738    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ad78ad05-b0af-4adf-a6bc-0f993b8a6288-registration-dir\") pod \"aws-ebs-csi-driver-node-7j9jw\" (UID: \"ad78ad05-b0af-4adf-a6bc-0f993b8a6288\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7j9jw"
Oct 30 18:33:16.181372 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.180763    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/aa3f3b62-a1ab-4113-bef3-ed1b290e47c1-cnibin\") pod \"multus-9ssmv\" (UID: \"aa3f3b62-a1ab-4113-bef3-ed1b290e47c1\") " pod="openshift-multus/multus-9ssmv"
Oct 30 18:33:16.181372 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.180780    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/aa3f3b62-a1ab-4113-bef3-ed1b290e47c1-host-var-lib-cni-multus\") pod \"multus-9ssmv\" (UID: \"aa3f3b62-a1ab-4113-bef3-ed1b290e47c1\") " pod="openshift-multus/multus-9ssmv"
Oct 30 18:33:16.181372 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.180794    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/aa3f3b62-a1ab-4113-bef3-ed1b290e47c1-etc-kubernetes\") pod \"multus-9ssmv\" (UID: \"aa3f3b62-a1ab-4113-bef3-ed1b290e47c1\") " pod="openshift-multus/multus-9ssmv"
Oct 30 18:33:16.181372 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.180795    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ad78ad05-b0af-4adf-a6bc-0f993b8a6288-registration-dir\") pod \"aws-ebs-csi-driver-node-7j9jw\" (UID: \"ad78ad05-b0af-4adf-a6bc-0f993b8a6288\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7j9jw"
Oct 30 18:33:16.181372 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.180805    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/aa3f3b62-a1ab-4113-bef3-ed1b290e47c1-cnibin\") pod \"multus-9ssmv\" (UID: \"aa3f3b62-a1ab-4113-bef3-ed1b290e47c1\") " pod="openshift-multus/multus-9ssmv"
Oct 30 18:33:16.181372 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.180811    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d81cef55-16ee-4677-ae12-86de3c9eb216-hosts-file\") pod \"node-resolver-76f9s\" (UID: \"d81cef55-16ee-4677-ae12-86de3c9eb216\") " pod="openshift-dns/node-resolver-76f9s"
Oct 30 18:33:16.181372 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.180821    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/aa3f3b62-a1ab-4113-bef3-ed1b290e47c1-host-var-lib-cni-multus\") pod \"multus-9ssmv\" (UID: \"aa3f3b62-a1ab-4113-bef3-ed1b290e47c1\") " pod="openshift-multus/multus-9ssmv"
Oct 30 18:33:16.181372 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.180831    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d81cef55-16ee-4677-ae12-86de3c9eb216-tmp-dir\") pod \"node-resolver-76f9s\" (UID: \"d81cef55-16ee-4677-ae12-86de3c9eb216\") " pod="openshift-dns/node-resolver-76f9s"
Oct 30 18:33:16.181372 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.180863    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/aa3f3b62-a1ab-4113-bef3-ed1b290e47c1-etc-kubernetes\") pod \"multus-9ssmv\" (UID: \"aa3f3b62-a1ab-4113-bef3-ed1b290e47c1\") " pod="openshift-multus/multus-9ssmv"
Oct 30 18:33:16.181372 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.180867    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/aa3f3b62-a1ab-4113-bef3-ed1b290e47c1-cni-binary-copy\") pod \"multus-9ssmv\" (UID: \"aa3f3b62-a1ab-4113-bef3-ed1b290e47c1\") " pod="openshift-multus/multus-9ssmv"
Oct 30 18:33:16.181372 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.180874    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d81cef55-16ee-4677-ae12-86de3c9eb216-hosts-file\") pod \"node-resolver-76f9s\" (UID: \"d81cef55-16ee-4677-ae12-86de3c9eb216\") " pod="openshift-dns/node-resolver-76f9s"
Oct 30 18:33:16.181372 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.180909    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/aa3f3b62-a1ab-4113-bef3-ed1b290e47c1-multus-socket-dir-parent\") pod \"multus-9ssmv\" (UID: \"aa3f3b62-a1ab-4113-bef3-ed1b290e47c1\") " pod="openshift-multus/multus-9ssmv"
Oct 30 18:33:16.181372 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.180933    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/aa3f3b62-a1ab-4113-bef3-ed1b290e47c1-host-run-netns\") pod \"multus-9ssmv\" (UID: \"aa3f3b62-a1ab-4113-bef3-ed1b290e47c1\") " pod="openshift-multus/multus-9ssmv"
Oct 30 18:33:16.181372 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.180948    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/aa3f3b62-a1ab-4113-bef3-ed1b290e47c1-multus-daemon-config\") pod \"multus-9ssmv\" (UID: \"aa3f3b62-a1ab-4113-bef3-ed1b290e47c1\") " pod="openshift-multus/multus-9ssmv"
Oct 30 18:33:16.181372 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.180967    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/aa3f3b62-a1ab-4113-bef3-ed1b290e47c1-multus-socket-dir-parent\") pod \"multus-9ssmv\" (UID: \"aa3f3b62-a1ab-4113-bef3-ed1b290e47c1\") " pod="openshift-multus/multus-9ssmv"
Oct 30 18:33:16.181372 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.180990    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/aa3f3b62-a1ab-4113-bef3-ed1b290e47c1-host-run-netns\") pod \"multus-9ssmv\" (UID: \"aa3f3b62-a1ab-4113-bef3-ed1b290e47c1\") " pod="openshift-multus/multus-9ssmv"
Oct 30 18:33:16.181837 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.181158    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d81cef55-16ee-4677-ae12-86de3c9eb216-tmp-dir\") pod \"node-resolver-76f9s\" (UID: \"d81cef55-16ee-4677-ae12-86de3c9eb216\") " pod="openshift-dns/node-resolver-76f9s"
Oct 30 18:33:16.181837 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.181323    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/aa3f3b62-a1ab-4113-bef3-ed1b290e47c1-multus-daemon-config\") pod \"multus-9ssmv\" (UID: \"aa3f3b62-a1ab-4113-bef3-ed1b290e47c1\") " pod="openshift-multus/multus-9ssmv"
Oct 30 18:33:16.181837 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.181332    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/aa3f3b62-a1ab-4113-bef3-ed1b290e47c1-cni-binary-copy\") pod \"multus-9ssmv\" (UID: \"aa3f3b62-a1ab-4113-bef3-ed1b290e47c1\") " pod="openshift-multus/multus-9ssmv"
Oct 30 18:33:16.241321 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.241277    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gzhnk\" (UniqueName: \"kubernetes.io/projected/d81cef55-16ee-4677-ae12-86de3c9eb216-kube-api-access-gzhnk\") pod \"node-resolver-76f9s\" (UID: \"d81cef55-16ee-4677-ae12-86de3c9eb216\") " pod="openshift-dns/node-resolver-76f9s"
Oct 30 18:33:16.241523 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.241507    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rvrxn\" (UniqueName: \"kubernetes.io/projected/aa3f3b62-a1ab-4113-bef3-ed1b290e47c1-kube-api-access-rvrxn\") pod \"multus-9ssmv\" (UID: \"aa3f3b62-a1ab-4113-bef3-ed1b290e47c1\") " pod="openshift-multus/multus-9ssmv"
Oct 30 18:33:16.241601 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.241586    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5f2ls\" (UniqueName: \"kubernetes.io/projected/ad78ad05-b0af-4adf-a6bc-0f993b8a6288-kube-api-access-5f2ls\") pod \"aws-ebs-csi-driver-node-7j9jw\" (UID: \"ad78ad05-b0af-4adf-a6bc-0f993b8a6288\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7j9jw"
Oct 30 18:33:16.259317 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.259265    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-g78lj"
Oct 30 18:33:16.265563 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.265538    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-rcjmv"
Oct 30 18:33:16.269734 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:16.269713    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46a80637_6ccf_4a56_9904_f5e2fe5784ad.slice/crio-9766c5bb2416907e5b0313e177e5075e6f6480fc8b492fcfa2d287d15133f64e WatchSource:0}: Error finding container 9766c5bb2416907e5b0313e177e5075e6f6480fc8b492fcfa2d287d15133f64e: Status 404 returned error can't find the container with id 9766c5bb2416907e5b0313e177e5075e6f6480fc8b492fcfa2d287d15133f64e
Oct 30 18:33:16.273610 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:16.273591    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod63090f48_9143_4118_9906_d759c0aff52c.slice/crio-a0d10b0331467cc7d179bb0e0cb083d789035c89166c8c5f594f142864c45f4c WatchSource:0}: Error finding container a0d10b0331467cc7d179bb0e0cb083d789035c89166c8c5f594f142864c45f4c: Status 404 returned error can't find the container with id a0d10b0331467cc7d179bb0e0cb083d789035c89166c8c5f594f142864c45f4c
Oct 30 18:33:16.283267 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.283251    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-z9jmw"
Oct 30 18:33:16.284371 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.284349    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-lmdpp"
Oct 30 18:33:16.291047 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.291027    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-xfcdl"
Oct 30 18:33:16.292306 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:16.292271    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2a80d89_61d1_4694_8734_004ce5cfb22b.slice/crio-9d10e6bc49abbcebfd08bc1e4ce102c4e145dc9ad9ca45f9e0044b0893c4f32d WatchSource:0}: Error finding container 9d10e6bc49abbcebfd08bc1e4ce102c4e145dc9ad9ca45f9e0044b0893c4f32d: Status 404 returned error can't find the container with id 9d10e6bc49abbcebfd08bc1e4ce102c4e145dc9ad9ca45f9e0044b0893c4f32d
Oct 30 18:33:16.298529 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.298351    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-76f9s"
Oct 30 18:33:16.306098 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:16.306076    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f3fe764_2652_46c9_8e54_7882c0e6bcf1.slice/crio-0698cafab13d74adaa3fd8179e2b7ec62ff9ce956db4148cafedadbb6d4d553c WatchSource:0}: Error finding container 0698cafab13d74adaa3fd8179e2b7ec62ff9ce956db4148cafedadbb6d4d553c: Status 404 returned error can't find the container with id 0698cafab13d74adaa3fd8179e2b7ec62ff9ce956db4148cafedadbb6d4d553c
Oct 30 18:33:16.306691 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:16.306670    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd81cef55_16ee_4677_ae12_86de3c9eb216.slice/crio-010820c82deec18c7b4002e46b65e8dab8cc3c9c8cd27ac759462efc6c4c32ac WatchSource:0}: Error finding container 010820c82deec18c7b4002e46b65e8dab8cc3c9c8cd27ac759462efc6c4c32ac: Status 404 returned error can't find the container with id 010820c82deec18c7b4002e46b65e8dab8cc3c9c8cd27ac759462efc6c4c32ac
Oct 30 18:33:16.308524 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.308512    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-9ssmv"
Oct 30 18:33:16.313088 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.313068    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7j9jw"
Oct 30 18:33:16.315234 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.315217    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-rzrjz"
Oct 30 18:33:16.316924 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:16.316906    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa3f3b62_a1ab_4113_bef3_ed1b290e47c1.slice/crio-8f3fb203f2d1c8bf14c057244ee2331e339db7c0ab0f9a744f96acb79a075483 WatchSource:0}: Error finding container 8f3fb203f2d1c8bf14c057244ee2331e339db7c0ab0f9a744f96acb79a075483: Status 404 returned error can't find the container with id 8f3fb203f2d1c8bf14c057244ee2331e339db7c0ab0f9a744f96acb79a075483
Oct 30 18:33:16.322110 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:16.322093    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad78ad05_b0af_4adf_a6bc_0f993b8a6288.slice/crio-add4754d17d14d035566b3612bce0fd00489041c05d84daf0553c4e983484ba6 WatchSource:0}: Error finding container add4754d17d14d035566b3612bce0fd00489041c05d84daf0553c4e983484ba6: Status 404 returned error can't find the container with id add4754d17d14d035566b3612bce0fd00489041c05d84daf0553c4e983484ba6
Oct 30 18:33:16.323941 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:16.323922    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e49faaa_a718_44c8_9237_5a41169faee7.slice/crio-e1430cbb57146c9fef763300e771803f7b3cb39a5af4e3bbf5c2b1ea173d02d7 WatchSource:0}: Error finding container e1430cbb57146c9fef763300e771803f7b3cb39a5af4e3bbf5c2b1ea173d02d7: Status 404 returned error can't find the container with id e1430cbb57146c9fef763300e771803f7b3cb39a5af4e3bbf5c2b1ea173d02d7
Oct 30 18:33:16.583945 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.583862    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a0823e8f-bbdd-477b-95ac-dad05d23cefa-metrics-certs\") pod \"network-metrics-daemon-v5phn\" (UID: \"a0823e8f-bbdd-477b-95ac-dad05d23cefa\") " pod="openshift-multus/network-metrics-daemon-v5phn"
Oct 30 18:33:16.583945 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.583895    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/46d9ac39-ee45-4e43-bda7-3931c3e599cf-original-pull-secret\") pod \"global-pull-secret-syncer-g2svp\" (UID: \"46d9ac39-ee45-4e43-bda7-3931c3e599cf\") " pod="kube-system/global-pull-secret-syncer-g2svp"
Oct 30 18:33:16.584138 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:16.583992    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 18:33:16.584138 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:16.583992    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 18:33:16.584138 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:16.584046    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46d9ac39-ee45-4e43-bda7-3931c3e599cf-original-pull-secret podName:46d9ac39-ee45-4e43-bda7-3931c3e599cf nodeName:}" failed. No retries permitted until 2026-10-30 18:33:17.584029907 +0000 UTC m=+4.075464346 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/46d9ac39-ee45-4e43-bda7-3931c3e599cf-original-pull-secret") pod "global-pull-secret-syncer-g2svp" (UID: "46d9ac39-ee45-4e43-bda7-3931c3e599cf") : object "kube-system"/"original-pull-secret" not registered
Oct 30 18:33:16.584138 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:16.584063    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a0823e8f-bbdd-477b-95ac-dad05d23cefa-metrics-certs podName:a0823e8f-bbdd-477b-95ac-dad05d23cefa nodeName:}" failed. No retries permitted until 2026-10-30 18:33:17.584054247 +0000 UTC m=+4.075488688 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a0823e8f-bbdd-477b-95ac-dad05d23cefa-metrics-certs") pod "network-metrics-daemon-v5phn" (UID: "a0823e8f-bbdd-477b-95ac-dad05d23cefa") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 18:33:16.684511 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.684465    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c45qr\" (UniqueName: \"kubernetes.io/projected/613020c0-f3e4-4650-896f-748a40f0ab8b-kube-api-access-c45qr\") pod \"network-check-target-8xskh\" (UID: \"613020c0-f3e4-4650-896f-748a40f0ab8b\") " pod="openshift-network-diagnostics/network-check-target-8xskh"
Oct 30 18:33:16.684682 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:16.684612    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 18:33:16.684682 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:16.684632    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 18:33:16.684682 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:16.684644    2574 projected.go:196] Error preparing data for projected volume kube-api-access-c45qr for pod openshift-network-diagnostics/network-check-target-8xskh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 18:33:16.684805 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:16.684705    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/613020c0-f3e4-4650-896f-748a40f0ab8b-kube-api-access-c45qr podName:613020c0-f3e4-4650-896f-748a40f0ab8b nodeName:}" failed. No retries permitted until 2026-10-30 18:33:17.684686363 +0000 UTC m=+4.176120817 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-c45qr" (UniqueName: "kubernetes.io/projected/613020c0-f3e4-4650-896f-748a40f0ab8b-kube-api-access-c45qr") pod "network-check-target-8xskh" (UID: "613020c0-f3e4-4650-896f-748a40f0ab8b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 18:33:16.947232 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.947153    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7j9jw" event={"ID":"ad78ad05-b0af-4adf-a6bc-0f993b8a6288","Type":"ContainerStarted","Data":"add4754d17d14d035566b3612bce0fd00489041c05d84daf0553c4e983484ba6"}
Oct 30 18:33:16.948529 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.948505    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-9ssmv" event={"ID":"aa3f3b62-a1ab-4113-bef3-ed1b290e47c1","Type":"ContainerStarted","Data":"8f3fb203f2d1c8bf14c057244ee2331e339db7c0ab0f9a744f96acb79a075483"}
Oct 30 18:33:16.949808 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.949777    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xfcdl" event={"ID":"3f3fe764-2652-46c9-8e54-7882c0e6bcf1","Type":"ContainerStarted","Data":"0698cafab13d74adaa3fd8179e2b7ec62ff9ce956db4148cafedadbb6d4d553c"}
Oct 30 18:33:16.950983 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.950959    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-lmdpp" event={"ID":"8ecf2b62-8053-4bba-a852-c6f6119d3288","Type":"ContainerStarted","Data":"a6818d98d464d242de00bec5b3468e251eaafeecc0329e9b18c563db7d7b4ba8"}
Oct 30 18:33:16.952150 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.952130    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-z9jmw" event={"ID":"f2a80d89-61d1-4694-8734-004ce5cfb22b","Type":"ContainerStarted","Data":"9d10e6bc49abbcebfd08bc1e4ce102c4e145dc9ad9ca45f9e0044b0893c4f32d"}
Oct 30 18:33:16.953147 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.953124    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-rcjmv" event={"ID":"63090f48-9143-4118-9906-d759c0aff52c","Type":"ContainerStarted","Data":"a0d10b0331467cc7d179bb0e0cb083d789035c89166c8c5f594f142864c45f4c"}
Oct 30 18:33:16.954263 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.954246    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rzrjz" event={"ID":"2e49faaa-a718-44c8-9237-5a41169faee7","Type":"ContainerStarted","Data":"e1430cbb57146c9fef763300e771803f7b3cb39a5af4e3bbf5c2b1ea173d02d7"}
Oct 30 18:33:16.955410 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.955382    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-76f9s" event={"ID":"d81cef55-16ee-4677-ae12-86de3c9eb216","Type":"ContainerStarted","Data":"010820c82deec18c7b4002e46b65e8dab8cc3c9c8cd27ac759462efc6c4c32ac"}
Oct 30 18:33:16.956495 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:16.956467    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-g78lj" event={"ID":"46a80637-6ccf-4a56-9904-f5e2fe5784ad","Type":"ContainerStarted","Data":"9766c5bb2416907e5b0313e177e5075e6f6480fc8b492fcfa2d287d15133f64e"}
Oct 30 18:33:17.052332 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:17.052273    2574 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-29 18:28:15 +0000 UTC" deadline="2028-07-22 21:39:35.792280832 +0000 UTC"
Oct 30 18:33:17.052332 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:17.052327    2574 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15147h6m18.739958699s"
Oct 30 18:33:17.593017 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:17.592943    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a0823e8f-bbdd-477b-95ac-dad05d23cefa-metrics-certs\") pod \"network-metrics-daemon-v5phn\" (UID: \"a0823e8f-bbdd-477b-95ac-dad05d23cefa\") " pod="openshift-multus/network-metrics-daemon-v5phn"
Oct 30 18:33:17.593017 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:17.592977    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/46d9ac39-ee45-4e43-bda7-3931c3e599cf-original-pull-secret\") pod \"global-pull-secret-syncer-g2svp\" (UID: \"46d9ac39-ee45-4e43-bda7-3931c3e599cf\") " pod="kube-system/global-pull-secret-syncer-g2svp"
Oct 30 18:33:17.593243 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:17.593078    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 18:33:17.593243 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:17.593119    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46d9ac39-ee45-4e43-bda7-3931c3e599cf-original-pull-secret podName:46d9ac39-ee45-4e43-bda7-3931c3e599cf nodeName:}" failed. No retries permitted until 2026-10-30 18:33:19.593107336 +0000 UTC m=+6.084541789 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/46d9ac39-ee45-4e43-bda7-3931c3e599cf-original-pull-secret") pod "global-pull-secret-syncer-g2svp" (UID: "46d9ac39-ee45-4e43-bda7-3931c3e599cf") : object "kube-system"/"original-pull-secret" not registered
Oct 30 18:33:17.593243 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:17.593163    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 18:33:17.593243 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:17.593186    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a0823e8f-bbdd-477b-95ac-dad05d23cefa-metrics-certs podName:a0823e8f-bbdd-477b-95ac-dad05d23cefa nodeName:}" failed. No retries permitted until 2026-10-30 18:33:19.593176224 +0000 UTC m=+6.084610680 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a0823e8f-bbdd-477b-95ac-dad05d23cefa-metrics-certs") pod "network-metrics-daemon-v5phn" (UID: "a0823e8f-bbdd-477b-95ac-dad05d23cefa") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 18:33:17.693476 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:17.693440    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c45qr\" (UniqueName: \"kubernetes.io/projected/613020c0-f3e4-4650-896f-748a40f0ab8b-kube-api-access-c45qr\") pod \"network-check-target-8xskh\" (UID: \"613020c0-f3e4-4650-896f-748a40f0ab8b\") " pod="openshift-network-diagnostics/network-check-target-8xskh"
Oct 30 18:33:17.693655 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:17.693621    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 18:33:17.693655 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:17.693640    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 18:33:17.693770 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:17.693678    2574 projected.go:196] Error preparing data for projected volume kube-api-access-c45qr for pod openshift-network-diagnostics/network-check-target-8xskh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 18:33:17.693770 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:17.693733    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/613020c0-f3e4-4650-896f-748a40f0ab8b-kube-api-access-c45qr podName:613020c0-f3e4-4650-896f-748a40f0ab8b nodeName:}" failed. No retries permitted until 2026-10-30 18:33:19.693715218 +0000 UTC m=+6.185149672 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-c45qr" (UniqueName: "kubernetes.io/projected/613020c0-f3e4-4650-896f-748a40f0ab8b-kube-api-access-c45qr") pod "network-check-target-8xskh" (UID: "613020c0-f3e4-4650-896f-748a40f0ab8b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 18:33:17.939853 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:17.939413    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v5phn"
Oct 30 18:33:17.939853 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:17.939459    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8xskh"
Oct 30 18:33:17.939853 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:17.939544    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v5phn" podUID="a0823e8f-bbdd-477b-95ac-dad05d23cefa"
Oct 30 18:33:17.939853 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:17.939651    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8xskh" podUID="613020c0-f3e4-4650-896f-748a40f0ab8b"
Oct 30 18:33:17.939853 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:17.939703    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g2svp"
Oct 30 18:33:17.939853 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:17.939782    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g2svp" podUID="46d9ac39-ee45-4e43-bda7-3931c3e599cf"
Oct 30 18:33:18.053512 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:18.053438    2574 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-29 18:28:15 +0000 UTC" deadline="2028-06-16 12:41:12.603627673 +0000 UTC"
Oct 30 18:33:18.053512 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:18.053484    2574 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14274h7m54.550146516s"
Oct 30 18:33:19.611695 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:19.611149    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a0823e8f-bbdd-477b-95ac-dad05d23cefa-metrics-certs\") pod \"network-metrics-daemon-v5phn\" (UID: \"a0823e8f-bbdd-477b-95ac-dad05d23cefa\") " pod="openshift-multus/network-metrics-daemon-v5phn"
Oct 30 18:33:19.611695 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:19.611447    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/46d9ac39-ee45-4e43-bda7-3931c3e599cf-original-pull-secret\") pod \"global-pull-secret-syncer-g2svp\" (UID: \"46d9ac39-ee45-4e43-bda7-3931c3e599cf\") " pod="kube-system/global-pull-secret-syncer-g2svp"
Oct 30 18:33:19.611695 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:19.611372    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 18:33:19.611695 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:19.611605    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 18:33:19.611695 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:19.611629    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a0823e8f-bbdd-477b-95ac-dad05d23cefa-metrics-certs podName:a0823e8f-bbdd-477b-95ac-dad05d23cefa nodeName:}" failed. No retries permitted until 2026-10-30 18:33:23.611606294 +0000 UTC m=+10.103040748 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a0823e8f-bbdd-477b-95ac-dad05d23cefa-metrics-certs") pod "network-metrics-daemon-v5phn" (UID: "a0823e8f-bbdd-477b-95ac-dad05d23cefa") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 18:33:19.611695 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:19.611656    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46d9ac39-ee45-4e43-bda7-3931c3e599cf-original-pull-secret podName:46d9ac39-ee45-4e43-bda7-3931c3e599cf nodeName:}" failed. No retries permitted until 2026-10-30 18:33:23.611639463 +0000 UTC m=+10.103073906 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/46d9ac39-ee45-4e43-bda7-3931c3e599cf-original-pull-secret") pod "global-pull-secret-syncer-g2svp" (UID: "46d9ac39-ee45-4e43-bda7-3931c3e599cf") : object "kube-system"/"original-pull-secret" not registered
Oct 30 18:33:19.712372 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:19.712346    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c45qr\" (UniqueName: \"kubernetes.io/projected/613020c0-f3e4-4650-896f-748a40f0ab8b-kube-api-access-c45qr\") pod \"network-check-target-8xskh\" (UID: \"613020c0-f3e4-4650-896f-748a40f0ab8b\") " pod="openshift-network-diagnostics/network-check-target-8xskh"
Oct 30 18:33:19.712505 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:19.712452    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 18:33:19.712505 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:19.712465    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 18:33:19.712505 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:19.712474    2574 projected.go:196] Error preparing data for projected volume kube-api-access-c45qr for pod openshift-network-diagnostics/network-check-target-8xskh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 18:33:19.712663 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:19.712514    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/613020c0-f3e4-4650-896f-748a40f0ab8b-kube-api-access-c45qr podName:613020c0-f3e4-4650-896f-748a40f0ab8b nodeName:}" failed. No retries permitted until 2026-10-30 18:33:23.712502787 +0000 UTC m=+10.203937240 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-c45qr" (UniqueName: "kubernetes.io/projected/613020c0-f3e4-4650-896f-748a40f0ab8b-kube-api-access-c45qr") pod "network-check-target-8xskh" (UID: "613020c0-f3e4-4650-896f-748a40f0ab8b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 18:33:19.940842 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:19.940385    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v5phn"
Oct 30 18:33:19.940842 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:19.940524    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v5phn" podUID="a0823e8f-bbdd-477b-95ac-dad05d23cefa"
Oct 30 18:33:19.940842 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:19.940584    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8xskh"
Oct 30 18:33:19.940842 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:19.940658    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8xskh" podUID="613020c0-f3e4-4650-896f-748a40f0ab8b"
Oct 30 18:33:19.941444 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:19.941223    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g2svp"
Oct 30 18:33:19.941444 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:19.941352    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g2svp" podUID="46d9ac39-ee45-4e43-bda7-3931c3e599cf"
Oct 30 18:33:19.982663 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:19.982244    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-46.us-west-1.compute.internal" event={"ID":"6f66fae2621414751921d90f1913a614","Type":"ContainerStarted","Data":"ba00e5f5c5b06a6028795d0f094174837c924b8adeab8e75fc728e20ebc48ee8"}
Oct 30 18:33:19.985355 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:19.985006    2574 generic.go:356] "Generic (PLEG): container finished" podID="563184d807a5bf00cb82141d4921b071" containerID="1e65f2f361c630d925ef31c64a07e71ddb2b0a47ad1089ca0042051b32f32f1a" exitCode=0
Oct 30 18:33:19.985355 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:19.985083    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-46.us-west-1.compute.internal" event={"ID":"563184d807a5bf00cb82141d4921b071","Type":"ContainerDied","Data":"1e65f2f361c630d925ef31c64a07e71ddb2b0a47ad1089ca0042051b32f32f1a"}
Oct 30 18:33:19.993590 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:19.993562    2574 generic.go:356] "Generic (PLEG): container finished" podID="2e49faaa-a718-44c8-9237-5a41169faee7" containerID="d9dfb149b80f5deffc4eabb8b6be243f2d902b34f56f7490fe6da8d321310c36" exitCode=0
Oct 30 18:33:19.993676 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:19.993599    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rzrjz" event={"ID":"2e49faaa-a718-44c8-9237-5a41169faee7","Type":"ContainerDied","Data":"d9dfb149b80f5deffc4eabb8b6be243f2d902b34f56f7490fe6da8d321310c36"}
Oct 30 18:33:20.268622 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:20.268112    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-46.us-west-1.compute.internal" podStartSLOduration=5.268094769 podStartE2EDuration="5.268094769s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:33:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:33:20.146975264 +0000 UTC m=+6.638409727" watchObservedRunningTime="2026-10-30 18:33:20.268094769 +0000 UTC m=+6.759529239"
Oct 30 18:33:21.006055 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:21.006010    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-46.us-west-1.compute.internal" event={"ID":"563184d807a5bf00cb82141d4921b071","Type":"ContainerStarted","Data":"7b4b0c07a4e86d73f8c0a42fae0f01482d9a0319b3ce15504accc2abd1a861ee"}
Oct 30 18:33:21.941779 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:21.941742    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8xskh"
Oct 30 18:33:21.941954 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:21.941890    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8xskh" podUID="613020c0-f3e4-4650-896f-748a40f0ab8b"
Oct 30 18:33:21.942019 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:21.941969    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g2svp"
Oct 30 18:33:21.942072 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:21.942047    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g2svp" podUID="46d9ac39-ee45-4e43-bda7-3931c3e599cf"
Oct 30 18:33:21.942131 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:21.942102    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v5phn"
Oct 30 18:33:21.942204 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:21.942179    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v5phn" podUID="a0823e8f-bbdd-477b-95ac-dad05d23cefa"
Oct 30 18:33:22.011443 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:22.011372    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-lmdpp" event={"ID":"8ecf2b62-8053-4bba-a852-c6f6119d3288","Type":"ContainerStarted","Data":"1a43e64a8d452237078db7eade2156c0f0a70264dfa882cd247252a6cc063006"}
Oct 30 18:33:22.124635 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:22.123843    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-46.us-west-1.compute.internal" podStartSLOduration=7.123822386 podStartE2EDuration="7.123822386s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:33:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:33:21.120928769 +0000 UTC m=+7.612363231" watchObservedRunningTime="2026-10-30 18:33:22.123822386 +0000 UTC m=+8.615256849"
Oct 30 18:33:22.485409 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:22.485337    2574 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-lmdpp"
Oct 30 18:33:22.486252 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:22.486073    2574 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-lmdpp"
Oct 30 18:33:22.599933 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:22.599851    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/konnectivity-agent-lmdpp" podStartSLOduration=3.751714434 podStartE2EDuration="8.599832352s" totalImagesPullingTime="4.848117918s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:33:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:33:22.124610812 +0000 UTC m=+8.616045275" watchObservedRunningTime="2026-10-30 18:33:22.599832352 +0000 UTC m=+9.091266815"
Oct 30 18:33:23.015827 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:23.015750    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-g78lj" event={"ID":"46a80637-6ccf-4a56-9904-f5e2fe5784ad","Type":"ContainerStarted","Data":"449870b20acba774cdd1abee1ca4b7fd360568f5277877b99952f3e2ed00b9b4"}
Oct 30 18:33:23.019226 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:23.019197    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7j9jw" event={"ID":"ad78ad05-b0af-4adf-a6bc-0f993b8a6288","Type":"ContainerStarted","Data":"4800d1c2d197eca8a2d6ac9ccd9353c0a9af068e88918774710d387e98604e04"}
Oct 30 18:33:23.644120 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:23.644082    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a0823e8f-bbdd-477b-95ac-dad05d23cefa-metrics-certs\") pod \"network-metrics-daemon-v5phn\" (UID: \"a0823e8f-bbdd-477b-95ac-dad05d23cefa\") " pod="openshift-multus/network-metrics-daemon-v5phn"
Oct 30 18:33:23.644120 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:23.644125    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/46d9ac39-ee45-4e43-bda7-3931c3e599cf-original-pull-secret\") pod \"global-pull-secret-syncer-g2svp\" (UID: \"46d9ac39-ee45-4e43-bda7-3931c3e599cf\") " pod="kube-system/global-pull-secret-syncer-g2svp"
Oct 30 18:33:23.644374 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:23.644230    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 18:33:23.644374 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:23.644272    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46d9ac39-ee45-4e43-bda7-3931c3e599cf-original-pull-secret podName:46d9ac39-ee45-4e43-bda7-3931c3e599cf nodeName:}" failed. No retries permitted until 2026-10-30 18:33:31.644261185 +0000 UTC m=+18.135695626 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/46d9ac39-ee45-4e43-bda7-3931c3e599cf-original-pull-secret") pod "global-pull-secret-syncer-g2svp" (UID: "46d9ac39-ee45-4e43-bda7-3931c3e599cf") : object "kube-system"/"original-pull-secret" not registered
Oct 30 18:33:23.644374 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:23.644355    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 18:33:23.644536 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:23.644390    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a0823e8f-bbdd-477b-95ac-dad05d23cefa-metrics-certs podName:a0823e8f-bbdd-477b-95ac-dad05d23cefa nodeName:}" failed. No retries permitted until 2026-10-30 18:33:31.644378659 +0000 UTC m=+18.135813105 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a0823e8f-bbdd-477b-95ac-dad05d23cefa-metrics-certs") pod "network-metrics-daemon-v5phn" (UID: "a0823e8f-bbdd-477b-95ac-dad05d23cefa") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 18:33:23.744605 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:23.744573    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c45qr\" (UniqueName: \"kubernetes.io/projected/613020c0-f3e4-4650-896f-748a40f0ab8b-kube-api-access-c45qr\") pod \"network-check-target-8xskh\" (UID: \"613020c0-f3e4-4650-896f-748a40f0ab8b\") " pod="openshift-network-diagnostics/network-check-target-8xskh"
Oct 30 18:33:23.744787 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:23.744708    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 18:33:23.744787 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:23.744723    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 18:33:23.744787 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:23.744734    2574 projected.go:196] Error preparing data for projected volume kube-api-access-c45qr for pod openshift-network-diagnostics/network-check-target-8xskh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 18:33:23.744787 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:23.744778    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/613020c0-f3e4-4650-896f-748a40f0ab8b-kube-api-access-c45qr podName:613020c0-f3e4-4650-896f-748a40f0ab8b nodeName:}" failed. No retries permitted until 2026-10-30 18:33:31.744767738 +0000 UTC m=+18.236202179 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-c45qr" (UniqueName: "kubernetes.io/projected/613020c0-f3e4-4650-896f-748a40f0ab8b-kube-api-access-c45qr") pod "network-check-target-8xskh" (UID: "613020c0-f3e4-4650-896f-748a40f0ab8b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 18:33:23.948572 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:23.948311    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g2svp"
Oct 30 18:33:23.948572 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:23.948437    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g2svp" podUID="46d9ac39-ee45-4e43-bda7-3931c3e599cf"
Oct 30 18:33:23.948572 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:23.948456    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8xskh"
Oct 30 18:33:23.948572 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:23.948500    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v5phn"
Oct 30 18:33:23.948867 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:23.948602    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8xskh" podUID="613020c0-f3e4-4650-896f-748a40f0ab8b"
Oct 30 18:33:23.948867 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:23.948703    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v5phn" podUID="a0823e8f-bbdd-477b-95ac-dad05d23cefa"
Oct 30 18:33:24.022198 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:24.022172    2574 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 30 18:33:24.840359 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:24.835361    2574 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Oct 30 18:33:24.953953 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:24.953631    2574 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-10-30T18:33:24.835383447Z","UUID":"8fe31af8-d127-411c-bc2f-bb2c39063170","Handler":null,"Name":"","Endpoint":""}
Oct 30 18:33:24.955130 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:24.955107    2574 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Oct 30 18:33:24.955246 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:24.955139    2574 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Oct 30 18:33:25.027153 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:25.026781    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-9ssmv" event={"ID":"aa3f3b62-a1ab-4113-bef3-ed1b290e47c1","Type":"ContainerStarted","Data":"60bafb8a96702b71239b34f631f56703157fd579dbebb40d669a442c8416b81a"}
Oct 30 18:33:25.028580 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:25.028519    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-76f9s" event={"ID":"d81cef55-16ee-4677-ae12-86de3c9eb216","Type":"ContainerStarted","Data":"d2f65d043952bb83e252a6f3a4aef17e2ff398fbc8df97f07830ea1ae58b0321"}
Oct 30 18:33:25.039042 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:25.039005    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7j9jw" event={"ID":"ad78ad05-b0af-4adf-a6bc-0f993b8a6288","Type":"ContainerStarted","Data":"023bd213178e02d741f9f0b6772149ddaccdb3edf266060b5a1c48f8c4714bb4"}
Oct 30 18:33:25.141231 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:25.141121    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-image-registry/node-ca-g78lj" podStartSLOduration=5.374558541 podStartE2EDuration="11.141104087s" totalImagesPullingTime="5.766545546s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:33:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:33:23.128431393 +0000 UTC m=+9.619865857" watchObservedRunningTime="2026-10-30 18:33:25.141104087 +0000 UTC m=+11.632538551"
Oct 30 18:33:25.141422 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:25.141256    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-9ssmv" podStartSLOduration=2.8610460079999998 podStartE2EDuration="11.141249728s" totalImagesPullingTime="8.28020372s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:33:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:33:25.140922665 +0000 UTC m=+11.632357128" watchObservedRunningTime="2026-10-30 18:33:25.141249728 +0000 UTC m=+11.632684191"
Oct 30 18:33:25.943080 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:25.943033    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8xskh"
Oct 30 18:33:25.943239 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:25.943155    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v5phn"
Oct 30 18:33:25.943325 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:25.943278    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8xskh" podUID="613020c0-f3e4-4650-896f-748a40f0ab8b"
Oct 30 18:33:25.943408 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:25.943316    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g2svp"
Oct 30 18:33:25.943463 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:25.943413    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v5phn" podUID="a0823e8f-bbdd-477b-95ac-dad05d23cefa"
Oct 30 18:33:25.943510 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:25.943492    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g2svp" podUID="46d9ac39-ee45-4e43-bda7-3931c3e599cf"
Oct 30 18:33:26.043942 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:26.043889    2574 generic.go:356] "Generic (PLEG): container finished" podID="2e49faaa-a718-44c8-9237-5a41169faee7" containerID="d26b775e7cc6ed33fca99f6766d410d23291f151fee83292926e7f8b3980b3f7" exitCode=0
Oct 30 18:33:26.044373 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:26.043977    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rzrjz" event={"ID":"2e49faaa-a718-44c8-9237-5a41169faee7","Type":"ContainerDied","Data":"d26b775e7cc6ed33fca99f6766d410d23291f151fee83292926e7f8b3980b3f7"}
Oct 30 18:33:26.160419 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:26.160340    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-dns/node-resolver-76f9s" podStartSLOduration=3.886827297 podStartE2EDuration="12.160320667s" totalImagesPullingTime="8.27349337s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:33:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:33:25.255176975 +0000 UTC m=+11.746611439" watchObservedRunningTime="2026-10-30 18:33:26.160320667 +0000 UTC m=+12.651755130"
Oct 30 18:33:27.048850 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:27.048759    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7j9jw" event={"ID":"ad78ad05-b0af-4adf-a6bc-0f993b8a6288","Type":"ContainerStarted","Data":"66f8620ca7fe2321ae52cb2edf952ebd3f96b56e924bb884d9148bb50b3f8cb2"}
Oct 30 18:33:27.050502 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:27.050469    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-z9jmw" event={"ID":"f2a80d89-61d1-4694-8734-004ce5cfb22b","Type":"ContainerStarted","Data":"b13de7d87fdcef2e35c86b4a58e42b75948dd03f4aae0edf7b14aff76c7694d8"}
Oct 30 18:33:27.052144 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:27.052120    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-rcjmv" event={"ID":"63090f48-9143-4118-9906-d759c0aff52c","Type":"ContainerStarted","Data":"70913ea50568e3e6f1616e6075a81fa162456bcbd32b61f6975d8b4b6bf3f1df"}
Oct 30 18:33:27.168710 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:27.168660    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7j9jw" podStartSLOduration=3.5548284900000002 podStartE2EDuration="13.168646132s" totalImagesPullingTime="9.613817642s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:33:14 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:33:27.167349829 +0000 UTC m=+13.658784291" watchObservedRunningTime="2026-10-30 18:33:27.168646132 +0000 UTC m=+13.660080595"
Oct 30 18:33:27.294299 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:27.294232    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-z9jmw" podStartSLOduration=2.907930706 podStartE2EDuration="13.294218026s" totalImagesPullingTime="10.38628732s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:33:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:33:27.293938379 +0000 UTC m=+13.785372840" watchObservedRunningTime="2026-10-30 18:33:27.294218026 +0000 UTC m=+13.785652488"
Oct 30 18:33:27.429971 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:27.429918    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-rcjmv" podStartSLOduration=4.864132539 podStartE2EDuration="13.429899946s" totalImagesPullingTime="8.565767407s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:33:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:33:27.429203386 +0000 UTC m=+13.920637849" watchObservedRunningTime="2026-10-30 18:33:27.429899946 +0000 UTC m=+13.921334407"
Oct 30 18:33:27.939420 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:27.939390    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g2svp"
Oct 30 18:33:27.939599 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:27.939462    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8xskh"
Oct 30 18:33:27.939599 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:27.939468    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v5phn"
Oct 30 18:33:27.939599 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:27.939556    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g2svp" podUID="46d9ac39-ee45-4e43-bda7-3931c3e599cf"
Oct 30 18:33:27.939754 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:27.939643    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8xskh" podUID="613020c0-f3e4-4650-896f-748a40f0ab8b"
Oct 30 18:33:27.939754 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:27.939718    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v5phn" podUID="a0823e8f-bbdd-477b-95ac-dad05d23cefa"
Oct 30 18:33:28.056274 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:28.056236    2574 generic.go:356] "Generic (PLEG): container finished" podID="2e49faaa-a718-44c8-9237-5a41169faee7" containerID="6b12d406f46c96167c2e4e52af445a7bae2edd96f117a61e1a2063485532dcf0" exitCode=0
Oct 30 18:33:28.056948 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:28.056365    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rzrjz" event={"ID":"2e49faaa-a718-44c8-9237-5a41169faee7","Type":"ContainerDied","Data":"6b12d406f46c96167c2e4e52af445a7bae2edd96f117a61e1a2063485532dcf0"}
Oct 30 18:33:28.683989 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:28.683954    2574 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-lmdpp"
Oct 30 18:33:28.684172 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:28.684092    2574 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 30 18:33:28.686233 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:28.685268    2574 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-lmdpp"
Oct 30 18:33:29.939151 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:29.938934    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g2svp"
Oct 30 18:33:29.939764 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:29.939004    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v5phn"
Oct 30 18:33:29.939764 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:29.939240    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g2svp" podUID="46d9ac39-ee45-4e43-bda7-3931c3e599cf"
Oct 30 18:33:29.939764 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:29.939368    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v5phn" podUID="a0823e8f-bbdd-477b-95ac-dad05d23cefa"
Oct 30 18:33:29.939764 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:29.939006    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8xskh"
Oct 30 18:33:29.939764 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:29.939457    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8xskh" podUID="613020c0-f3e4-4650-896f-748a40f0ab8b"
Oct 30 18:33:30.063807 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:30.063773    2574 generic.go:356] "Generic (PLEG): container finished" podID="2e49faaa-a718-44c8-9237-5a41169faee7" containerID="5b63a8c037044e4cf0d2e567fbf7d272e9c6563846302d3a5e865b84758cfd72" exitCode=0
Oct 30 18:33:30.063981 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:30.063831    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rzrjz" event={"ID":"2e49faaa-a718-44c8-9237-5a41169faee7","Type":"ContainerDied","Data":"5b63a8c037044e4cf0d2e567fbf7d272e9c6563846302d3a5e865b84758cfd72"}
Oct 30 18:33:31.710519 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:31.710491    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a0823e8f-bbdd-477b-95ac-dad05d23cefa-metrics-certs\") pod \"network-metrics-daemon-v5phn\" (UID: \"a0823e8f-bbdd-477b-95ac-dad05d23cefa\") " pod="openshift-multus/network-metrics-daemon-v5phn"
Oct 30 18:33:31.711158 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:31.710530    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/46d9ac39-ee45-4e43-bda7-3931c3e599cf-original-pull-secret\") pod \"global-pull-secret-syncer-g2svp\" (UID: \"46d9ac39-ee45-4e43-bda7-3931c3e599cf\") " pod="kube-system/global-pull-secret-syncer-g2svp"
Oct 30 18:33:31.711158 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:31.710649    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 18:33:31.711158 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:31.710653    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 18:33:31.711158 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:31.710701    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46d9ac39-ee45-4e43-bda7-3931c3e599cf-original-pull-secret podName:46d9ac39-ee45-4e43-bda7-3931c3e599cf nodeName:}" failed. No retries permitted until 2026-10-30 18:33:47.710682095 +0000 UTC m=+34.202116551 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/46d9ac39-ee45-4e43-bda7-3931c3e599cf-original-pull-secret") pod "global-pull-secret-syncer-g2svp" (UID: "46d9ac39-ee45-4e43-bda7-3931c3e599cf") : object "kube-system"/"original-pull-secret" not registered
Oct 30 18:33:31.711158 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:31.710720    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a0823e8f-bbdd-477b-95ac-dad05d23cefa-metrics-certs podName:a0823e8f-bbdd-477b-95ac-dad05d23cefa nodeName:}" failed. No retries permitted until 2026-10-30 18:33:47.710710588 +0000 UTC m=+34.202145029 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a0823e8f-bbdd-477b-95ac-dad05d23cefa-metrics-certs") pod "network-metrics-daemon-v5phn" (UID: "a0823e8f-bbdd-477b-95ac-dad05d23cefa") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 18:33:31.811838 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:31.811808    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c45qr\" (UniqueName: \"kubernetes.io/projected/613020c0-f3e4-4650-896f-748a40f0ab8b-kube-api-access-c45qr\") pod \"network-check-target-8xskh\" (UID: \"613020c0-f3e4-4650-896f-748a40f0ab8b\") " pod="openshift-network-diagnostics/network-check-target-8xskh"
Oct 30 18:33:31.811970 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:31.811931    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 18:33:31.811970 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:31.811947    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 18:33:31.811970 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:31.811956    2574 projected.go:196] Error preparing data for projected volume kube-api-access-c45qr for pod openshift-network-diagnostics/network-check-target-8xskh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 18:33:31.812076 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:31.812006    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/613020c0-f3e4-4650-896f-748a40f0ab8b-kube-api-access-c45qr podName:613020c0-f3e4-4650-896f-748a40f0ab8b nodeName:}" failed. No retries permitted until 2026-10-30 18:33:47.811993561 +0000 UTC m=+34.303428013 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-c45qr" (UniqueName: "kubernetes.io/projected/613020c0-f3e4-4650-896f-748a40f0ab8b-kube-api-access-c45qr") pod "network-check-target-8xskh" (UID: "613020c0-f3e4-4650-896f-748a40f0ab8b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 18:33:31.939565 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:31.939319    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v5phn"
Oct 30 18:33:31.939732 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:31.939366    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8xskh"
Oct 30 18:33:31.939732 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:31.939393    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g2svp"
Oct 30 18:33:31.939732 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:31.939684    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v5phn" podUID="a0823e8f-bbdd-477b-95ac-dad05d23cefa"
Oct 30 18:33:31.939875 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:31.939767    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g2svp" podUID="46d9ac39-ee45-4e43-bda7-3931c3e599cf"
Oct 30 18:33:31.939875 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:31.939841    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8xskh" podUID="613020c0-f3e4-4650-896f-748a40f0ab8b"
Oct 30 18:33:32.070834 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:32.070804    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xfcdl_3f3fe764-2652-46c9-8e54-7882c0e6bcf1/ovn-acl-logging/0.log"
Oct 30 18:33:32.071235 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:32.071196    2574 generic.go:356] "Generic (PLEG): container finished" podID="3f3fe764-2652-46c9-8e54-7882c0e6bcf1" containerID="c5c342c1ccf42795a343c08c13f829ff016aa630fb7262cbe75edbe1e6bffb55" exitCode=1
Oct 30 18:33:32.071332 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:32.071264    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xfcdl" event={"ID":"3f3fe764-2652-46c9-8e54-7882c0e6bcf1","Type":"ContainerStarted","Data":"2f70ec5fd4b20e7f288f011a5c8b8b6a57bafb77da14bbe24d307ddd534b0313"}
Oct 30 18:33:32.071367 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:32.071357    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xfcdl" event={"ID":"3f3fe764-2652-46c9-8e54-7882c0e6bcf1","Type":"ContainerStarted","Data":"23d5f413a787750ad2ae3b326c6ca5ec0ac0fd07c059703d40eef3d8a25d70c8"}
Oct 30 18:33:32.071427 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:32.071395    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xfcdl" event={"ID":"3f3fe764-2652-46c9-8e54-7882c0e6bcf1","Type":"ContainerStarted","Data":"791d6a31d8745efae967b76f2404014a8f558090f79426bb5fbebd2cf96a4ab3"}
Oct 30 18:33:32.071458 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:32.071433    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xfcdl" event={"ID":"3f3fe764-2652-46c9-8e54-7882c0e6bcf1","Type":"ContainerStarted","Data":"a5dc7992d59ad25dca4eb3172f3855d400ce3aa965356106eb9bbb2cb21ec5e5"}
Oct 30 18:33:32.071485 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:32.071462    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xfcdl" event={"ID":"3f3fe764-2652-46c9-8e54-7882c0e6bcf1","Type":"ContainerDied","Data":"c5c342c1ccf42795a343c08c13f829ff016aa630fb7262cbe75edbe1e6bffb55"}
Oct 30 18:33:32.071525 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:32.071503    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xfcdl" event={"ID":"3f3fe764-2652-46c9-8e54-7882c0e6bcf1","Type":"ContainerStarted","Data":"3458dcc91ec2f1681cc81d79d76cca2e8efcaf63f937d5d89623b2f6739b9bbf"}
Oct 30 18:33:33.075225 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:33.075196    2574 generic.go:356] "Generic (PLEG): container finished" podID="2e49faaa-a718-44c8-9237-5a41169faee7" containerID="4b5e52c31b6f486d330744e5a02a1e5e7cda001334c2de6ec0339b16fff4cb92" exitCode=0
Oct 30 18:33:33.075658 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:33.075234    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rzrjz" event={"ID":"2e49faaa-a718-44c8-9237-5a41169faee7","Type":"ContainerDied","Data":"4b5e52c31b6f486d330744e5a02a1e5e7cda001334c2de6ec0339b16fff4cb92"}
Oct 30 18:33:33.939632 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:33.939591    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v5phn"
Oct 30 18:33:33.939782 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:33.939707    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v5phn" podUID="a0823e8f-bbdd-477b-95ac-dad05d23cefa"
Oct 30 18:33:33.939782 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:33.939755    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g2svp"
Oct 30 18:33:33.939893 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:33.939820    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g2svp" podUID="46d9ac39-ee45-4e43-bda7-3931c3e599cf"
Oct 30 18:33:33.939893 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:33.939851    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8xskh"
Oct 30 18:33:33.939956 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:33.939930    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8xskh" podUID="613020c0-f3e4-4650-896f-748a40f0ab8b"
Oct 30 18:33:34.079941 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:34.079915    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xfcdl_3f3fe764-2652-46c9-8e54-7882c0e6bcf1/ovn-acl-logging/0.log"
Oct 30 18:33:34.080468 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:34.080275    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xfcdl" event={"ID":"3f3fe764-2652-46c9-8e54-7882c0e6bcf1","Type":"ContainerStarted","Data":"14eb12181ff837035e920dc3a9d264fb44f9a7c9b851e5a894c137cbdf73ff6c"}
Oct 30 18:33:34.083010 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:34.082988    2574 generic.go:356] "Generic (PLEG): container finished" podID="2e49faaa-a718-44c8-9237-5a41169faee7" containerID="5b0d439590091c2234cd6b4627899cc9c5f5e50d164316600d14a4d65f8bf752" exitCode=0
Oct 30 18:33:34.083112 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:34.083014    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rzrjz" event={"ID":"2e49faaa-a718-44c8-9237-5a41169faee7","Type":"ContainerDied","Data":"5b0d439590091c2234cd6b4627899cc9c5f5e50d164316600d14a4d65f8bf752"}
Oct 30 18:33:35.087810 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:35.087773    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rzrjz" event={"ID":"2e49faaa-a718-44c8-9237-5a41169faee7","Type":"ContainerStarted","Data":"729935d077472df0b81e976b217c1889257faeda753e33076cb33eff81ea54df"}
Oct 30 18:33:35.312546 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:35.312501    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-rzrjz" podStartSLOduration=7.849894584 podStartE2EDuration="21.312487492s" totalImagesPullingTime="13.462592908s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:33:14 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:33:35.312270547 +0000 UTC m=+21.803705008" watchObservedRunningTime="2026-10-30 18:33:35.312487492 +0000 UTC m=+21.803921954"
Oct 30 18:33:35.939096 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:35.939062    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v5phn"
Oct 30 18:33:35.939247 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:35.939062    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8xskh"
Oct 30 18:33:35.939247 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:35.939198    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v5phn" podUID="a0823e8f-bbdd-477b-95ac-dad05d23cefa"
Oct 30 18:33:35.939247 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:35.939227    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g2svp"
Oct 30 18:33:35.939418 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:35.939300    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8xskh" podUID="613020c0-f3e4-4650-896f-748a40f0ab8b"
Oct 30 18:33:35.939418 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:35.939401    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g2svp" podUID="46d9ac39-ee45-4e43-bda7-3931c3e599cf"
Oct 30 18:33:36.093202 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:36.092993    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xfcdl_3f3fe764-2652-46c9-8e54-7882c0e6bcf1/ovn-acl-logging/0.log"
Oct 30 18:33:36.093863 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:36.093513    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xfcdl" event={"ID":"3f3fe764-2652-46c9-8e54-7882c0e6bcf1","Type":"ContainerStarted","Data":"6533173c8c9eda94079a439835eeb4f6426c69b43ba8f0842bda9036f2ca693a"}
Oct 30 18:33:36.094648 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:36.094616    2574 scope.go:122] "RemoveContainer" containerID="c5c342c1ccf42795a343c08c13f829ff016aa630fb7262cbe75edbe1e6bffb55"
Oct 30 18:33:37.100248 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:37.100219    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xfcdl_3f3fe764-2652-46c9-8e54-7882c0e6bcf1/ovn-acl-logging/0.log"
Oct 30 18:33:37.100800 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:37.100660    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xfcdl" event={"ID":"3f3fe764-2652-46c9-8e54-7882c0e6bcf1","Type":"ContainerStarted","Data":"48f9e0ef414db50bc55e2c578e39ed109e2e92206bf21e442407f50e38e81159"}
Oct 30 18:33:37.101219 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:37.101197    2574 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-xfcdl"
Oct 30 18:33:37.101361 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:37.101269    2574 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-xfcdl"
Oct 30 18:33:37.101361 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:37.101332    2574 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-xfcdl"
Oct 30 18:33:37.118475 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:37.118452    2574 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-xfcdl"
Oct 30 18:33:37.118939 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:37.118923    2574 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-xfcdl"
Oct 30 18:33:37.515222 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:37.515123    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-xfcdl" podStartSLOduration=8.511340692 podStartE2EDuration="23.515109857s" totalImagesPullingTime="15.003769165s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:33:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:33:37.380753771 +0000 UTC m=+23.872188232" watchObservedRunningTime="2026-10-30 18:33:37.515109857 +0000 UTC m=+24.006544319"
Oct 30 18:33:37.938856 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:37.938830    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g2svp"
Oct 30 18:33:37.938856 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:37.938849    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v5phn"
Oct 30 18:33:37.939018 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:37.938906    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8xskh"
Oct 30 18:33:37.939018 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:37.938980    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g2svp" podUID="46d9ac39-ee45-4e43-bda7-3931c3e599cf"
Oct 30 18:33:37.939088 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:37.939033    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v5phn" podUID="a0823e8f-bbdd-477b-95ac-dad05d23cefa"
Oct 30 18:33:37.939141 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:37.939121    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8xskh" podUID="613020c0-f3e4-4650-896f-748a40f0ab8b"
Oct 30 18:33:38.489902 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:38.489872    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-8xskh"]
Oct 30 18:33:38.490428 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:38.490011    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8xskh"
Oct 30 18:33:38.490428 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:38.490127    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8xskh" podUID="613020c0-f3e4-4650-896f-748a40f0ab8b"
Oct 30 18:33:38.493086 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:38.493063    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-g2svp"]
Oct 30 18:33:38.493197 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:38.493160    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g2svp"
Oct 30 18:33:38.493326 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:38.493255    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g2svp" podUID="46d9ac39-ee45-4e43-bda7-3931c3e599cf"
Oct 30 18:33:38.494095 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:38.494073    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-v5phn"]
Oct 30 18:33:38.494186 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:38.494155    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v5phn"
Oct 30 18:33:38.494234 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:38.494219    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v5phn" podUID="a0823e8f-bbdd-477b-95ac-dad05d23cefa"
Oct 30 18:33:39.938815 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:39.938786    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g2svp"
Oct 30 18:33:39.939125 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:39.938784    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8xskh"
Oct 30 18:33:39.939125 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:39.938900    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g2svp" podUID="46d9ac39-ee45-4e43-bda7-3931c3e599cf"
Oct 30 18:33:39.939125 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:39.938985    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8xskh" podUID="613020c0-f3e4-4650-896f-748a40f0ab8b"
Oct 30 18:33:39.939125 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:39.938786    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v5phn"
Oct 30 18:33:39.939125 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:39.939087    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v5phn" podUID="a0823e8f-bbdd-477b-95ac-dad05d23cefa"
Oct 30 18:33:41.941411 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:41.941381    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8xskh"
Oct 30 18:33:41.941949 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:41.941477    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g2svp"
Oct 30 18:33:41.941949 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:41.941484    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8xskh" podUID="613020c0-f3e4-4650-896f-748a40f0ab8b"
Oct 30 18:33:41.941949 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:41.941495    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v5phn"
Oct 30 18:33:41.941949 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:41.941558    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g2svp" podUID="46d9ac39-ee45-4e43-bda7-3931c3e599cf"
Oct 30 18:33:41.941949 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:41.941642    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v5phn" podUID="a0823e8f-bbdd-477b-95ac-dad05d23cefa"
Oct 30 18:33:43.942312 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:43.942122    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g2svp"
Oct 30 18:33:43.942636 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:43.942149    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8xskh"
Oct 30 18:33:43.942636 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:43.942391    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-g2svp" podUID="46d9ac39-ee45-4e43-bda7-3931c3e599cf"
Oct 30 18:33:43.942636 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:43.942464    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8xskh" podUID="613020c0-f3e4-4650-896f-748a40f0ab8b"
Oct 30 18:33:43.942636 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:43.942181    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v5phn"
Oct 30 18:33:43.942636 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:43.942623    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v5phn" podUID="a0823e8f-bbdd-477b-95ac-dad05d23cefa"
Oct 30 18:33:44.719247 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:44.719218    2574 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-46.us-west-1.compute.internal" event="NodeReady"
Oct 30 18:33:44.719457 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:44.719375    2574 kubelet_node_status.go:477] "Fast updating node status as it just became ready"
Oct 30 18:33:44.834229 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:44.834107    2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-f9ldt"]
Oct 30 18:33:44.837257 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:44.837243    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-f9ldt"
Oct 30 18:33:44.884213 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:44.883782    2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-4b95q"]
Oct 30 18:33:44.889243 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:44.889224    2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-86c478bb88-r8rgt"]
Oct 30 18:33:44.889396 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:44.889376    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-4b95q"
Oct 30 18:33:44.891734 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:44.891718    2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-5bqdc"]
Oct 30 18:33:44.891851 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:44.891833    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-86c478bb88-r8rgt"
Oct 30 18:33:44.894015 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:44.893997    2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-7c4986d449-xvgqc"]
Oct 30 18:33:44.894130 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:44.894115    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-5bqdc"
Oct 30 18:33:44.896846 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:44.896828    2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-2grbp"]
Oct 30 18:33:44.896938 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:44.896924    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-7c4986d449-xvgqc"
Oct 30 18:33:44.899506 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:44.899490    2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-7cd56b8dbc-7x4rb"]
Oct 30 18:33:44.899632 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:44.899617    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-2grbp"
Oct 30 18:33:44.902004 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:44.901989    2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-544d749c8c-kr5qq"]
Oct 30 18:33:44.902198 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:44.902176    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7cd56b8dbc-7x4rb"
Oct 30 18:33:44.904865 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:44.904849    2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-7dfcf9fd6c-vh769"]
Oct 30 18:33:44.904965 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:44.904941    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-kr5qq"
Oct 30 18:33:44.906399 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:44.906372    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nk8r7\" (UniqueName: \"kubernetes.io/projected/df9e5074-3bb1-4817-8855-b045cee61004-kube-api-access-nk8r7\") pod \"kas-connection-checker-685895c7c9-f9ldt\" (UID: \"df9e5074-3bb1-4817-8855-b045cee61004\") " pod="kube-system/kas-connection-checker-685895c7c9-f9ldt"
Oct 30 18:33:44.907406 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:44.907392    2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-884576c57-gmswj"]
Oct 30 18:33:44.907517 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:44.907503    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-7dfcf9fd6c-vh769"
Oct 30 18:33:44.910190 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:44.910172    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-gmswj"
Oct 30 18:33:44.932883 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:44.932864    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"openshift-service-ca.crt\""
Oct 30 18:33:44.932994 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:44.932923    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"kube-root-ca.crt\""
Oct 30 18:33:44.934704 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:44.933756    2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-64bf5b6b76-dpvvs"]
Oct 30 18:33:44.937652 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:44.937629    2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-jwwq2"]
Oct 30 18:33:44.938295 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:44.937934    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-64bf5b6b76-dpvvs"
Oct 30 18:33:44.941068 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:44.941051    2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-pm58m"]
Oct 30 18:33:44.941166 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:44.941151    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-jwwq2"
Oct 30 18:33:44.943914 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:44.943894    2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-74757bccd5-qmrkp"]
Oct 30 18:33:44.944218 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:44.944095    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-pm58m"
Oct 30 18:33:44.947453 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:44.946850    2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-6f78d59f99-4969l"]
Oct 30 18:33:44.947673 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:44.947653    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-74757bccd5-qmrkp"
Oct 30 18:33:44.950556 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:44.950516    2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-r2nx2"]
Oct 30 18:33:44.954925 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:44.954738    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-4969l"
Oct 30 18:33:44.957019 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:44.955647    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Oct 30 18:33:44.957159 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:44.955967    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-dockercfg-mlq55\""
Oct 30 18:33:44.958805 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:44.958787    2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-g9fhp"]
Oct 30 18:33:44.958942 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:44.958924    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-r2nx2"
Oct 30 18:33:44.964402 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:44.964339    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-f9ldt"]
Oct 30 18:33:44.966020 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:44.964456    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-4b95q"]
Oct 30 18:33:44.966020 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:44.964485    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-86c478bb88-r8rgt"]
Oct 30 18:33:44.966020 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:44.964506    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-544d749c8c-kr5qq"]
Oct 30 18:33:44.966020 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:44.964527    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7cd56b8dbc-7x4rb"]
Oct 30 18:33:44.966020 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:44.964545    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-5bqdc"]
Oct 30 18:33:44.966020 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:44.964564    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-jwwq2"]
Oct 30 18:33:44.966020 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:44.964584    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-2grbp"]
Oct 30 18:33:44.966020 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:44.964656    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-g9fhp"
Oct 30 18:33:44.966020 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:44.965380    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-pm58m"]
Oct 30 18:33:44.966020 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:44.965799    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-884576c57-gmswj"]
Oct 30 18:33:44.991470 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:44.991450    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Oct 30 18:33:44.991915 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:44.991900    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Oct 30 18:33:44.991964 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:44.991905    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Oct 30 18:33:44.991964 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:44.991929    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Oct 30 18:33:45.005669 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.005649    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Oct 30 18:33:45.007248 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.007230    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351-registry-tls\") pod \"image-registry-64bf5b6b76-dpvvs\" (UID: \"9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351\") " pod="openshift-image-registry/image-registry-64bf5b6b76-dpvvs"
Oct 30 18:33:45.007328 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.007256    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fzrkt\" (UniqueName: \"kubernetes.io/projected/15d485f9-5a4b-4f4b-9e59-a6eb062b4c5c-kube-api-access-fzrkt\") pod \"cluster-samples-operator-6ddb6c8b94-pm58m\" (UID: \"15d485f9-5a4b-4f4b-9e59-a6eb062b4c5c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-pm58m"
Oct 30 18:33:45.007328 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.007276    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8789ec5b-226e-4551-bf68-3ac585006266-image-registry-private-configuration\") pod \"image-registry-74757bccd5-qmrkp\" (UID: \"8789ec5b-226e-4551-bf68-3ac585006266\") " pod="openshift-image-registry/image-registry-74757bccd5-qmrkp"
Oct 30 18:33:45.007328 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.007311    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/68195e19-b1ff-4148-9fae-3545f2997ff5-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-gmswj\" (UID: \"68195e19-b1ff-4148-9fae-3545f2997ff5\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-gmswj"
Oct 30 18:33:45.007474 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.007363    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vxw47\" (UniqueName: \"kubernetes.io/projected/9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351-kube-api-access-vxw47\") pod \"image-registry-64bf5b6b76-dpvvs\" (UID: \"9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351\") " pod="openshift-image-registry/image-registry-64bf5b6b76-dpvvs"
Oct 30 18:33:45.007474 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.007398    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/36a04851-81db-40cd-97ef-8e2e00a08604-config\") pod \"service-ca-operator-544d749c8c-kr5qq\" (UID: \"36a04851-81db-40cd-97ef-8e2e00a08604\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-kr5qq"
Oct 30 18:33:45.007474 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.007418    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351-installation-pull-secrets\") pod \"image-registry-64bf5b6b76-dpvvs\" (UID: \"9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351\") " pod="openshift-image-registry/image-registry-64bf5b6b76-dpvvs"
Oct 30 18:33:45.007474 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.007437    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/09c13759-50d4-40e6-b854-4691822b8c8a-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-5bqdc\" (UID: \"09c13759-50d4-40e6-b854-4691822b8c8a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-5bqdc"
Oct 30 18:33:45.007474 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.007454    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m4j2q\" (UniqueName: \"kubernetes.io/projected/382e8b31-a7fa-43f3-9f15-4c61627b0c04-kube-api-access-m4j2q\") pod \"console-operator-7cd56b8dbc-7x4rb\" (UID: \"382e8b31-a7fa-43f3-9f15-4c61627b0c04\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-7x4rb"
Oct 30 18:33:45.007697 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.007486    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k5crl\" (UniqueName: \"kubernetes.io/projected/09c13759-50d4-40e6-b854-4691822b8c8a-kube-api-access-k5crl\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-5bqdc\" (UID: \"09c13759-50d4-40e6-b854-4691822b8c8a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-5bqdc"
Oct 30 18:33:45.007697 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.007527    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7c09088b-3a07-426d-8804-f0f5bac4d27c-serving-cert\") pod \"insights-operator-7c4986d449-xvgqc\" (UID: \"7c09088b-3a07-426d-8804-f0f5bac4d27c\") " pod="openshift-insights/insights-operator-7c4986d449-xvgqc"
Oct 30 18:33:45.007697 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.007602    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351-ca-trust-extracted\") pod \"image-registry-64bf5b6b76-dpvvs\" (UID: \"9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351\") " pod="openshift-image-registry/image-registry-64bf5b6b76-dpvvs"
Oct 30 18:33:45.007697 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.007647    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7c09088b-3a07-426d-8804-f0f5bac4d27c-service-ca-bundle\") pod \"insights-operator-7c4986d449-xvgqc\" (UID: \"7c09088b-3a07-426d-8804-f0f5bac4d27c\") " pod="openshift-insights/insights-operator-7c4986d449-xvgqc"
Oct 30 18:33:45.007697 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.007675    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8789ec5b-226e-4551-bf68-3ac585006266-bound-sa-token\") pod \"image-registry-74757bccd5-qmrkp\" (UID: \"8789ec5b-226e-4551-bf68-3ac585006266\") " pod="openshift-image-registry/image-registry-74757bccd5-qmrkp"
Oct 30 18:33:45.007697 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.007693    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/da28351c-1c8b-40be-85a0-3aad427e3a17-metrics-certs\") pod \"router-default-7dfcf9fd6c-vh769\" (UID: \"da28351c-1c8b-40be-85a0-3aad427e3a17\") " pod="openshift-ingress/router-default-7dfcf9fd6c-vh769"
Oct 30 18:33:45.007902 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.007709    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cftdb\" (UniqueName: \"kubernetes.io/projected/da28351c-1c8b-40be-85a0-3aad427e3a17-kube-api-access-cftdb\") pod \"router-default-7dfcf9fd6c-vh769\" (UID: \"da28351c-1c8b-40be-85a0-3aad427e3a17\") " pod="openshift-ingress/router-default-7dfcf9fd6c-vh769"
Oct 30 18:33:45.007902 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.007743    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7c09088b-3a07-426d-8804-f0f5bac4d27c-tmp\") pod \"insights-operator-7c4986d449-xvgqc\" (UID: \"7c09088b-3a07-426d-8804-f0f5bac4d27c\") " pod="openshift-insights/insights-operator-7c4986d449-xvgqc"
Oct 30 18:33:45.007902 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.007768    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8789ec5b-226e-4551-bf68-3ac585006266-registry-tls\") pod \"image-registry-74757bccd5-qmrkp\" (UID: \"8789ec5b-226e-4551-bf68-3ac585006266\") " pod="openshift-image-registry/image-registry-74757bccd5-qmrkp"
Oct 30 18:33:45.007902 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.007803    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/71fce796-0032-481b-80b3-66fbb731452c-nginx-conf\") pod \"networking-console-plugin-86c478bb88-r8rgt\" (UID: \"71fce796-0032-481b-80b3-66fbb731452c\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-r8rgt"
Oct 30 18:33:45.007902 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.007826    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/da28351c-1c8b-40be-85a0-3aad427e3a17-stats-auth\") pod \"router-default-7dfcf9fd6c-vh769\" (UID: \"da28351c-1c8b-40be-85a0-3aad427e3a17\") " pod="openshift-ingress/router-default-7dfcf9fd6c-vh769"
Oct 30 18:33:45.007902 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.007863    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/da28351c-1c8b-40be-85a0-3aad427e3a17-service-ca-bundle\") pod \"router-default-7dfcf9fd6c-vh769\" (UID: \"da28351c-1c8b-40be-85a0-3aad427e3a17\") " pod="openshift-ingress/router-default-7dfcf9fd6c-vh769"
Oct 30 18:33:45.007902 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.007880    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cvxhl\" (UniqueName: \"kubernetes.io/projected/7c09088b-3a07-426d-8804-f0f5bac4d27c-kube-api-access-cvxhl\") pod \"insights-operator-7c4986d449-xvgqc\" (UID: \"7c09088b-3a07-426d-8804-f0f5bac4d27c\") " pod="openshift-insights/insights-operator-7c4986d449-xvgqc"
Oct 30 18:33:45.008114 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.007913    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vf948\" (UniqueName: \"kubernetes.io/projected/8789ec5b-226e-4551-bf68-3ac585006266-kube-api-access-vf948\") pod \"image-registry-74757bccd5-qmrkp\" (UID: \"8789ec5b-226e-4551-bf68-3ac585006266\") " pod="openshift-image-registry/image-registry-74757bccd5-qmrkp"
Oct 30 18:33:45.008114 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.007946    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-btt6c\" (UniqueName: \"kubernetes.io/projected/dedb8b54-d0ce-434c-9461-db90a189c121-kube-api-access-btt6c\") pod \"kas-connection-checker-685895c7c9-4b95q\" (UID: \"dedb8b54-d0ce-434c-9461-db90a189c121\") " pod="kube-system/kas-connection-checker-685895c7c9-4b95q"
Oct 30 18:33:45.008114 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.007978    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/09c13759-50d4-40e6-b854-4691822b8c8a-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-5bqdc\" (UID: \"09c13759-50d4-40e6-b854-4691822b8c8a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-5bqdc"
Oct 30 18:33:45.008114 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.008002    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/382e8b31-a7fa-43f3-9f15-4c61627b0c04-config\") pod \"console-operator-7cd56b8dbc-7x4rb\" (UID: \"382e8b31-a7fa-43f3-9f15-4c61627b0c04\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-7x4rb"
Oct 30 18:33:45.008114 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.008056    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/68195e19-b1ff-4148-9fae-3545f2997ff5-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-gmswj\" (UID: \"68195e19-b1ff-4148-9fae-3545f2997ff5\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-gmswj"
Oct 30 18:33:45.008114 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.008082    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/15d485f9-5a4b-4f4b-9e59-a6eb062b4c5c-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-pm58m\" (UID: \"15d485f9-5a4b-4f4b-9e59-a6eb062b4c5c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-pm58m"
Oct 30 18:33:45.008114 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.008106    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8789ec5b-226e-4551-bf68-3ac585006266-ca-trust-extracted\") pod \"image-registry-74757bccd5-qmrkp\" (UID: \"8789ec5b-226e-4551-bf68-3ac585006266\") " pod="openshift-image-registry/image-registry-74757bccd5-qmrkp"
Oct 30 18:33:45.008324 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.008120    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4rztt\" (UniqueName: \"kubernetes.io/projected/68195e19-b1ff-4148-9fae-3545f2997ff5-kube-api-access-4rztt\") pod \"cluster-monitoring-operator-884576c57-gmswj\" (UID: \"68195e19-b1ff-4148-9fae-3545f2997ff5\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-gmswj"
Oct 30 18:33:45.008324 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.008142    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351-registry-certificates\") pod \"image-registry-64bf5b6b76-dpvvs\" (UID: \"9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351\") " pod="openshift-image-registry/image-registry-64bf5b6b76-dpvvs"
Oct 30 18:33:45.008324 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.008180    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/382e8b31-a7fa-43f3-9f15-4c61627b0c04-trusted-ca\") pod \"console-operator-7cd56b8dbc-7x4rb\" (UID: \"382e8b31-a7fa-43f3-9f15-4c61627b0c04\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-7x4rb"
Oct 30 18:33:45.008324 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.008233    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/7c09088b-3a07-426d-8804-f0f5bac4d27c-snapshots\") pod \"insights-operator-7c4986d449-xvgqc\" (UID: \"7c09088b-3a07-426d-8804-f0f5bac4d27c\") " pod="openshift-insights/insights-operator-7c4986d449-xvgqc"
Oct 30 18:33:45.008324 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.008258    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/36a04851-81db-40cd-97ef-8e2e00a08604-serving-cert\") pod \"service-ca-operator-544d749c8c-kr5qq\" (UID: \"36a04851-81db-40cd-97ef-8e2e00a08604\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-kr5qq"
Oct 30 18:33:45.008324 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.008275    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8789ec5b-226e-4551-bf68-3ac585006266-trusted-ca\") pod \"image-registry-74757bccd5-qmrkp\" (UID: \"8789ec5b-226e-4551-bf68-3ac585006266\") " pod="openshift-image-registry/image-registry-74757bccd5-qmrkp"
Oct 30 18:33:45.008324 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.008310    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/71fce796-0032-481b-80b3-66fbb731452c-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-r8rgt\" (UID: \"71fce796-0032-481b-80b3-66fbb731452c\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-r8rgt"
Oct 30 18:33:45.008525 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.008338    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351-bound-sa-token\") pod \"image-registry-64bf5b6b76-dpvvs\" (UID: \"9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351\") " pod="openshift-image-registry/image-registry-64bf5b6b76-dpvvs"
Oct 30 18:33:45.008525 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.008367    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8789ec5b-226e-4551-bf68-3ac585006266-registry-certificates\") pod \"image-registry-74757bccd5-qmrkp\" (UID: \"8789ec5b-226e-4551-bf68-3ac585006266\") " pod="openshift-image-registry/image-registry-74757bccd5-qmrkp"
Oct 30 18:33:45.008525 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.008382    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kk8qr\" (UniqueName: \"kubernetes.io/projected/f4ac17da-6b5b-4441-9b4e-ebe696e06507-kube-api-access-kk8qr\") pod \"volume-data-source-validator-58c9f5f46d-jwwq2\" (UID: \"f4ac17da-6b5b-4441-9b4e-ebe696e06507\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-jwwq2"
Oct 30 18:33:45.008525 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.008405    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351-image-registry-private-configuration\") pod \"image-registry-64bf5b6b76-dpvvs\" (UID: \"9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351\") " pod="openshift-image-registry/image-registry-64bf5b6b76-dpvvs"
Oct 30 18:33:45.008525 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.008423    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x4hfg\" (UniqueName: \"kubernetes.io/projected/e77c4479-d6e5-48a8-ac82-102a9f1a2c4a-kube-api-access-x4hfg\") pod \"kas-connection-checker-685895c7c9-2grbp\" (UID: \"e77c4479-d6e5-48a8-ac82-102a9f1a2c4a\") " pod="kube-system/kas-connection-checker-685895c7c9-2grbp"
Oct 30 18:33:45.008525 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.008444    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/15d485f9-5a4b-4f4b-9e59-a6eb062b4c5c-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-pm58m\" (UID: \"15d485f9-5a4b-4f4b-9e59-a6eb062b4c5c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-pm58m"
Oct 30 18:33:45.008525 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.008477    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8789ec5b-226e-4551-bf68-3ac585006266-installation-pull-secrets\") pod \"image-registry-74757bccd5-qmrkp\" (UID: \"8789ec5b-226e-4551-bf68-3ac585006266\") " pod="openshift-image-registry/image-registry-74757bccd5-qmrkp"
Oct 30 18:33:45.008525 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.008518    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nk8r7\" (UniqueName: \"kubernetes.io/projected/df9e5074-3bb1-4817-8855-b045cee61004-kube-api-access-nk8r7\") pod \"kas-connection-checker-685895c7c9-f9ldt\" (UID: \"df9e5074-3bb1-4817-8855-b045cee61004\") " pod="kube-system/kas-connection-checker-685895c7c9-f9ldt"
Oct 30 18:33:45.008736 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.008548    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9j2qw\" (UniqueName: \"kubernetes.io/projected/81de5dce-c40f-4bd5-878c-5d75a0487066-kube-api-access-9j2qw\") pod \"network-check-source-6f78d59f99-4969l\" (UID: \"81de5dce-c40f-4bd5-878c-5d75a0487066\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-4969l"
Oct 30 18:33:45.008736 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.008633    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351-trusted-ca\") pod \"image-registry-64bf5b6b76-dpvvs\" (UID: \"9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351\") " pod="openshift-image-registry/image-registry-64bf5b6b76-dpvvs"
Oct 30 18:33:45.008736 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.008653    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/382e8b31-a7fa-43f3-9f15-4c61627b0c04-serving-cert\") pod \"console-operator-7cd56b8dbc-7x4rb\" (UID: \"382e8b31-a7fa-43f3-9f15-4c61627b0c04\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-7x4rb"
Oct 30 18:33:45.008736 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.008667    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/da28351c-1c8b-40be-85a0-3aad427e3a17-default-certificate\") pod \"router-default-7dfcf9fd6c-vh769\" (UID: \"da28351c-1c8b-40be-85a0-3aad427e3a17\") " pod="openshift-ingress/router-default-7dfcf9fd6c-vh769"
Oct 30 18:33:45.008736 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.008700    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rw8g9\" (UniqueName: \"kubernetes.io/projected/36a04851-81db-40cd-97ef-8e2e00a08604-kube-api-access-rw8g9\") pod \"service-ca-operator-544d749c8c-kr5qq\" (UID: \"36a04851-81db-40cd-97ef-8e2e00a08604\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-kr5qq"
Oct 30 18:33:45.008736 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.008726    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7c09088b-3a07-426d-8804-f0f5bac4d27c-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-xvgqc\" (UID: \"7c09088b-3a07-426d-8804-f0f5bac4d27c\") " pod="openshift-insights/insights-operator-7c4986d449-xvgqc"
Oct 30 18:33:45.013409 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.013390    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Oct 30 18:33:45.013478 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.013394    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Oct 30 18:33:45.013478 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.013390    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Oct 30 18:33:45.013478 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.013400    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Oct 30 18:33:45.013570 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.013497    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Oct 30 18:33:45.040985 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.040795    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Oct 30 18:33:45.040985 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.040896    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Oct 30 18:33:45.040985 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.040947    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Oct 30 18:33:45.040985 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.040991    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Oct 30 18:33:45.041262 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.041247    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Oct 30 18:33:45.041857 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.041841    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Oct 30 18:33:45.042329 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.042314    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Oct 30 18:33:45.042374 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.042337    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Oct 30 18:33:45.042510 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.042457    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Oct 30 18:33:45.042557 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.042545    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Oct 30 18:33:45.063630 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.063564    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Oct 30 18:33:45.063771 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.063655    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Oct 30 18:33:45.063771 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.063716    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Oct 30 18:33:45.064541 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.064527    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-7c4986d449-xvgqc"]
Oct 30 18:33:45.091514 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.091462    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-64bf5b6b76-dpvvs"]
Oct 30 18:33:45.091758 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.091740    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Oct 30 18:33:45.091899 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.091776    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Oct 30 18:33:45.091899 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.091830    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-b9rv9\""
Oct 30 18:33:45.091899 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.091869    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Oct 30 18:33:45.091899 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.091890    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Oct 30 18:33:45.092371 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.091952    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-lmrrc\""
Oct 30 18:33:45.092371 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.091957    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Oct 30 18:33:45.092371 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.091976    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Oct 30 18:33:45.092371 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.091990    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Oct 30 18:33:45.092371 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.091993    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-blpnv\""
Oct 30 18:33:45.092371 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.092003    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Oct 30 18:33:45.092371 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.092041    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-qv4ml\""
Oct 30 18:33:45.092371 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.092060    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Oct 30 18:33:45.092371 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.092108    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Oct 30 18:33:45.092371 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.092188    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Oct 30 18:33:45.092371 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.092219    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Oct 30 18:33:45.092371 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.092219    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Oct 30 18:33:45.092371 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.092273    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-brwnt\""
Oct 30 18:33:45.092371 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.092276    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-c65fk\""
Oct 30 18:33:45.092371 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.092314    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-lwjtg\""
Oct 30 18:33:45.092371 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.092358    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-pw2xp\""
Oct 30 18:33:45.092889 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.092402    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-sqz88\""
Oct 30 18:33:45.092889 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.092485    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Oct 30 18:33:45.092889 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.092493    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Oct 30 18:33:45.092889 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.092508    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Oct 30 18:33:45.092889 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.092517    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Oct 30 18:33:45.092889 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.092525    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-config\""
Oct 30 18:33:45.092889 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.092529    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Oct 30 18:33:45.092889 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.092560    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-r2nx2"]
Oct 30 18:33:45.092889 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.092565    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Oct 30 18:33:45.092889 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.092571    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Oct 30 18:33:45.092889 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.092586    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-rqt8c\""
Oct 30 18:33:45.092889 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.092614    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-lh6d4\""
Oct 30 18:33:45.092889 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.092625    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-77vj9\""
Oct 30 18:33:45.092889 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.092636    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Oct 30 18:33:45.092889 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.092692    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Oct 30 18:33:45.093350 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.093242    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-7dfcf9fd6c-vh769"]
Oct 30 18:33:45.093803 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.093789    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-6f78d59f99-4969l"]
Oct 30 18:33:45.094378 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.094364    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-74757bccd5-qmrkp"]
Oct 30 18:33:45.094911 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.094895    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-g9fhp"]
Oct 30 18:33:45.096492 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.096476    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nk8r7\" (UniqueName: \"kubernetes.io/projected/df9e5074-3bb1-4817-8855-b045cee61004-kube-api-access-nk8r7\") pod \"kas-connection-checker-685895c7c9-f9ldt\" (UID: \"df9e5074-3bb1-4817-8855-b045cee61004\") " pod="kube-system/kas-connection-checker-685895c7c9-f9ldt"
Oct 30 18:33:45.109415 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.109391    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/71fce796-0032-481b-80b3-66fbb731452c-nginx-conf\") pod \"networking-console-plugin-86c478bb88-r8rgt\" (UID: \"71fce796-0032-481b-80b3-66fbb731452c\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-r8rgt"
Oct 30 18:33:45.109489 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.109417    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/da28351c-1c8b-40be-85a0-3aad427e3a17-stats-auth\") pod \"router-default-7dfcf9fd6c-vh769\" (UID: \"da28351c-1c8b-40be-85a0-3aad427e3a17\") " pod="openshift-ingress/router-default-7dfcf9fd6c-vh769"
Oct 30 18:33:45.109489 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.109436    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/97b19aa3-42be-4def-8d7a-8e3516852e79-metrics-tls\") pod \"dns-default-r2nx2\" (UID: \"97b19aa3-42be-4def-8d7a-8e3516852e79\") " pod="openshift-dns/dns-default-r2nx2"
Oct 30 18:33:45.109489 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.109451    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-24cp9\" (UniqueName: \"kubernetes.io/projected/e6b2a584-ae78-4e80-b595-9b1a5897d556-kube-api-access-24cp9\") pod \"ingress-canary-g9fhp\" (UID: \"e6b2a584-ae78-4e80-b595-9b1a5897d556\") " pod="openshift-ingress-canary/ingress-canary-g9fhp"
Oct 30 18:33:45.109489 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.109478    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/da28351c-1c8b-40be-85a0-3aad427e3a17-service-ca-bundle\") pod \"router-default-7dfcf9fd6c-vh769\" (UID: \"da28351c-1c8b-40be-85a0-3aad427e3a17\") " pod="openshift-ingress/router-default-7dfcf9fd6c-vh769"
Oct 30 18:33:45.109599 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.109537    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cvxhl\" (UniqueName: \"kubernetes.io/projected/7c09088b-3a07-426d-8804-f0f5bac4d27c-kube-api-access-cvxhl\") pod \"insights-operator-7c4986d449-xvgqc\" (UID: \"7c09088b-3a07-426d-8804-f0f5bac4d27c\") " pod="openshift-insights/insights-operator-7c4986d449-xvgqc"
Oct 30 18:33:45.109599 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.109561    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vf948\" (UniqueName: \"kubernetes.io/projected/8789ec5b-226e-4551-bf68-3ac585006266-kube-api-access-vf948\") pod \"image-registry-74757bccd5-qmrkp\" (UID: \"8789ec5b-226e-4551-bf68-3ac585006266\") " pod="openshift-image-registry/image-registry-74757bccd5-qmrkp"
Oct 30 18:33:45.109599 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.109578    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-btt6c\" (UniqueName: \"kubernetes.io/projected/dedb8b54-d0ce-434c-9461-db90a189c121-kube-api-access-btt6c\") pod \"kas-connection-checker-685895c7c9-4b95q\" (UID: \"dedb8b54-d0ce-434c-9461-db90a189c121\") " pod="kube-system/kas-connection-checker-685895c7c9-4b95q"
Oct 30 18:33:45.109734 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.109597    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/09c13759-50d4-40e6-b854-4691822b8c8a-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-5bqdc\" (UID: \"09c13759-50d4-40e6-b854-4691822b8c8a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-5bqdc"
Oct 30 18:33:45.109734 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.109621    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/382e8b31-a7fa-43f3-9f15-4c61627b0c04-config\") pod \"console-operator-7cd56b8dbc-7x4rb\" (UID: \"382e8b31-a7fa-43f3-9f15-4c61627b0c04\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-7x4rb"
Oct 30 18:33:45.109828 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:45.109729    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/da28351c-1c8b-40be-85a0-3aad427e3a17-service-ca-bundle podName:da28351c-1c8b-40be-85a0-3aad427e3a17 nodeName:}" failed. No retries permitted until 2026-10-30 18:33:45.609708106 +0000 UTC m=+32.101142581 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/da28351c-1c8b-40be-85a0-3aad427e3a17-service-ca-bundle") pod "router-default-7dfcf9fd6c-vh769" (UID: "da28351c-1c8b-40be-85a0-3aad427e3a17") : configmap references non-existent config key: service-ca.crt
Oct 30 18:33:45.109828 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.109778    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/68195e19-b1ff-4148-9fae-3545f2997ff5-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-gmswj\" (UID: \"68195e19-b1ff-4148-9fae-3545f2997ff5\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-gmswj"
Oct 30 18:33:45.109828 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.109814    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e6b2a584-ae78-4e80-b595-9b1a5897d556-cert\") pod \"ingress-canary-g9fhp\" (UID: \"e6b2a584-ae78-4e80-b595-9b1a5897d556\") " pod="openshift-ingress-canary/ingress-canary-g9fhp"
Oct 30 18:33:45.109977 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.109843    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/15d485f9-5a4b-4f4b-9e59-a6eb062b4c5c-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-pm58m\" (UID: \"15d485f9-5a4b-4f4b-9e59-a6eb062b4c5c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-pm58m"
Oct 30 18:33:45.109977 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.109874    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8789ec5b-226e-4551-bf68-3ac585006266-ca-trust-extracted\") pod \"image-registry-74757bccd5-qmrkp\" (UID: \"8789ec5b-226e-4551-bf68-3ac585006266\") " pod="openshift-image-registry/image-registry-74757bccd5-qmrkp"
Oct 30 18:33:45.109977 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.109900    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4rztt\" (UniqueName: \"kubernetes.io/projected/68195e19-b1ff-4148-9fae-3545f2997ff5-kube-api-access-4rztt\") pod \"cluster-monitoring-operator-884576c57-gmswj\" (UID: \"68195e19-b1ff-4148-9fae-3545f2997ff5\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-gmswj"
Oct 30 18:33:45.109977 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.109926    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351-registry-certificates\") pod \"image-registry-64bf5b6b76-dpvvs\" (UID: \"9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351\") " pod="openshift-image-registry/image-registry-64bf5b6b76-dpvvs"
Oct 30 18:33:45.109977 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.109952    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/382e8b31-a7fa-43f3-9f15-4c61627b0c04-trusted-ca\") pod \"console-operator-7cd56b8dbc-7x4rb\" (UID: \"382e8b31-a7fa-43f3-9f15-4c61627b0c04\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-7x4rb"
Oct 30 18:33:45.110201 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.109978    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tc2fp\" (UniqueName: \"kubernetes.io/projected/97b19aa3-42be-4def-8d7a-8e3516852e79-kube-api-access-tc2fp\") pod \"dns-default-r2nx2\" (UID: \"97b19aa3-42be-4def-8d7a-8e3516852e79\") " pod="openshift-dns/dns-default-r2nx2"
Oct 30 18:33:45.110506 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.110323    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/71fce796-0032-481b-80b3-66fbb731452c-nginx-conf\") pod \"networking-console-plugin-86c478bb88-r8rgt\" (UID: \"71fce796-0032-481b-80b3-66fbb731452c\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-r8rgt"
Oct 30 18:33:45.110506 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.110369    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/382e8b31-a7fa-43f3-9f15-4c61627b0c04-config\") pod \"console-operator-7cd56b8dbc-7x4rb\" (UID: \"382e8b31-a7fa-43f3-9f15-4c61627b0c04\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-7x4rb"
Oct 30 18:33:45.110506 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.110371    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/09c13759-50d4-40e6-b854-4691822b8c8a-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-5bqdc\" (UID: \"09c13759-50d4-40e6-b854-4691822b8c8a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-5bqdc"
Oct 30 18:33:45.110506 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.110453    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/7c09088b-3a07-426d-8804-f0f5bac4d27c-snapshots\") pod \"insights-operator-7c4986d449-xvgqc\" (UID: \"7c09088b-3a07-426d-8804-f0f5bac4d27c\") " pod="openshift-insights/insights-operator-7c4986d449-xvgqc"
Oct 30 18:33:45.111036 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.110735    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8789ec5b-226e-4551-bf68-3ac585006266-ca-trust-extracted\") pod \"image-registry-74757bccd5-qmrkp\" (UID: \"8789ec5b-226e-4551-bf68-3ac585006266\") " pod="openshift-image-registry/image-registry-74757bccd5-qmrkp"
Oct 30 18:33:45.111036 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.110981    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/68195e19-b1ff-4148-9fae-3545f2997ff5-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-gmswj\" (UID: \"68195e19-b1ff-4148-9fae-3545f2997ff5\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-gmswj"
Oct 30 18:33:45.111697 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.111173    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351-registry-certificates\") pod \"image-registry-64bf5b6b76-dpvvs\" (UID: \"9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351\") " pod="openshift-image-registry/image-registry-64bf5b6b76-dpvvs"
Oct 30 18:33:45.111697 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.111373    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/15d485f9-5a4b-4f4b-9e59-a6eb062b4c5c-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-pm58m\" (UID: \"15d485f9-5a4b-4f4b-9e59-a6eb062b4c5c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-pm58m"
Oct 30 18:33:45.111697 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.111425    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/7c09088b-3a07-426d-8804-f0f5bac4d27c-snapshots\") pod \"insights-operator-7c4986d449-xvgqc\" (UID: \"7c09088b-3a07-426d-8804-f0f5bac4d27c\") " pod="openshift-insights/insights-operator-7c4986d449-xvgqc"
Oct 30 18:33:45.111842 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.111795    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/382e8b31-a7fa-43f3-9f15-4c61627b0c04-trusted-ca\") pod \"console-operator-7cd56b8dbc-7x4rb\" (UID: \"382e8b31-a7fa-43f3-9f15-4c61627b0c04\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-7x4rb"
Oct 30 18:33:45.111947 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.111918    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/36a04851-81db-40cd-97ef-8e2e00a08604-serving-cert\") pod \"service-ca-operator-544d749c8c-kr5qq\" (UID: \"36a04851-81db-40cd-97ef-8e2e00a08604\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-kr5qq"
Oct 30 18:33:45.112050 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.111990    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8789ec5b-226e-4551-bf68-3ac585006266-trusted-ca\") pod \"image-registry-74757bccd5-qmrkp\" (UID: \"8789ec5b-226e-4551-bf68-3ac585006266\") " pod="openshift-image-registry/image-registry-74757bccd5-qmrkp"
Oct 30 18:33:45.112109 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.112027    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/71fce796-0032-481b-80b3-66fbb731452c-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-r8rgt\" (UID: \"71fce796-0032-481b-80b3-66fbb731452c\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-r8rgt"
Oct 30 18:33:45.112109 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.112094    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351-bound-sa-token\") pod \"image-registry-64bf5b6b76-dpvvs\" (UID: \"9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351\") " pod="openshift-image-registry/image-registry-64bf5b6b76-dpvvs"
Oct 30 18:33:45.112200 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.112132    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8789ec5b-226e-4551-bf68-3ac585006266-registry-certificates\") pod \"image-registry-74757bccd5-qmrkp\" (UID: \"8789ec5b-226e-4551-bf68-3ac585006266\") " pod="openshift-image-registry/image-registry-74757bccd5-qmrkp"
Oct 30 18:33:45.112200 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.112165    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kk8qr\" (UniqueName: \"kubernetes.io/projected/f4ac17da-6b5b-4441-9b4e-ebe696e06507-kube-api-access-kk8qr\") pod \"volume-data-source-validator-58c9f5f46d-jwwq2\" (UID: \"f4ac17da-6b5b-4441-9b4e-ebe696e06507\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-jwwq2"
Oct 30 18:33:45.112313 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.112192    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351-image-registry-private-configuration\") pod \"image-registry-64bf5b6b76-dpvvs\" (UID: \"9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351\") " pod="openshift-image-registry/image-registry-64bf5b6b76-dpvvs"
Oct 30 18:33:45.112313 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.112225    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/97b19aa3-42be-4def-8d7a-8e3516852e79-tmp-dir\") pod \"dns-default-r2nx2\" (UID: \"97b19aa3-42be-4def-8d7a-8e3516852e79\") " pod="openshift-dns/dns-default-r2nx2"
Oct 30 18:33:45.112313 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.112261    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x4hfg\" (UniqueName: \"kubernetes.io/projected/e77c4479-d6e5-48a8-ac82-102a9f1a2c4a-kube-api-access-x4hfg\") pod \"kas-connection-checker-685895c7c9-2grbp\" (UID: \"e77c4479-d6e5-48a8-ac82-102a9f1a2c4a\") " pod="kube-system/kas-connection-checker-685895c7c9-2grbp"
Oct 30 18:33:45.112454 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.112325    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/15d485f9-5a4b-4f4b-9e59-a6eb062b4c5c-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-pm58m\" (UID: \"15d485f9-5a4b-4f4b-9e59-a6eb062b4c5c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-pm58m"
Oct 30 18:33:45.112454 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.112361    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8789ec5b-226e-4551-bf68-3ac585006266-installation-pull-secrets\") pod \"image-registry-74757bccd5-qmrkp\" (UID: \"8789ec5b-226e-4551-bf68-3ac585006266\") " pod="openshift-image-registry/image-registry-74757bccd5-qmrkp"
Oct 30 18:33:45.112454 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.112394    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9j2qw\" (UniqueName: \"kubernetes.io/projected/81de5dce-c40f-4bd5-878c-5d75a0487066-kube-api-access-9j2qw\") pod \"network-check-source-6f78d59f99-4969l\" (UID: \"81de5dce-c40f-4bd5-878c-5d75a0487066\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-4969l"
Oct 30 18:33:45.112454 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.112425    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351-trusted-ca\") pod \"image-registry-64bf5b6b76-dpvvs\" (UID: \"9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351\") " pod="openshift-image-registry/image-registry-64bf5b6b76-dpvvs"
Oct 30 18:33:45.112626 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.112455    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/382e8b31-a7fa-43f3-9f15-4c61627b0c04-serving-cert\") pod \"console-operator-7cd56b8dbc-7x4rb\" (UID: \"382e8b31-a7fa-43f3-9f15-4c61627b0c04\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-7x4rb"
Oct 30 18:33:45.112626 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.112485    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/da28351c-1c8b-40be-85a0-3aad427e3a17-default-certificate\") pod \"router-default-7dfcf9fd6c-vh769\" (UID: \"da28351c-1c8b-40be-85a0-3aad427e3a17\") " pod="openshift-ingress/router-default-7dfcf9fd6c-vh769"
Oct 30 18:33:45.112626 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.112517    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rw8g9\" (UniqueName: \"kubernetes.io/projected/36a04851-81db-40cd-97ef-8e2e00a08604-kube-api-access-rw8g9\") pod \"service-ca-operator-544d749c8c-kr5qq\" (UID: \"36a04851-81db-40cd-97ef-8e2e00a08604\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-kr5qq"
Oct 30 18:33:45.112626 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.112542    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7c09088b-3a07-426d-8804-f0f5bac4d27c-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-xvgqc\" (UID: \"7c09088b-3a07-426d-8804-f0f5bac4d27c\") " pod="openshift-insights/insights-operator-7c4986d449-xvgqc"
Oct 30 18:33:45.112626 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.112571    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351-registry-tls\") pod \"image-registry-64bf5b6b76-dpvvs\" (UID: \"9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351\") " pod="openshift-image-registry/image-registry-64bf5b6b76-dpvvs"
Oct 30 18:33:45.112626 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.112603    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fzrkt\" (UniqueName: \"kubernetes.io/projected/15d485f9-5a4b-4f4b-9e59-a6eb062b4c5c-kube-api-access-fzrkt\") pod \"cluster-samples-operator-6ddb6c8b94-pm58m\" (UID: \"15d485f9-5a4b-4f4b-9e59-a6eb062b4c5c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-pm58m"
Oct 30 18:33:45.112871 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.112640    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8789ec5b-226e-4551-bf68-3ac585006266-image-registry-private-configuration\") pod \"image-registry-74757bccd5-qmrkp\" (UID: \"8789ec5b-226e-4551-bf68-3ac585006266\") " pod="openshift-image-registry/image-registry-74757bccd5-qmrkp"
Oct 30 18:33:45.112871 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.112670    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/68195e19-b1ff-4148-9fae-3545f2997ff5-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-gmswj\" (UID: \"68195e19-b1ff-4148-9fae-3545f2997ff5\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-gmswj"
Oct 30 18:33:45.112871 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.112702    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vxw47\" (UniqueName: \"kubernetes.io/projected/9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351-kube-api-access-vxw47\") pod \"image-registry-64bf5b6b76-dpvvs\" (UID: \"9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351\") " pod="openshift-image-registry/image-registry-64bf5b6b76-dpvvs"
Oct 30 18:33:45.112871 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.112727    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/36a04851-81db-40cd-97ef-8e2e00a08604-config\") pod \"service-ca-operator-544d749c8c-kr5qq\" (UID: \"36a04851-81db-40cd-97ef-8e2e00a08604\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-kr5qq"
Oct 30 18:33:45.112871 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.112756    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351-installation-pull-secrets\") pod \"image-registry-64bf5b6b76-dpvvs\" (UID: \"9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351\") " pod="openshift-image-registry/image-registry-64bf5b6b76-dpvvs"
Oct 30 18:33:45.112871 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.112790    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/09c13759-50d4-40e6-b854-4691822b8c8a-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-5bqdc\" (UID: \"09c13759-50d4-40e6-b854-4691822b8c8a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-5bqdc"
Oct 30 18:33:45.112871 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.112824    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m4j2q\" (UniqueName: \"kubernetes.io/projected/382e8b31-a7fa-43f3-9f15-4c61627b0c04-kube-api-access-m4j2q\") pod \"console-operator-7cd56b8dbc-7x4rb\" (UID: \"382e8b31-a7fa-43f3-9f15-4c61627b0c04\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-7x4rb"
Oct 30 18:33:45.112871 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.112853    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k5crl\" (UniqueName: \"kubernetes.io/projected/09c13759-50d4-40e6-b854-4691822b8c8a-kube-api-access-k5crl\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-5bqdc\" (UID: \"09c13759-50d4-40e6-b854-4691822b8c8a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-5bqdc"
Oct 30 18:33:45.113182 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.112881    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7c09088b-3a07-426d-8804-f0f5bac4d27c-serving-cert\") pod \"insights-operator-7c4986d449-xvgqc\" (UID: \"7c09088b-3a07-426d-8804-f0f5bac4d27c\") " pod="openshift-insights/insights-operator-7c4986d449-xvgqc"
Oct 30 18:33:45.113182 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.112919    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351-ca-trust-extracted\") pod \"image-registry-64bf5b6b76-dpvvs\" (UID: \"9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351\") " pod="openshift-image-registry/image-registry-64bf5b6b76-dpvvs"
Oct 30 18:33:45.113182 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.112950    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7c09088b-3a07-426d-8804-f0f5bac4d27c-service-ca-bundle\") pod \"insights-operator-7c4986d449-xvgqc\" (UID: \"7c09088b-3a07-426d-8804-f0f5bac4d27c\") " pod="openshift-insights/insights-operator-7c4986d449-xvgqc"
Oct 30 18:33:45.113182 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.112979    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8789ec5b-226e-4551-bf68-3ac585006266-bound-sa-token\") pod \"image-registry-74757bccd5-qmrkp\" (UID: \"8789ec5b-226e-4551-bf68-3ac585006266\") " pod="openshift-image-registry/image-registry-74757bccd5-qmrkp"
Oct 30 18:33:45.113182 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.113009    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/da28351c-1c8b-40be-85a0-3aad427e3a17-metrics-certs\") pod \"router-default-7dfcf9fd6c-vh769\" (UID: \"da28351c-1c8b-40be-85a0-3aad427e3a17\") " pod="openshift-ingress/router-default-7dfcf9fd6c-vh769"
Oct 30 18:33:45.113182 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.113035    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cftdb\" (UniqueName: \"kubernetes.io/projected/da28351c-1c8b-40be-85a0-3aad427e3a17-kube-api-access-cftdb\") pod \"router-default-7dfcf9fd6c-vh769\" (UID: \"da28351c-1c8b-40be-85a0-3aad427e3a17\") " pod="openshift-ingress/router-default-7dfcf9fd6c-vh769"
Oct 30 18:33:45.113182 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.113065    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/97b19aa3-42be-4def-8d7a-8e3516852e79-config-volume\") pod \"dns-default-r2nx2\" (UID: \"97b19aa3-42be-4def-8d7a-8e3516852e79\") " pod="openshift-dns/dns-default-r2nx2"
Oct 30 18:33:45.113182 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.113098    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7c09088b-3a07-426d-8804-f0f5bac4d27c-tmp\") pod \"insights-operator-7c4986d449-xvgqc\" (UID: \"7c09088b-3a07-426d-8804-f0f5bac4d27c\") " pod="openshift-insights/insights-operator-7c4986d449-xvgqc"
Oct 30 18:33:45.113182 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.113127    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8789ec5b-226e-4551-bf68-3ac585006266-registry-tls\") pod \"image-registry-74757bccd5-qmrkp\" (UID: \"8789ec5b-226e-4551-bf68-3ac585006266\") " pod="openshift-image-registry/image-registry-74757bccd5-qmrkp"
Oct 30 18:33:45.113562 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:45.113336    2574 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 30 18:33:45.113562 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:45.113351    2574 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-74757bccd5-qmrkp: secret "image-registry-tls" not found
Oct 30 18:33:45.113562 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:45.113406    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8789ec5b-226e-4551-bf68-3ac585006266-registry-tls podName:8789ec5b-226e-4551-bf68-3ac585006266 nodeName:}" failed. No retries permitted until 2026-10-30 18:33:45.613391078 +0000 UTC m=+32.104825531 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8789ec5b-226e-4551-bf68-3ac585006266-registry-tls") pod "image-registry-74757bccd5-qmrkp" (UID: "8789ec5b-226e-4551-bf68-3ac585006266") : secret "image-registry-tls" not found
Oct 30 18:33:45.116192 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.114313    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8789ec5b-226e-4551-bf68-3ac585006266-trusted-ca\") pod \"image-registry-74757bccd5-qmrkp\" (UID: \"8789ec5b-226e-4551-bf68-3ac585006266\") " pod="openshift-image-registry/image-registry-74757bccd5-qmrkp"
Oct 30 18:33:45.116192 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:45.114434    2574 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Oct 30 18:33:45.116192 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:45.114488    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/71fce796-0032-481b-80b3-66fbb731452c-networking-console-plugin-cert podName:71fce796-0032-481b-80b3-66fbb731452c nodeName:}" failed. No retries permitted until 2026-10-30 18:33:45.614472757 +0000 UTC m=+32.105907212 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/71fce796-0032-481b-80b3-66fbb731452c-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-r8rgt" (UID: "71fce796-0032-481b-80b3-66fbb731452c") : secret "networking-console-plugin-cert" not found
Oct 30 18:33:45.116192 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.115134    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8789ec5b-226e-4551-bf68-3ac585006266-registry-certificates\") pod \"image-registry-74757bccd5-qmrkp\" (UID: \"8789ec5b-226e-4551-bf68-3ac585006266\") " pod="openshift-image-registry/image-registry-74757bccd5-qmrkp"
Oct 30 18:33:45.116192 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:45.116094    2574 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Oct 30 18:33:45.116642 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:45.116244    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68195e19-b1ff-4148-9fae-3545f2997ff5-cluster-monitoring-operator-tls podName:68195e19-b1ff-4148-9fae-3545f2997ff5 nodeName:}" failed. No retries permitted until 2026-10-30 18:33:45.616150597 +0000 UTC m=+32.107585039 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/68195e19-b1ff-4148-9fae-3545f2997ff5-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-gmswj" (UID: "68195e19-b1ff-4148-9fae-3545f2997ff5") : secret "cluster-monitoring-operator-tls" not found
Oct 30 18:33:45.117037 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.116986    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/da28351c-1c8b-40be-85a0-3aad427e3a17-stats-auth\") pod \"router-default-7dfcf9fd6c-vh769\" (UID: \"da28351c-1c8b-40be-85a0-3aad427e3a17\") " pod="openshift-ingress/router-default-7dfcf9fd6c-vh769"
Oct 30 18:33:45.118532 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.117540    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351-trusted-ca\") pod \"image-registry-64bf5b6b76-dpvvs\" (UID: \"9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351\") " pod="openshift-image-registry/image-registry-64bf5b6b76-dpvvs"
Oct 30 18:33:45.118532 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.117938    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7c09088b-3a07-426d-8804-f0f5bac4d27c-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-xvgqc\" (UID: \"7c09088b-3a07-426d-8804-f0f5bac4d27c\") " pod="openshift-insights/insights-operator-7c4986d449-xvgqc"
Oct 30 18:33:45.118532 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:45.118531    2574 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 30 18:33:45.118758 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:45.118548    2574 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-64bf5b6b76-dpvvs: secret "image-registry-tls" not found
Oct 30 18:33:45.118758 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:45.118609    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351-registry-tls podName:9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351 nodeName:}" failed. No retries permitted until 2026-10-30 18:33:45.61859349 +0000 UTC m=+32.110027932 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351-registry-tls") pod "image-registry-64bf5b6b76-dpvvs" (UID: "9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351") : secret "image-registry-tls" not found
Oct 30 18:33:45.118758 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.118743    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351-ca-trust-extracted\") pod \"image-registry-64bf5b6b76-dpvvs\" (UID: \"9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351\") " pod="openshift-image-registry/image-registry-64bf5b6b76-dpvvs"
Oct 30 18:33:45.121110 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:45.119350    2574 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Oct 30 18:33:45.121110 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:45.119414    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/da28351c-1c8b-40be-85a0-3aad427e3a17-metrics-certs podName:da28351c-1c8b-40be-85a0-3aad427e3a17 nodeName:}" failed. No retries permitted until 2026-10-30 18:33:45.619397794 +0000 UTC m=+32.110832255 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/da28351c-1c8b-40be-85a0-3aad427e3a17-metrics-certs") pod "router-default-7dfcf9fd6c-vh769" (UID: "da28351c-1c8b-40be-85a0-3aad427e3a17") : secret "router-metrics-certs-default" not found
Oct 30 18:33:45.121110 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.119534    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7c09088b-3a07-426d-8804-f0f5bac4d27c-service-ca-bundle\") pod \"insights-operator-7c4986d449-xvgqc\" (UID: \"7c09088b-3a07-426d-8804-f0f5bac4d27c\") " pod="openshift-insights/insights-operator-7c4986d449-xvgqc"
Oct 30 18:33:45.121110 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.119848    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7c09088b-3a07-426d-8804-f0f5bac4d27c-tmp\") pod \"insights-operator-7c4986d449-xvgqc\" (UID: \"7c09088b-3a07-426d-8804-f0f5bac4d27c\") " pod="openshift-insights/insights-operator-7c4986d449-xvgqc"
Oct 30 18:33:45.121110 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.119881    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/da28351c-1c8b-40be-85a0-3aad427e3a17-default-certificate\") pod \"router-default-7dfcf9fd6c-vh769\" (UID: \"da28351c-1c8b-40be-85a0-3aad427e3a17\") " pod="openshift-ingress/router-default-7dfcf9fd6c-vh769"
Oct 30 18:33:45.121110 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:45.120220    2574 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Oct 30 18:33:45.121110 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.120212    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/382e8b31-a7fa-43f3-9f15-4c61627b0c04-serving-cert\") pod \"console-operator-7cd56b8dbc-7x4rb\" (UID: \"382e8b31-a7fa-43f3-9f15-4c61627b0c04\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-7x4rb"
Oct 30 18:33:45.121110 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:45.120405    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/15d485f9-5a4b-4f4b-9e59-a6eb062b4c5c-samples-operator-tls podName:15d485f9-5a4b-4f4b-9e59-a6eb062b4c5c nodeName:}" failed. No retries permitted until 2026-10-30 18:33:45.620305949 +0000 UTC m=+32.111740404 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/15d485f9-5a4b-4f4b-9e59-a6eb062b4c5c-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-pm58m" (UID: "15d485f9-5a4b-4f4b-9e59-a6eb062b4c5c") : secret "samples-operator-tls" not found
Oct 30 18:33:45.121110 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.120873    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/36a04851-81db-40cd-97ef-8e2e00a08604-config\") pod \"service-ca-operator-544d749c8c-kr5qq\" (UID: \"36a04851-81db-40cd-97ef-8e2e00a08604\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-kr5qq"
Oct 30 18:33:45.121110 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.120923    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/36a04851-81db-40cd-97ef-8e2e00a08604-serving-cert\") pod \"service-ca-operator-544d749c8c-kr5qq\" (UID: \"36a04851-81db-40cd-97ef-8e2e00a08604\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-kr5qq"
Oct 30 18:33:45.121110 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.121001    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7c09088b-3a07-426d-8804-f0f5bac4d27c-serving-cert\") pod \"insights-operator-7c4986d449-xvgqc\" (UID: \"7c09088b-3a07-426d-8804-f0f5bac4d27c\") " pod="openshift-insights/insights-operator-7c4986d449-xvgqc"
Oct 30 18:33:45.121110 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.121003    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351-image-registry-private-configuration\") pod \"image-registry-64bf5b6b76-dpvvs\" (UID: \"9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351\") " pod="openshift-image-registry/image-registry-64bf5b6b76-dpvvs"
Oct 30 18:33:45.122006 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.121979    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8789ec5b-226e-4551-bf68-3ac585006266-installation-pull-secrets\") pod \"image-registry-74757bccd5-qmrkp\" (UID: \"8789ec5b-226e-4551-bf68-3ac585006266\") " pod="openshift-image-registry/image-registry-74757bccd5-qmrkp"
Oct 30 18:33:45.122830 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.122800    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/09c13759-50d4-40e6-b854-4691822b8c8a-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-5bqdc\" (UID: \"09c13759-50d4-40e6-b854-4691822b8c8a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-5bqdc"
Oct 30 18:33:45.122915 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.122898    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8789ec5b-226e-4551-bf68-3ac585006266-image-registry-private-configuration\") pod \"image-registry-74757bccd5-qmrkp\" (UID: \"8789ec5b-226e-4551-bf68-3ac585006266\") " pod="openshift-image-registry/image-registry-74757bccd5-qmrkp"
Oct 30 18:33:45.123032 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.123017    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351-installation-pull-secrets\") pod \"image-registry-64bf5b6b76-dpvvs\" (UID: \"9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351\") " pod="openshift-image-registry/image-registry-64bf5b6b76-dpvvs"
Oct 30 18:33:45.147947 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.147927    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-f9ldt"
Oct 30 18:33:45.173654 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.173627    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vf948\" (UniqueName: \"kubernetes.io/projected/8789ec5b-226e-4551-bf68-3ac585006266-kube-api-access-vf948\") pod \"image-registry-74757bccd5-qmrkp\" (UID: \"8789ec5b-226e-4551-bf68-3ac585006266\") " pod="openshift-image-registry/image-registry-74757bccd5-qmrkp"
Oct 30 18:33:45.173809 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.173694    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4rztt\" (UniqueName: \"kubernetes.io/projected/68195e19-b1ff-4148-9fae-3545f2997ff5-kube-api-access-4rztt\") pod \"cluster-monitoring-operator-884576c57-gmswj\" (UID: \"68195e19-b1ff-4148-9fae-3545f2997ff5\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-gmswj"
Oct 30 18:33:45.173809 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.173746    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cvxhl\" (UniqueName: \"kubernetes.io/projected/7c09088b-3a07-426d-8804-f0f5bac4d27c-kube-api-access-cvxhl\") pod \"insights-operator-7c4986d449-xvgqc\" (UID: \"7c09088b-3a07-426d-8804-f0f5bac4d27c\") " pod="openshift-insights/insights-operator-7c4986d449-xvgqc"
Oct 30 18:33:45.176225 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.176201    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kk8qr\" (UniqueName: \"kubernetes.io/projected/f4ac17da-6b5b-4441-9b4e-ebe696e06507-kube-api-access-kk8qr\") pod \"volume-data-source-validator-58c9f5f46d-jwwq2\" (UID: \"f4ac17da-6b5b-4441-9b4e-ebe696e06507\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-jwwq2"
Oct 30 18:33:45.177180 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.177151    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351-bound-sa-token\") pod \"image-registry-64bf5b6b76-dpvvs\" (UID: \"9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351\") " pod="openshift-image-registry/image-registry-64bf5b6b76-dpvvs"
Oct 30 18:33:45.177987 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.177965    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-btt6c\" (UniqueName: \"kubernetes.io/projected/dedb8b54-d0ce-434c-9461-db90a189c121-kube-api-access-btt6c\") pod \"kas-connection-checker-685895c7c9-4b95q\" (UID: \"dedb8b54-d0ce-434c-9461-db90a189c121\") " pod="kube-system/kas-connection-checker-685895c7c9-4b95q"
Oct 30 18:33:45.181532 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.181451    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x4hfg\" (UniqueName: \"kubernetes.io/projected/e77c4479-d6e5-48a8-ac82-102a9f1a2c4a-kube-api-access-x4hfg\") pod \"kas-connection-checker-685895c7c9-2grbp\" (UID: \"e77c4479-d6e5-48a8-ac82-102a9f1a2c4a\") " pod="kube-system/kas-connection-checker-685895c7c9-2grbp"
Oct 30 18:33:45.182258 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.182238    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m4j2q\" (UniqueName: \"kubernetes.io/projected/382e8b31-a7fa-43f3-9f15-4c61627b0c04-kube-api-access-m4j2q\") pod \"console-operator-7cd56b8dbc-7x4rb\" (UID: \"382e8b31-a7fa-43f3-9f15-4c61627b0c04\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-7x4rb"
Oct 30 18:33:45.183039 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.183016    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fzrkt\" (UniqueName: \"kubernetes.io/projected/15d485f9-5a4b-4f4b-9e59-a6eb062b4c5c-kube-api-access-fzrkt\") pod \"cluster-samples-operator-6ddb6c8b94-pm58m\" (UID: \"15d485f9-5a4b-4f4b-9e59-a6eb062b4c5c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-pm58m"
Oct 30 18:33:45.183593 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.183572    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k5crl\" (UniqueName: \"kubernetes.io/projected/09c13759-50d4-40e6-b854-4691822b8c8a-kube-api-access-k5crl\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-5bqdc\" (UID: \"09c13759-50d4-40e6-b854-4691822b8c8a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-5bqdc"
Oct 30 18:33:45.201399 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.201378    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-4b95q"
Oct 30 18:33:45.202390 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.202369    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9j2qw\" (UniqueName: \"kubernetes.io/projected/81de5dce-c40f-4bd5-878c-5d75a0487066-kube-api-access-9j2qw\") pod \"network-check-source-6f78d59f99-4969l\" (UID: \"81de5dce-c40f-4bd5-878c-5d75a0487066\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-4969l"
Oct 30 18:33:45.204901 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.204860    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cftdb\" (UniqueName: \"kubernetes.io/projected/da28351c-1c8b-40be-85a0-3aad427e3a17-kube-api-access-cftdb\") pod \"router-default-7dfcf9fd6c-vh769\" (UID: \"da28351c-1c8b-40be-85a0-3aad427e3a17\") " pod="openshift-ingress/router-default-7dfcf9fd6c-vh769"
Oct 30 18:33:45.205597 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.205479    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rw8g9\" (UniqueName: \"kubernetes.io/projected/36a04851-81db-40cd-97ef-8e2e00a08604-kube-api-access-rw8g9\") pod \"service-ca-operator-544d749c8c-kr5qq\" (UID: \"36a04851-81db-40cd-97ef-8e2e00a08604\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-kr5qq"
Oct 30 18:33:45.206413 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.206159    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8789ec5b-226e-4551-bf68-3ac585006266-bound-sa-token\") pod \"image-registry-74757bccd5-qmrkp\" (UID: \"8789ec5b-226e-4551-bf68-3ac585006266\") " pod="openshift-image-registry/image-registry-74757bccd5-qmrkp"
Oct 30 18:33:45.208210 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.208164    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vxw47\" (UniqueName: \"kubernetes.io/projected/9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351-kube-api-access-vxw47\") pod \"image-registry-64bf5b6b76-dpvvs\" (UID: \"9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351\") " pod="openshift-image-registry/image-registry-64bf5b6b76-dpvvs"
Oct 30 18:33:45.213603 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.213580    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-5bqdc"
Oct 30 18:33:45.213798 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.213780    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/97b19aa3-42be-4def-8d7a-8e3516852e79-config-volume\") pod \"dns-default-r2nx2\" (UID: \"97b19aa3-42be-4def-8d7a-8e3516852e79\") " pod="openshift-dns/dns-default-r2nx2"
Oct 30 18:33:45.213859 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.213828    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/97b19aa3-42be-4def-8d7a-8e3516852e79-metrics-tls\") pod \"dns-default-r2nx2\" (UID: \"97b19aa3-42be-4def-8d7a-8e3516852e79\") " pod="openshift-dns/dns-default-r2nx2"
Oct 30 18:33:45.213927 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.213854    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-24cp9\" (UniqueName: \"kubernetes.io/projected/e6b2a584-ae78-4e80-b595-9b1a5897d556-kube-api-access-24cp9\") pod \"ingress-canary-g9fhp\" (UID: \"e6b2a584-ae78-4e80-b595-9b1a5897d556\") " pod="openshift-ingress-canary/ingress-canary-g9fhp"
Oct 30 18:33:45.213976 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.213931    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e6b2a584-ae78-4e80-b595-9b1a5897d556-cert\") pod \"ingress-canary-g9fhp\" (UID: \"e6b2a584-ae78-4e80-b595-9b1a5897d556\") " pod="openshift-ingress-canary/ingress-canary-g9fhp"
Oct 30 18:33:45.213976 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:45.213953    2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 30 18:33:45.213976 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.213968    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tc2fp\" (UniqueName: \"kubernetes.io/projected/97b19aa3-42be-4def-8d7a-8e3516852e79-kube-api-access-tc2fp\") pod \"dns-default-r2nx2\" (UID: \"97b19aa3-42be-4def-8d7a-8e3516852e79\") " pod="openshift-dns/dns-default-r2nx2"
Oct 30 18:33:45.214104 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:45.214013    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/97b19aa3-42be-4def-8d7a-8e3516852e79-metrics-tls podName:97b19aa3-42be-4def-8d7a-8e3516852e79 nodeName:}" failed. No retries permitted until 2026-10-30 18:33:45.713994789 +0000 UTC m=+32.205429230 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/97b19aa3-42be-4def-8d7a-8e3516852e79-metrics-tls") pod "dns-default-r2nx2" (UID: "97b19aa3-42be-4def-8d7a-8e3516852e79") : secret "dns-default-metrics-tls" not found
Oct 30 18:33:45.214104 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.214067    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/97b19aa3-42be-4def-8d7a-8e3516852e79-tmp-dir\") pod \"dns-default-r2nx2\" (UID: \"97b19aa3-42be-4def-8d7a-8e3516852e79\") " pod="openshift-dns/dns-default-r2nx2"
Oct 30 18:33:45.214425 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.214402    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/97b19aa3-42be-4def-8d7a-8e3516852e79-tmp-dir\") pod \"dns-default-r2nx2\" (UID: \"97b19aa3-42be-4def-8d7a-8e3516852e79\") " pod="openshift-dns/dns-default-r2nx2"
Oct 30 18:33:45.214425 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.214415    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/97b19aa3-42be-4def-8d7a-8e3516852e79-config-volume\") pod \"dns-default-r2nx2\" (UID: \"97b19aa3-42be-4def-8d7a-8e3516852e79\") " pod="openshift-dns/dns-default-r2nx2"
Oct 30 18:33:45.216813 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.216788    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e6b2a584-ae78-4e80-b595-9b1a5897d556-cert\") pod \"ingress-canary-g9fhp\" (UID: \"e6b2a584-ae78-4e80-b595-9b1a5897d556\") " pod="openshift-ingress-canary/ingress-canary-g9fhp"
Oct 30 18:33:45.217928 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.217909    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-7c4986d449-xvgqc"
Oct 30 18:33:45.223779 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.223761    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-2grbp"
Oct 30 18:33:45.230256 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.229462    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7cd56b8dbc-7x4rb"
Oct 30 18:33:45.237383 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.237362    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-kr5qq"
Oct 30 18:33:45.277165 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.276925    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-jwwq2"
Oct 30 18:33:45.288333 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.286861    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-24cp9\" (UniqueName: \"kubernetes.io/projected/e6b2a584-ae78-4e80-b595-9b1a5897d556-kube-api-access-24cp9\") pod \"ingress-canary-g9fhp\" (UID: \"e6b2a584-ae78-4e80-b595-9b1a5897d556\") " pod="openshift-ingress-canary/ingress-canary-g9fhp"
Oct 30 18:33:45.292991 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.292967    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-4969l"
Oct 30 18:33:45.294713 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.294683    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tc2fp\" (UniqueName: \"kubernetes.io/projected/97b19aa3-42be-4def-8d7a-8e3516852e79-kube-api-access-tc2fp\") pod \"dns-default-r2nx2\" (UID: \"97b19aa3-42be-4def-8d7a-8e3516852e79\") " pod="openshift-dns/dns-default-r2nx2"
Oct 30 18:33:45.306046 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.305724    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-g9fhp"
Oct 30 18:33:45.531883 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.531820    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-f9ldt"]
Oct 30 18:33:45.534639 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:45.534615    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf9e5074_3bb1_4817_8855_b045cee61004.slice/crio-1d584626e578f70bd2bd96914c22b5060fe64a2954330b204e9b4c0683253984 WatchSource:0}: Error finding container 1d584626e578f70bd2bd96914c22b5060fe64a2954330b204e9b4c0683253984: Status 404 returned error can't find the container with id 1d584626e578f70bd2bd96914c22b5060fe64a2954330b204e9b4c0683253984
Oct 30 18:33:45.587801 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.587770    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-4b95q"]
Oct 30 18:33:45.591709 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:45.591682    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddedb8b54_d0ce_434c_9461_db90a189c121.slice/crio-ecf8942e95e312a77dfd631bc979e0c4fb823eb197c2536e76ed89c513f0cd6e WatchSource:0}: Error finding container ecf8942e95e312a77dfd631bc979e0c4fb823eb197c2536e76ed89c513f0cd6e: Status 404 returned error can't find the container with id ecf8942e95e312a77dfd631bc979e0c4fb823eb197c2536e76ed89c513f0cd6e
Oct 30 18:33:45.620184 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.617794    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/71fce796-0032-481b-80b3-66fbb731452c-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-r8rgt\" (UID: \"71fce796-0032-481b-80b3-66fbb731452c\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-r8rgt"
Oct 30 18:33:45.620184 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.617898    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/68195e19-b1ff-4148-9fae-3545f2997ff5-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-gmswj\" (UID: \"68195e19-b1ff-4148-9fae-3545f2997ff5\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-gmswj"
Oct 30 18:33:45.620184 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.617960    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8789ec5b-226e-4551-bf68-3ac585006266-registry-tls\") pod \"image-registry-74757bccd5-qmrkp\" (UID: \"8789ec5b-226e-4551-bf68-3ac585006266\") " pod="openshift-image-registry/image-registry-74757bccd5-qmrkp"
Oct 30 18:33:45.620184 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.618013    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/da28351c-1c8b-40be-85a0-3aad427e3a17-service-ca-bundle\") pod \"router-default-7dfcf9fd6c-vh769\" (UID: \"da28351c-1c8b-40be-85a0-3aad427e3a17\") " pod="openshift-ingress/router-default-7dfcf9fd6c-vh769"
Oct 30 18:33:45.620184 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:45.618205    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/da28351c-1c8b-40be-85a0-3aad427e3a17-service-ca-bundle podName:da28351c-1c8b-40be-85a0-3aad427e3a17 nodeName:}" failed. No retries permitted until 2026-10-30 18:33:46.618187814 +0000 UTC m=+33.109622276 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/da28351c-1c8b-40be-85a0-3aad427e3a17-service-ca-bundle") pod "router-default-7dfcf9fd6c-vh769" (UID: "da28351c-1c8b-40be-85a0-3aad427e3a17") : configmap references non-existent config key: service-ca.crt
Oct 30 18:33:45.620184 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:45.618304    2574 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Oct 30 18:33:45.620184 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:45.618346    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/71fce796-0032-481b-80b3-66fbb731452c-networking-console-plugin-cert podName:71fce796-0032-481b-80b3-66fbb731452c nodeName:}" failed. No retries permitted until 2026-10-30 18:33:46.618332718 +0000 UTC m=+33.109767181 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/71fce796-0032-481b-80b3-66fbb731452c-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-r8rgt" (UID: "71fce796-0032-481b-80b3-66fbb731452c") : secret "networking-console-plugin-cert" not found
Oct 30 18:33:45.620837 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:45.620813    2574 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 30 18:33:45.620837 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:45.620839    2574 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-74757bccd5-qmrkp: secret "image-registry-tls" not found
Oct 30 18:33:45.620983 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:45.620895    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8789ec5b-226e-4551-bf68-3ac585006266-registry-tls podName:8789ec5b-226e-4551-bf68-3ac585006266 nodeName:}" failed. No retries permitted until 2026-10-30 18:33:46.620878389 +0000 UTC m=+33.112312844 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8789ec5b-226e-4551-bf68-3ac585006266-registry-tls") pod "image-registry-74757bccd5-qmrkp" (UID: "8789ec5b-226e-4551-bf68-3ac585006266") : secret "image-registry-tls" not found
Oct 30 18:33:45.620983 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:45.620926    2574 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Oct 30 18:33:45.620983 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:45.620973    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68195e19-b1ff-4148-9fae-3545f2997ff5-cluster-monitoring-operator-tls podName:68195e19-b1ff-4148-9fae-3545f2997ff5 nodeName:}" failed. No retries permitted until 2026-10-30 18:33:46.620961604 +0000 UTC m=+33.112396060 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/68195e19-b1ff-4148-9fae-3545f2997ff5-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-gmswj" (UID: "68195e19-b1ff-4148-9fae-3545f2997ff5") : secret "cluster-monitoring-operator-tls" not found
Oct 30 18:33:45.637922 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.637901    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-2grbp"]
Oct 30 18:33:45.644571 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.644303    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-5bqdc"]
Oct 30 18:33:45.644571 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.644408    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7cd56b8dbc-7x4rb"]
Oct 30 18:33:45.645721 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.645693    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-7c4986d449-xvgqc"]
Oct 30 18:33:45.681348 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.681325    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-544d749c8c-kr5qq"]
Oct 30 18:33:45.716741 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.716712    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-jwwq2"]
Oct 30 18:33:45.720357 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.720335    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-6f78d59f99-4969l"]
Oct 30 18:33:45.720634 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.720614    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/da28351c-1c8b-40be-85a0-3aad427e3a17-metrics-certs\") pod \"router-default-7dfcf9fd6c-vh769\" (UID: \"da28351c-1c8b-40be-85a0-3aad427e3a17\") " pod="openshift-ingress/router-default-7dfcf9fd6c-vh769"
Oct 30 18:33:45.720687 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.720671    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/97b19aa3-42be-4def-8d7a-8e3516852e79-metrics-tls\") pod \"dns-default-r2nx2\" (UID: \"97b19aa3-42be-4def-8d7a-8e3516852e79\") " pod="openshift-dns/dns-default-r2nx2"
Oct 30 18:33:45.720792 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:45.720774    2574 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Oct 30 18:33:45.720843 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.720810    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/15d485f9-5a4b-4f4b-9e59-a6eb062b4c5c-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-pm58m\" (UID: \"15d485f9-5a4b-4f4b-9e59-a6eb062b4c5c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-pm58m"
Oct 30 18:33:45.720843 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:45.720818    2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 30 18:33:45.720843 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:45.720837    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/da28351c-1c8b-40be-85a0-3aad427e3a17-metrics-certs podName:da28351c-1c8b-40be-85a0-3aad427e3a17 nodeName:}" failed. No retries permitted until 2026-10-30 18:33:46.720817726 +0000 UTC m=+33.212252189 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/da28351c-1c8b-40be-85a0-3aad427e3a17-metrics-certs") pod "router-default-7dfcf9fd6c-vh769" (UID: "da28351c-1c8b-40be-85a0-3aad427e3a17") : secret "router-metrics-certs-default" not found
Oct 30 18:33:45.720943 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:45.720859    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/97b19aa3-42be-4def-8d7a-8e3516852e79-metrics-tls podName:97b19aa3-42be-4def-8d7a-8e3516852e79 nodeName:}" failed. No retries permitted until 2026-10-30 18:33:46.720847864 +0000 UTC m=+33.212282305 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/97b19aa3-42be-4def-8d7a-8e3516852e79-metrics-tls") pod "dns-default-r2nx2" (UID: "97b19aa3-42be-4def-8d7a-8e3516852e79") : secret "dns-default-metrics-tls" not found
Oct 30 18:33:45.720943 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:45.720887    2574 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Oct 30 18:33:45.720943 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.720899    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351-registry-tls\") pod \"image-registry-64bf5b6b76-dpvvs\" (UID: \"9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351\") " pod="openshift-image-registry/image-registry-64bf5b6b76-dpvvs"
Oct 30 18:33:45.720943 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:45.720930    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/15d485f9-5a4b-4f4b-9e59-a6eb062b4c5c-samples-operator-tls podName:15d485f9-5a4b-4f4b-9e59-a6eb062b4c5c nodeName:}" failed. No retries permitted until 2026-10-30 18:33:46.720916965 +0000 UTC m=+33.212351419 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/15d485f9-5a4b-4f4b-9e59-a6eb062b4c5c-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-pm58m" (UID: "15d485f9-5a4b-4f4b-9e59-a6eb062b4c5c") : secret "samples-operator-tls" not found
Oct 30 18:33:45.721079 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:45.720970    2574 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 30 18:33:45.721079 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:45.720978    2574 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-64bf5b6b76-dpvvs: secret "image-registry-tls" not found
Oct 30 18:33:45.721079 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:45.721013    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351-registry-tls podName:9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351 nodeName:}" failed. No retries permitted until 2026-10-30 18:33:46.721001438 +0000 UTC m=+33.212435879 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351-registry-tls") pod "image-registry-64bf5b6b76-dpvvs" (UID: "9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351") : secret "image-registry-tls" not found
Oct 30 18:33:45.722576 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:45.722552    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4ac17da_6b5b_4441_9b4e_ebe696e06507.slice/crio-41566f398fb6e7ad1be747e3b375979733d9965f00edb0dd1f8a0328234a824c WatchSource:0}: Error finding container 41566f398fb6e7ad1be747e3b375979733d9965f00edb0dd1f8a0328234a824c: Status 404 returned error can't find the container with id 41566f398fb6e7ad1be747e3b375979733d9965f00edb0dd1f8a0328234a824c
Oct 30 18:33:45.723335 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:45.723316    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81de5dce_c40f_4bd5_878c_5d75a0487066.slice/crio-02e1780cae1ffe8664994cc74406d074948af8b6ea7d829904b28cc97b45939f WatchSource:0}: Error finding container 02e1780cae1ffe8664994cc74406d074948af8b6ea7d829904b28cc97b45939f: Status 404 returned error can't find the container with id 02e1780cae1ffe8664994cc74406d074948af8b6ea7d829904b28cc97b45939f
Oct 30 18:33:45.740026 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.740001    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-g9fhp"]
Oct 30 18:33:45.742660 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:45.742640    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6b2a584_ae78_4e80_b595_9b1a5897d556.slice/crio-110c3deb893a6eff9717f921ccf7dc3f3e56cf7408a56a85d1d6b3e8a95633a4 WatchSource:0}: Error finding container 110c3deb893a6eff9717f921ccf7dc3f3e56cf7408a56a85d1d6b3e8a95633a4: Status 404 returned error can't find the container with id 110c3deb893a6eff9717f921ccf7dc3f3e56cf7408a56a85d1d6b3e8a95633a4
Oct 30 18:33:45.939560 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.939531    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g2svp"
Oct 30 18:33:45.939667 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.939537    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v5phn"
Oct 30 18:33:45.939769 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.939741    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8xskh"
Oct 30 18:33:45.991901 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.991875    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Oct 30 18:33:45.992309 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.992270    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-727z7\""
Oct 30 18:33:45.992309 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.992270    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-8wmd6\""
Oct 30 18:33:45.992414 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.992335    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-wkpbb\""
Oct 30 18:33:45.992414 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:45.992344    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Oct 30 18:33:46.126037 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:46.126000    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-4969l" event={"ID":"81de5dce-c40f-4bd5-878c-5d75a0487066","Type":"ContainerStarted","Data":"02e1780cae1ffe8664994cc74406d074948af8b6ea7d829904b28cc97b45939f"}
Oct 30 18:33:46.127581 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:46.127559    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-4b95q" event={"ID":"dedb8b54-d0ce-434c-9461-db90a189c121","Type":"ContainerStarted","Data":"794703a64f4147d76247e1327de469243dce1475f8108132ca66e82747a0eb34"}
Oct 30 18:33:46.127691 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:46.127600    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-4b95q" event={"ID":"dedb8b54-d0ce-434c-9461-db90a189c121","Type":"ContainerStarted","Data":"ecf8942e95e312a77dfd631bc979e0c4fb823eb197c2536e76ed89c513f0cd6e"}
Oct 30 18:33:46.128771 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:46.128744    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-g9fhp" event={"ID":"e6b2a584-ae78-4e80-b595-9b1a5897d556","Type":"ContainerStarted","Data":"110c3deb893a6eff9717f921ccf7dc3f3e56cf7408a56a85d1d6b3e8a95633a4"}
Oct 30 18:33:46.129975 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:46.129956    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-5bqdc" event={"ID":"09c13759-50d4-40e6-b854-4691822b8c8a","Type":"ContainerStarted","Data":"93eecdc0633397bbec713dfdc4aff8aeb6399c863290c63bbd8a0da3fb70b0fc"}
Oct 30 18:33:46.131107 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:46.131087    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-jwwq2" event={"ID":"f4ac17da-6b5b-4441-9b4e-ebe696e06507","Type":"ContainerStarted","Data":"41566f398fb6e7ad1be747e3b375979733d9965f00edb0dd1f8a0328234a824c"}
Oct 30 18:33:46.132218 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:46.132193    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-7x4rb" event={"ID":"382e8b31-a7fa-43f3-9f15-4c61627b0c04","Type":"ContainerStarted","Data":"3dacffc196bf6205ffae4eaaf96fc7405b8753485def2dab566ce25b8106f9ae"}
Oct 30 18:33:46.133278 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:46.133260    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-kr5qq" event={"ID":"36a04851-81db-40cd-97ef-8e2e00a08604","Type":"ContainerStarted","Data":"b3f8d0948c4c12d4822539070451f179a0b9a930adb6cdfd15c3c8b13a7f4395"}
Oct 30 18:33:46.135133 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:46.135108    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-2grbp" event={"ID":"e77c4479-d6e5-48a8-ac82-102a9f1a2c4a","Type":"ContainerStarted","Data":"fefb7f419b1eeffeb55276a9a199a6006107d0455ad88a9b4d4824c99e4f571d"}
Oct 30 18:33:46.135229 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:46.135150    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-2grbp" event={"ID":"e77c4479-d6e5-48a8-ac82-102a9f1a2c4a","Type":"ContainerStarted","Data":"1fb8715703bb95f8a132c554ff1169dc975c64de24b0bc642f7bf41a12efe875"}
Oct 30 18:33:46.137027 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:46.136991    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-7c4986d449-xvgqc" event={"ID":"7c09088b-3a07-426d-8804-f0f5bac4d27c","Type":"ContainerStarted","Data":"264171f78a701c6a4af300c38636f113df3b276804fb65397c6d59c290a3b89f"}
Oct 30 18:33:46.138702 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:46.138679    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-f9ldt" event={"ID":"df9e5074-3bb1-4817-8855-b045cee61004","Type":"ContainerStarted","Data":"04891d8ea1b3ecd88a13b76889494cf91196c3b89b7834e7d584fe806606c78f"}
Oct 30 18:33:46.138779 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:46.138721    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-f9ldt" event={"ID":"df9e5074-3bb1-4817-8855-b045cee61004","Type":"ContainerStarted","Data":"1d584626e578f70bd2bd96914c22b5060fe64a2954330b204e9b4c0683253984"}
Oct 30 18:33:46.628706 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:46.628676    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/68195e19-b1ff-4148-9fae-3545f2997ff5-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-gmswj\" (UID: \"68195e19-b1ff-4148-9fae-3545f2997ff5\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-gmswj"
Oct 30 18:33:46.628882 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:46.628748    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8789ec5b-226e-4551-bf68-3ac585006266-registry-tls\") pod \"image-registry-74757bccd5-qmrkp\" (UID: \"8789ec5b-226e-4551-bf68-3ac585006266\") " pod="openshift-image-registry/image-registry-74757bccd5-qmrkp"
Oct 30 18:33:46.628882 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:46.628799    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/da28351c-1c8b-40be-85a0-3aad427e3a17-service-ca-bundle\") pod \"router-default-7dfcf9fd6c-vh769\" (UID: \"da28351c-1c8b-40be-85a0-3aad427e3a17\") " pod="openshift-ingress/router-default-7dfcf9fd6c-vh769"
Oct 30 18:33:46.628882 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:46.628830    2574 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Oct 30 18:33:46.628882 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:46.628864    2574 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 30 18:33:46.628882 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:46.628869    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/71fce796-0032-481b-80b3-66fbb731452c-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-r8rgt\" (UID: \"71fce796-0032-481b-80b3-66fbb731452c\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-r8rgt"
Oct 30 18:33:46.628882 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:46.628878    2574 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-74757bccd5-qmrkp: secret "image-registry-tls" not found
Oct 30 18:33:46.629219 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:46.628904    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68195e19-b1ff-4148-9fae-3545f2997ff5-cluster-monitoring-operator-tls podName:68195e19-b1ff-4148-9fae-3545f2997ff5 nodeName:}" failed. No retries permitted until 2026-10-30 18:33:48.628883777 +0000 UTC m=+35.120318218 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/68195e19-b1ff-4148-9fae-3545f2997ff5-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-gmswj" (UID: "68195e19-b1ff-4148-9fae-3545f2997ff5") : secret "cluster-monitoring-operator-tls" not found
Oct 30 18:33:46.629219 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:46.628926    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8789ec5b-226e-4551-bf68-3ac585006266-registry-tls podName:8789ec5b-226e-4551-bf68-3ac585006266 nodeName:}" failed. No retries permitted until 2026-10-30 18:33:48.628917445 +0000 UTC m=+35.120351886 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8789ec5b-226e-4551-bf68-3ac585006266-registry-tls") pod "image-registry-74757bccd5-qmrkp" (UID: "8789ec5b-226e-4551-bf68-3ac585006266") : secret "image-registry-tls" not found
Oct 30 18:33:46.629219 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:46.628961    2574 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Oct 30 18:33:46.629219 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:46.629008    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/da28351c-1c8b-40be-85a0-3aad427e3a17-service-ca-bundle podName:da28351c-1c8b-40be-85a0-3aad427e3a17 nodeName:}" failed. No retries permitted until 2026-10-30 18:33:48.62897736 +0000 UTC m=+35.120411818 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/da28351c-1c8b-40be-85a0-3aad427e3a17-service-ca-bundle") pod "router-default-7dfcf9fd6c-vh769" (UID: "da28351c-1c8b-40be-85a0-3aad427e3a17") : configmap references non-existent config key: service-ca.crt
Oct 30 18:33:46.629219 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:46.629034    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/71fce796-0032-481b-80b3-66fbb731452c-networking-console-plugin-cert podName:71fce796-0032-481b-80b3-66fbb731452c nodeName:}" failed. No retries permitted until 2026-10-30 18:33:48.629023742 +0000 UTC m=+35.120458204 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/71fce796-0032-481b-80b3-66fbb731452c-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-r8rgt" (UID: "71fce796-0032-481b-80b3-66fbb731452c") : secret "networking-console-plugin-cert" not found
Oct 30 18:33:46.729658 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:46.729618    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/da28351c-1c8b-40be-85a0-3aad427e3a17-metrics-certs\") pod \"router-default-7dfcf9fd6c-vh769\" (UID: \"da28351c-1c8b-40be-85a0-3aad427e3a17\") " pod="openshift-ingress/router-default-7dfcf9fd6c-vh769"
Oct 30 18:33:46.729826 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:46.729689    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/97b19aa3-42be-4def-8d7a-8e3516852e79-metrics-tls\") pod \"dns-default-r2nx2\" (UID: \"97b19aa3-42be-4def-8d7a-8e3516852e79\") " pod="openshift-dns/dns-default-r2nx2"
Oct 30 18:33:46.729897 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:46.729826    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/15d485f9-5a4b-4f4b-9e59-a6eb062b4c5c-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-pm58m\" (UID: \"15d485f9-5a4b-4f4b-9e59-a6eb062b4c5c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-pm58m"
Oct 30 18:33:46.729897 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:46.729865    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351-registry-tls\") pod \"image-registry-64bf5b6b76-dpvvs\" (UID: \"9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351\") " pod="openshift-image-registry/image-registry-64bf5b6b76-dpvvs"
Oct 30 18:33:46.730030 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:46.730014    2574 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 30 18:33:46.730099 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:46.730030    2574 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-64bf5b6b76-dpvvs: secret "image-registry-tls" not found
Oct 30 18:33:46.730099 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:46.730096    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351-registry-tls podName:9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351 nodeName:}" failed. No retries permitted until 2026-10-30 18:33:48.73007832 +0000 UTC m=+35.221512761 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351-registry-tls") pod "image-registry-64bf5b6b76-dpvvs" (UID: "9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351") : secret "image-registry-tls" not found
Oct 30 18:33:46.730202 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:46.730170    2574 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Oct 30 18:33:46.730267 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:46.730204    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/da28351c-1c8b-40be-85a0-3aad427e3a17-metrics-certs podName:da28351c-1c8b-40be-85a0-3aad427e3a17 nodeName:}" failed. No retries permitted until 2026-10-30 18:33:48.73019419 +0000 UTC m=+35.221628644 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/da28351c-1c8b-40be-85a0-3aad427e3a17-metrics-certs") pod "router-default-7dfcf9fd6c-vh769" (UID: "da28351c-1c8b-40be-85a0-3aad427e3a17") : secret "router-metrics-certs-default" not found
Oct 30 18:33:46.730267 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:46.730258    2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 30 18:33:46.730409 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:46.730369    2574 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Oct 30 18:33:46.730768 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:46.730752    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/97b19aa3-42be-4def-8d7a-8e3516852e79-metrics-tls podName:97b19aa3-42be-4def-8d7a-8e3516852e79 nodeName:}" failed. No retries permitted until 2026-10-30 18:33:48.730277252 +0000 UTC m=+35.221711707 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/97b19aa3-42be-4def-8d7a-8e3516852e79-metrics-tls") pod "dns-default-r2nx2" (UID: "97b19aa3-42be-4def-8d7a-8e3516852e79") : secret "dns-default-metrics-tls" not found
Oct 30 18:33:46.730839 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:46.730791    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/15d485f9-5a4b-4f4b-9e59-a6eb062b4c5c-samples-operator-tls podName:15d485f9-5a4b-4f4b-9e59-a6eb062b4c5c nodeName:}" failed. No retries permitted until 2026-10-30 18:33:48.730772913 +0000 UTC m=+35.222207368 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/15d485f9-5a4b-4f4b-9e59-a6eb062b4c5c-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-pm58m" (UID: "15d485f9-5a4b-4f4b-9e59-a6eb062b4c5c") : secret "samples-operator-tls" not found
Oct 30 18:33:47.072666 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:47.072548    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kas-connection-checker-685895c7c9-4b95q" podStartSLOduration=304.072530726 podStartE2EDuration="5m4.072530726s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:28:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:33:47.071559948 +0000 UTC m=+33.562994410" watchObservedRunningTime="2026-10-30 18:33:47.072530726 +0000 UTC m=+33.563965188"
Oct 30 18:33:47.306646 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:47.306587    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kas-connection-checker-685895c7c9-f9ldt" podStartSLOduration=304.306566913 podStartE2EDuration="5m4.306566913s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:28:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:33:47.191988478 +0000 UTC m=+33.683422941" watchObservedRunningTime="2026-10-30 18:33:47.306566913 +0000 UTC m=+33.798001374"
Oct 30 18:33:47.738476 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:47.738443    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a0823e8f-bbdd-477b-95ac-dad05d23cefa-metrics-certs\") pod \"network-metrics-daemon-v5phn\" (UID: \"a0823e8f-bbdd-477b-95ac-dad05d23cefa\") " pod="openshift-multus/network-metrics-daemon-v5phn"
Oct 30 18:33:47.738764 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:47.738490    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/46d9ac39-ee45-4e43-bda7-3931c3e599cf-original-pull-secret\") pod \"global-pull-secret-syncer-g2svp\" (UID: \"46d9ac39-ee45-4e43-bda7-3931c3e599cf\") " pod="kube-system/global-pull-secret-syncer-g2svp"
Oct 30 18:33:47.739038 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:47.739017    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Oct 30 18:33:47.739138 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:47.739089    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a0823e8f-bbdd-477b-95ac-dad05d23cefa-metrics-certs podName:a0823e8f-bbdd-477b-95ac-dad05d23cefa nodeName:}" failed. No retries permitted until 2026-10-30 18:34:19.739070525 +0000 UTC m=+66.230504985 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a0823e8f-bbdd-477b-95ac-dad05d23cefa-metrics-certs") pod "network-metrics-daemon-v5phn" (UID: "a0823e8f-bbdd-477b-95ac-dad05d23cefa") : secret "metrics-daemon-secret" not found
Oct 30 18:33:47.743942 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:47.743890    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/46d9ac39-ee45-4e43-bda7-3931c3e599cf-original-pull-secret\") pod \"global-pull-secret-syncer-g2svp\" (UID: \"46d9ac39-ee45-4e43-bda7-3931c3e599cf\") " pod="kube-system/global-pull-secret-syncer-g2svp"
Oct 30 18:33:47.751761 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:47.751739    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-g2svp"
Oct 30 18:33:47.839772 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:47.839740    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c45qr\" (UniqueName: \"kubernetes.io/projected/613020c0-f3e4-4650-896f-748a40f0ab8b-kube-api-access-c45qr\") pod \"network-check-target-8xskh\" (UID: \"613020c0-f3e4-4650-896f-748a40f0ab8b\") " pod="openshift-network-diagnostics/network-check-target-8xskh"
Oct 30 18:33:47.846826 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:47.846784    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c45qr\" (UniqueName: \"kubernetes.io/projected/613020c0-f3e4-4650-896f-748a40f0ab8b-kube-api-access-c45qr\") pod \"network-check-target-8xskh\" (UID: \"613020c0-f3e4-4650-896f-748a40f0ab8b\") " pod="openshift-network-diagnostics/network-check-target-8xskh"
Oct 30 18:33:48.063447 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:48.063110    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8xskh"
Oct 30 18:33:48.150545 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:48.150487    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kas-connection-checker-685895c7c9-2grbp" podStartSLOduration=305.150467319 podStartE2EDuration="5m5.150467319s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:28:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:33:47.308405717 +0000 UTC m=+33.799840181" watchObservedRunningTime="2026-10-30 18:33:48.150467319 +0000 UTC m=+34.641901809"
Oct 30 18:33:48.151470 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:48.151448    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-g2svp"]
Oct 30 18:33:48.487830 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:48.487787    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-8xskh"]
Oct 30 18:33:48.648811 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:48.648771    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod613020c0_f3e4_4650_896f_748a40f0ab8b.slice/crio-41306bd73702bf41bacd24f69e9da18eaa60d7a3d5c5782b2c004af7846ac805 WatchSource:0}: Error finding container 41306bd73702bf41bacd24f69e9da18eaa60d7a3d5c5782b2c004af7846ac805: Status 404 returned error can't find the container with id 41306bd73702bf41bacd24f69e9da18eaa60d7a3d5c5782b2c004af7846ac805
Oct 30 18:33:48.650766 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:48.650739    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/68195e19-b1ff-4148-9fae-3545f2997ff5-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-gmswj\" (UID: \"68195e19-b1ff-4148-9fae-3545f2997ff5\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-gmswj"
Oct 30 18:33:48.650873 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:48.650813    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8789ec5b-226e-4551-bf68-3ac585006266-registry-tls\") pod \"image-registry-74757bccd5-qmrkp\" (UID: \"8789ec5b-226e-4551-bf68-3ac585006266\") " pod="openshift-image-registry/image-registry-74757bccd5-qmrkp"
Oct 30 18:33:48.650873 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:48.650862    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/da28351c-1c8b-40be-85a0-3aad427e3a17-service-ca-bundle\") pod \"router-default-7dfcf9fd6c-vh769\" (UID: \"da28351c-1c8b-40be-85a0-3aad427e3a17\") " pod="openshift-ingress/router-default-7dfcf9fd6c-vh769"
Oct 30 18:33:48.651055 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:48.650889    2574 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Oct 30 18:33:48.651055 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:48.650920    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/71fce796-0032-481b-80b3-66fbb731452c-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-r8rgt\" (UID: \"71fce796-0032-481b-80b3-66fbb731452c\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-r8rgt"
Oct 30 18:33:48.651055 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:48.650943    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68195e19-b1ff-4148-9fae-3545f2997ff5-cluster-monitoring-operator-tls podName:68195e19-b1ff-4148-9fae-3545f2997ff5 nodeName:}" failed. No retries permitted until 2026-10-30 18:33:52.650926391 +0000 UTC m=+39.142360832 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/68195e19-b1ff-4148-9fae-3545f2997ff5-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-gmswj" (UID: "68195e19-b1ff-4148-9fae-3545f2997ff5") : secret "cluster-monitoring-operator-tls" not found
Oct 30 18:33:48.651055 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:48.650977    2574 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Oct 30 18:33:48.651055 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:48.651008    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/71fce796-0032-481b-80b3-66fbb731452c-networking-console-plugin-cert podName:71fce796-0032-481b-80b3-66fbb731452c nodeName:}" failed. No retries permitted until 2026-10-30 18:33:52.650998845 +0000 UTC m=+39.142433305 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/71fce796-0032-481b-80b3-66fbb731452c-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-r8rgt" (UID: "71fce796-0032-481b-80b3-66fbb731452c") : secret "networking-console-plugin-cert" not found
Oct 30 18:33:48.651429 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:48.651061    2574 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 30 18:33:48.651429 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:48.651071    2574 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-74757bccd5-qmrkp: secret "image-registry-tls" not found
Oct 30 18:33:48.651429 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:48.651100    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8789ec5b-226e-4551-bf68-3ac585006266-registry-tls podName:8789ec5b-226e-4551-bf68-3ac585006266 nodeName:}" failed. No retries permitted until 2026-10-30 18:33:52.651090735 +0000 UTC m=+39.142525190 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8789ec5b-226e-4551-bf68-3ac585006266-registry-tls") pod "image-registry-74757bccd5-qmrkp" (UID: "8789ec5b-226e-4551-bf68-3ac585006266") : secret "image-registry-tls" not found
Oct 30 18:33:48.651429 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:48.651159    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/da28351c-1c8b-40be-85a0-3aad427e3a17-service-ca-bundle podName:da28351c-1c8b-40be-85a0-3aad427e3a17 nodeName:}" failed. No retries permitted until 2026-10-30 18:33:52.65114931 +0000 UTC m=+39.142583764 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/da28351c-1c8b-40be-85a0-3aad427e3a17-service-ca-bundle") pod "router-default-7dfcf9fd6c-vh769" (UID: "da28351c-1c8b-40be-85a0-3aad427e3a17") : configmap references non-existent config key: service-ca.crt
Oct 30 18:33:48.751882 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:48.751848    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/15d485f9-5a4b-4f4b-9e59-a6eb062b4c5c-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-pm58m\" (UID: \"15d485f9-5a4b-4f4b-9e59-a6eb062b4c5c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-pm58m"
Oct 30 18:33:48.752024 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:48.751909    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351-registry-tls\") pod \"image-registry-64bf5b6b76-dpvvs\" (UID: \"9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351\") " pod="openshift-image-registry/image-registry-64bf5b6b76-dpvvs"
Oct 30 18:33:48.752024 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:48.751966    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/da28351c-1c8b-40be-85a0-3aad427e3a17-metrics-certs\") pod \"router-default-7dfcf9fd6c-vh769\" (UID: \"da28351c-1c8b-40be-85a0-3aad427e3a17\") " pod="openshift-ingress/router-default-7dfcf9fd6c-vh769"
Oct 30 18:33:48.752122 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:48.752111    2574 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Oct 30 18:33:48.752209 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:48.752197    2574 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 30 18:33:48.752298 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:48.752211    2574 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-64bf5b6b76-dpvvs: secret "image-registry-tls" not found
Oct 30 18:33:48.752361 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:48.752311    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/97b19aa3-42be-4def-8d7a-8e3516852e79-metrics-tls\") pod \"dns-default-r2nx2\" (UID: \"97b19aa3-42be-4def-8d7a-8e3516852e79\") " pod="openshift-dns/dns-default-r2nx2"
Oct 30 18:33:48.752361 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:48.752357    2574 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Oct 30 18:33:48.752456 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:48.752406    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/da28351c-1c8b-40be-85a0-3aad427e3a17-metrics-certs podName:da28351c-1c8b-40be-85a0-3aad427e3a17 nodeName:}" failed. No retries permitted until 2026-10-30 18:33:52.752381035 +0000 UTC m=+39.243815476 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/da28351c-1c8b-40be-85a0-3aad427e3a17-metrics-certs") pod "router-default-7dfcf9fd6c-vh769" (UID: "da28351c-1c8b-40be-85a0-3aad427e3a17") : secret "router-metrics-certs-default" not found
Oct 30 18:33:48.752456 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:48.752428    2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 30 18:33:48.752456 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:48.752430    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/15d485f9-5a4b-4f4b-9e59-a6eb062b4c5c-samples-operator-tls podName:15d485f9-5a4b-4f4b-9e59-a6eb062b4c5c nodeName:}" failed. No retries permitted until 2026-10-30 18:33:52.752420654 +0000 UTC m=+39.243855095 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/15d485f9-5a4b-4f4b-9e59-a6eb062b4c5c-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-pm58m" (UID: "15d485f9-5a4b-4f4b-9e59-a6eb062b4c5c") : secret "samples-operator-tls" not found
Oct 30 18:33:48.752456 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:48.752451    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351-registry-tls podName:9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351 nodeName:}" failed. No retries permitted until 2026-10-30 18:33:52.752442437 +0000 UTC m=+39.243876878 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351-registry-tls") pod "image-registry-64bf5b6b76-dpvvs" (UID: "9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351") : secret "image-registry-tls" not found
Oct 30 18:33:48.752672 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:48.752468    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/97b19aa3-42be-4def-8d7a-8e3516852e79-metrics-tls podName:97b19aa3-42be-4def-8d7a-8e3516852e79 nodeName:}" failed. No retries permitted until 2026-10-30 18:33:52.752460098 +0000 UTC m=+39.243894539 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/97b19aa3-42be-4def-8d7a-8e3516852e79-metrics-tls") pod "dns-default-r2nx2" (UID: "97b19aa3-42be-4def-8d7a-8e3516852e79") : secret "dns-default-metrics-tls" not found
Oct 30 18:33:49.170230 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:49.170191    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-jwwq2" event={"ID":"f4ac17da-6b5b-4441-9b4e-ebe696e06507","Type":"ContainerStarted","Data":"fb8be5f54ecac0fed745a37326524f2e7ad0f1b855562734b953d37e6329768d"}
Oct 30 18:33:49.174961 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:49.174934    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-8xskh" event={"ID":"613020c0-f3e4-4650-896f-748a40f0ab8b","Type":"ContainerStarted","Data":"41306bd73702bf41bacd24f69e9da18eaa60d7a3d5c5782b2c004af7846ac805"}
Oct 30 18:33:49.177613 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:49.177588    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-g2svp" event={"ID":"46d9ac39-ee45-4e43-bda7-3931c3e599cf","Type":"ContainerStarted","Data":"f5a68d540ea6105c69282b72fcef5f4df6de3fef4565d6e65827425d667c1633"}
Oct 30 18:33:49.287121 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:49.287060    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-jwwq2" podStartSLOduration=250.337714893 podStartE2EDuration="4m13.287041417s" totalImagesPullingTime="2.949326524s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:29:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:33:49.285751586 +0000 UTC m=+35.777186051" watchObservedRunningTime="2026-10-30 18:33:49.287041417 +0000 UTC m=+35.778475880"
Oct 30 18:33:50.186544 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:50.186470    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-7x4rb_382e8b31-a7fa-43f3-9f15-4c61627b0c04/console-operator/0.log"
Oct 30 18:33:50.186544 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:50.186524    2574 generic.go:356] "Generic (PLEG): container finished" podID="382e8b31-a7fa-43f3-9f15-4c61627b0c04" containerID="9a8666624da6b935f790f3680e6cc4a97c576e0b6c028c8b81454e96419084d7" exitCode=255
Oct 30 18:33:50.187058 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:50.186619    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-7x4rb" event={"ID":"382e8b31-a7fa-43f3-9f15-4c61627b0c04","Type":"ContainerDied","Data":"9a8666624da6b935f790f3680e6cc4a97c576e0b6c028c8b81454e96419084d7"}
Oct 30 18:33:50.187058 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:50.186863    2574 scope.go:122] "RemoveContainer" containerID="9a8666624da6b935f790f3680e6cc4a97c576e0b6c028c8b81454e96419084d7"
Oct 30 18:33:51.193019 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:51.192921    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-7x4rb_382e8b31-a7fa-43f3-9f15-4c61627b0c04/console-operator/1.log"
Oct 30 18:33:51.193719 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:51.193700    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-7x4rb_382e8b31-a7fa-43f3-9f15-4c61627b0c04/console-operator/0.log"
Oct 30 18:33:51.193823 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:51.193745    2574 generic.go:356] "Generic (PLEG): container finished" podID="382e8b31-a7fa-43f3-9f15-4c61627b0c04" containerID="b7e8a752d2699b7e5514e278ff1b7c9fd161b532812a42ac3605f0cdaad0c160" exitCode=255
Oct 30 18:33:51.193823 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:51.193800    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-7x4rb" event={"ID":"382e8b31-a7fa-43f3-9f15-4c61627b0c04","Type":"ContainerDied","Data":"b7e8a752d2699b7e5514e278ff1b7c9fd161b532812a42ac3605f0cdaad0c160"}
Oct 30 18:33:51.193927 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:51.193870    2574 scope.go:122] "RemoveContainer" containerID="9a8666624da6b935f790f3680e6cc4a97c576e0b6c028c8b81454e96419084d7"
Oct 30 18:33:51.194124 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:51.194103    2574 scope.go:122] "RemoveContainer" containerID="b7e8a752d2699b7e5514e278ff1b7c9fd161b532812a42ac3605f0cdaad0c160"
Oct 30 18:33:51.194396 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:51.194365    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-7x4rb_openshift-console-operator(382e8b31-a7fa-43f3-9f15-4c61627b0c04)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-7x4rb" podUID="382e8b31-a7fa-43f3-9f15-4c61627b0c04"
Oct 30 18:33:51.195987 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:51.195964    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-7c4986d449-xvgqc" event={"ID":"7c09088b-3a07-426d-8804-f0f5bac4d27c","Type":"ContainerStarted","Data":"276addd4803db94cae7b9a84b5533cf0ec00c4618e8b8d16f123052f2f82ceb4"}
Oct 30 18:33:51.842873 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:51.840904    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-insights/insights-operator-7c4986d449-xvgqc" podStartSLOduration=247.940251291 podStartE2EDuration="4m12.840883545s" totalImagesPullingTime="4.900632254s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:29:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:33:51.433343977 +0000 UTC m=+37.924778440" watchObservedRunningTime="2026-10-30 18:33:51.840883545 +0000 UTC m=+38.332318008"
Oct 30 18:33:51.842873 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:51.842030    2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-jbsw6"]
Oct 30 18:33:51.858660 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:51.858632    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-jbsw6"
Oct 30 18:33:51.913943 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:51.913912    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Oct 30 18:33:51.914826 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:51.914799    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-jbsw6"]
Oct 30 18:33:51.923127 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:51.923103    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Oct 30 18:33:51.923273 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:51.923224    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-k76xp\""
Oct 30 18:33:51.994481 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:51.994422    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/57ca8600-79e7-4f59-9d11-b1c9ec3d358a-data-volume\") pod \"insights-runtime-extractor-jbsw6\" (UID: \"57ca8600-79e7-4f59-9d11-b1c9ec3d358a\") " pod="openshift-insights/insights-runtime-extractor-jbsw6"
Oct 30 18:33:51.994658 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:51.994512    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/57ca8600-79e7-4f59-9d11-b1c9ec3d358a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jbsw6\" (UID: \"57ca8600-79e7-4f59-9d11-b1c9ec3d358a\") " pod="openshift-insights/insights-runtime-extractor-jbsw6"
Oct 30 18:33:51.994658 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:51.994604    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/57ca8600-79e7-4f59-9d11-b1c9ec3d358a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jbsw6\" (UID: \"57ca8600-79e7-4f59-9d11-b1c9ec3d358a\") " pod="openshift-insights/insights-runtime-extractor-jbsw6"
Oct 30 18:33:51.994842 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:51.994665    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/57ca8600-79e7-4f59-9d11-b1c9ec3d358a-crio-socket\") pod \"insights-runtime-extractor-jbsw6\" (UID: \"57ca8600-79e7-4f59-9d11-b1c9ec3d358a\") " pod="openshift-insights/insights-runtime-extractor-jbsw6"
Oct 30 18:33:51.994842 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:51.994753    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wrmcg\" (UniqueName: \"kubernetes.io/projected/57ca8600-79e7-4f59-9d11-b1c9ec3d358a-kube-api-access-wrmcg\") pod \"insights-runtime-extractor-jbsw6\" (UID: \"57ca8600-79e7-4f59-9d11-b1c9ec3d358a\") " pod="openshift-insights/insights-runtime-extractor-jbsw6"
Oct 30 18:33:52.096474 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:52.095665    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wrmcg\" (UniqueName: \"kubernetes.io/projected/57ca8600-79e7-4f59-9d11-b1c9ec3d358a-kube-api-access-wrmcg\") pod \"insights-runtime-extractor-jbsw6\" (UID: \"57ca8600-79e7-4f59-9d11-b1c9ec3d358a\") " pod="openshift-insights/insights-runtime-extractor-jbsw6"
Oct 30 18:33:52.096474 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:52.095794    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/57ca8600-79e7-4f59-9d11-b1c9ec3d358a-data-volume\") pod \"insights-runtime-extractor-jbsw6\" (UID: \"57ca8600-79e7-4f59-9d11-b1c9ec3d358a\") " pod="openshift-insights/insights-runtime-extractor-jbsw6"
Oct 30 18:33:52.096474 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:52.095820    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/57ca8600-79e7-4f59-9d11-b1c9ec3d358a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jbsw6\" (UID: \"57ca8600-79e7-4f59-9d11-b1c9ec3d358a\") " pod="openshift-insights/insights-runtime-extractor-jbsw6"
Oct 30 18:33:52.096474 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:52.095882    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/57ca8600-79e7-4f59-9d11-b1c9ec3d358a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jbsw6\" (UID: \"57ca8600-79e7-4f59-9d11-b1c9ec3d358a\") " pod="openshift-insights/insights-runtime-extractor-jbsw6"
Oct 30 18:33:52.096474 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:52.095946    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/57ca8600-79e7-4f59-9d11-b1c9ec3d358a-crio-socket\") pod \"insights-runtime-extractor-jbsw6\" (UID: \"57ca8600-79e7-4f59-9d11-b1c9ec3d358a\") " pod="openshift-insights/insights-runtime-extractor-jbsw6"
Oct 30 18:33:52.096474 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:52.096102    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/57ca8600-79e7-4f59-9d11-b1c9ec3d358a-crio-socket\") pod \"insights-runtime-extractor-jbsw6\" (UID: \"57ca8600-79e7-4f59-9d11-b1c9ec3d358a\") " pod="openshift-insights/insights-runtime-extractor-jbsw6"
Oct 30 18:33:52.096474 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:52.096398    2574 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Oct 30 18:33:52.096474 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:52.096440    2574 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 30 18:33:52.096474 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:52.096470    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/57ca8600-79e7-4f59-9d11-b1c9ec3d358a-kube-rbac-proxy-cm podName:57ca8600-79e7-4f59-9d11-b1c9ec3d358a nodeName:}" failed. No retries permitted until 2026-10-30 18:33:52.596449769 +0000 UTC m=+39.087884230 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/57ca8600-79e7-4f59-9d11-b1c9ec3d358a-kube-rbac-proxy-cm") pod "insights-runtime-extractor-jbsw6" (UID: "57ca8600-79e7-4f59-9d11-b1c9ec3d358a") : configmap "kube-rbac-proxy" not found
Oct 30 18:33:52.096994 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:52.096489    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/57ca8600-79e7-4f59-9d11-b1c9ec3d358a-insights-runtime-extractor-tls podName:57ca8600-79e7-4f59-9d11-b1c9ec3d358a nodeName:}" failed. No retries permitted until 2026-10-30 18:33:52.596478331 +0000 UTC m=+39.087912792 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/57ca8600-79e7-4f59-9d11-b1c9ec3d358a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jbsw6" (UID: "57ca8600-79e7-4f59-9d11-b1c9ec3d358a") : secret "insights-runtime-extractor-tls" not found
Oct 30 18:33:52.096994 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:52.096721    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/57ca8600-79e7-4f59-9d11-b1c9ec3d358a-data-volume\") pod \"insights-runtime-extractor-jbsw6\" (UID: \"57ca8600-79e7-4f59-9d11-b1c9ec3d358a\") " pod="openshift-insights/insights-runtime-extractor-jbsw6"
Oct 30 18:33:52.157875 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:52.157847    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wrmcg\" (UniqueName: \"kubernetes.io/projected/57ca8600-79e7-4f59-9d11-b1c9ec3d358a-kube-api-access-wrmcg\") pod \"insights-runtime-extractor-jbsw6\" (UID: \"57ca8600-79e7-4f59-9d11-b1c9ec3d358a\") " pod="openshift-insights/insights-runtime-extractor-jbsw6"
Oct 30 18:33:52.204708 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:52.204682    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-7x4rb_382e8b31-a7fa-43f3-9f15-4c61627b0c04/console-operator/1.log"
Oct 30 18:33:52.205154 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:52.205032    2574 scope.go:122] "RemoveContainer" containerID="b7e8a752d2699b7e5514e278ff1b7c9fd161b532812a42ac3605f0cdaad0c160"
Oct 30 18:33:52.205242 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:52.205214    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-7x4rb_openshift-console-operator(382e8b31-a7fa-43f3-9f15-4c61627b0c04)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-7x4rb" podUID="382e8b31-a7fa-43f3-9f15-4c61627b0c04"
Oct 30 18:33:52.208201 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:52.208173    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-kr5qq" event={"ID":"36a04851-81db-40cd-97ef-8e2e00a08604","Type":"ContainerStarted","Data":"2e0d65f31c3b3a319575ad6c5edc8c6071ebc79d492e57cf2b3edc5ede37edbe"}
Oct 30 18:33:52.210378 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:52.210353    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-5bqdc" event={"ID":"09c13759-50d4-40e6-b854-4691822b8c8a","Type":"ContainerStarted","Data":"caf9fc34319c58ec44f0f47e3b04f2501dfca1c85bb798c8fcb7f1823eecac04"}
Oct 30 18:33:52.439660 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:52.438529    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-kr5qq" podStartSLOduration=256.728809971 podStartE2EDuration="4m22.438515267s" totalImagesPullingTime="5.709705296s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:29:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:33:52.437445635 +0000 UTC m=+38.928880097" watchObservedRunningTime="2026-10-30 18:33:52.438515267 +0000 UTC m=+38.929949725"
Oct 30 18:33:52.600970 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:52.600785    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/57ca8600-79e7-4f59-9d11-b1c9ec3d358a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jbsw6\" (UID: \"57ca8600-79e7-4f59-9d11-b1c9ec3d358a\") " pod="openshift-insights/insights-runtime-extractor-jbsw6"
Oct 30 18:33:52.600970 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:52.600874    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/57ca8600-79e7-4f59-9d11-b1c9ec3d358a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jbsw6\" (UID: \"57ca8600-79e7-4f59-9d11-b1c9ec3d358a\") " pod="openshift-insights/insights-runtime-extractor-jbsw6"
Oct 30 18:33:52.601179 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:52.601106    2574 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 30 18:33:52.601179 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:52.601167    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/57ca8600-79e7-4f59-9d11-b1c9ec3d358a-insights-runtime-extractor-tls podName:57ca8600-79e7-4f59-9d11-b1c9ec3d358a nodeName:}" failed. No retries permitted until 2026-10-30 18:33:53.601150068 +0000 UTC m=+40.092584522 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/57ca8600-79e7-4f59-9d11-b1c9ec3d358a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jbsw6" (UID: "57ca8600-79e7-4f59-9d11-b1c9ec3d358a") : secret "insights-runtime-extractor-tls" not found
Oct 30 18:33:52.601385 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:52.601362    2574 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Oct 30 18:33:52.601452 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:52.601413    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/57ca8600-79e7-4f59-9d11-b1c9ec3d358a-kube-rbac-proxy-cm podName:57ca8600-79e7-4f59-9d11-b1c9ec3d358a nodeName:}" failed. No retries permitted until 2026-10-30 18:33:53.60140032 +0000 UTC m=+40.092834775 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/57ca8600-79e7-4f59-9d11-b1c9ec3d358a-kube-rbac-proxy-cm") pod "insights-runtime-extractor-jbsw6" (UID: "57ca8600-79e7-4f59-9d11-b1c9ec3d358a") : configmap "kube-rbac-proxy" not found
Oct 30 18:33:52.702101 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:52.701340    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/71fce796-0032-481b-80b3-66fbb731452c-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-r8rgt\" (UID: \"71fce796-0032-481b-80b3-66fbb731452c\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-r8rgt"
Oct 30 18:33:52.702101 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:52.701443    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/68195e19-b1ff-4148-9fae-3545f2997ff5-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-gmswj\" (UID: \"68195e19-b1ff-4148-9fae-3545f2997ff5\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-gmswj"
Oct 30 18:33:52.702101 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:52.701504    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8789ec5b-226e-4551-bf68-3ac585006266-registry-tls\") pod \"image-registry-74757bccd5-qmrkp\" (UID: \"8789ec5b-226e-4551-bf68-3ac585006266\") " pod="openshift-image-registry/image-registry-74757bccd5-qmrkp"
Oct 30 18:33:52.702101 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:52.701550    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/da28351c-1c8b-40be-85a0-3aad427e3a17-service-ca-bundle\") pod \"router-default-7dfcf9fd6c-vh769\" (UID: \"da28351c-1c8b-40be-85a0-3aad427e3a17\") " pod="openshift-ingress/router-default-7dfcf9fd6c-vh769"
Oct 30 18:33:52.702101 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:52.701703    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/da28351c-1c8b-40be-85a0-3aad427e3a17-service-ca-bundle podName:da28351c-1c8b-40be-85a0-3aad427e3a17 nodeName:}" failed. No retries permitted until 2026-10-30 18:34:00.7016838 +0000 UTC m=+47.193118242 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/da28351c-1c8b-40be-85a0-3aad427e3a17-service-ca-bundle") pod "router-default-7dfcf9fd6c-vh769" (UID: "da28351c-1c8b-40be-85a0-3aad427e3a17") : configmap references non-existent config key: service-ca.crt
Oct 30 18:33:52.702101 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:52.701788    2574 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Oct 30 18:33:52.702101 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:52.701820    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/71fce796-0032-481b-80b3-66fbb731452c-networking-console-plugin-cert podName:71fce796-0032-481b-80b3-66fbb731452c nodeName:}" failed. No retries permitted until 2026-10-30 18:34:00.7018119 +0000 UTC m=+47.193246341 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/71fce796-0032-481b-80b3-66fbb731452c-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-r8rgt" (UID: "71fce796-0032-481b-80b3-66fbb731452c") : secret "networking-console-plugin-cert" not found
Oct 30 18:33:52.702101 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:52.701974    2574 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 30 18:33:52.702101 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:52.702026    2574 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-74757bccd5-qmrkp: secret "image-registry-tls" not found
Oct 30 18:33:52.702101 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:52.702081    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8789ec5b-226e-4551-bf68-3ac585006266-registry-tls podName:8789ec5b-226e-4551-bf68-3ac585006266 nodeName:}" failed. No retries permitted until 2026-10-30 18:34:00.702067432 +0000 UTC m=+47.193501876 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8789ec5b-226e-4551-bf68-3ac585006266-registry-tls") pod "image-registry-74757bccd5-qmrkp" (UID: "8789ec5b-226e-4551-bf68-3ac585006266") : secret "image-registry-tls" not found
Oct 30 18:33:52.702730 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:52.702655    2574 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Oct 30 18:33:52.702730 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:52.702713    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68195e19-b1ff-4148-9fae-3545f2997ff5-cluster-monitoring-operator-tls podName:68195e19-b1ff-4148-9fae-3545f2997ff5 nodeName:}" failed. No retries permitted until 2026-10-30 18:34:00.702701861 +0000 UTC m=+47.194136300 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/68195e19-b1ff-4148-9fae-3545f2997ff5-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-gmswj" (UID: "68195e19-b1ff-4148-9fae-3545f2997ff5") : secret "cluster-monitoring-operator-tls" not found
Oct 30 18:33:52.802488 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:52.802441    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/15d485f9-5a4b-4f4b-9e59-a6eb062b4c5c-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-pm58m\" (UID: \"15d485f9-5a4b-4f4b-9e59-a6eb062b4c5c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-pm58m"
Oct 30 18:33:52.802654 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:52.802507    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351-registry-tls\") pod \"image-registry-64bf5b6b76-dpvvs\" (UID: \"9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351\") " pod="openshift-image-registry/image-registry-64bf5b6b76-dpvvs"
Oct 30 18:33:52.802654 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:52.802560    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/da28351c-1c8b-40be-85a0-3aad427e3a17-metrics-certs\") pod \"router-default-7dfcf9fd6c-vh769\" (UID: \"da28351c-1c8b-40be-85a0-3aad427e3a17\") " pod="openshift-ingress/router-default-7dfcf9fd6c-vh769"
Oct 30 18:33:52.802654 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:52.802600    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/97b19aa3-42be-4def-8d7a-8e3516852e79-metrics-tls\") pod \"dns-default-r2nx2\" (UID: \"97b19aa3-42be-4def-8d7a-8e3516852e79\") " pod="openshift-dns/dns-default-r2nx2"
Oct 30 18:33:52.803132 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:52.802822    2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 30 18:33:52.803132 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:52.802907    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/97b19aa3-42be-4def-8d7a-8e3516852e79-metrics-tls podName:97b19aa3-42be-4def-8d7a-8e3516852e79 nodeName:}" failed. No retries permitted until 2026-10-30 18:34:00.802892426 +0000 UTC m=+47.294326867 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/97b19aa3-42be-4def-8d7a-8e3516852e79-metrics-tls") pod "dns-default-r2nx2" (UID: "97b19aa3-42be-4def-8d7a-8e3516852e79") : secret "dns-default-metrics-tls" not found
Oct 30 18:33:52.803132 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:52.802927    2574 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 30 18:33:52.803132 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:52.802947    2574 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-64bf5b6b76-dpvvs: secret "image-registry-tls" not found
Oct 30 18:33:52.803132 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:52.803001    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351-registry-tls podName:9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351 nodeName:}" failed. No retries permitted until 2026-10-30 18:34:00.802983845 +0000 UTC m=+47.294418299 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351-registry-tls") pod "image-registry-64bf5b6b76-dpvvs" (UID: "9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351") : secret "image-registry-tls" not found
Oct 30 18:33:52.803132 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:52.803059    2574 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Oct 30 18:33:52.803132 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:52.803073    2574 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Oct 30 18:33:52.803132 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:52.803089    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/da28351c-1c8b-40be-85a0-3aad427e3a17-metrics-certs podName:da28351c-1c8b-40be-85a0-3aad427e3a17 nodeName:}" failed. No retries permitted until 2026-10-30 18:34:00.803081092 +0000 UTC m=+47.294515533 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/da28351c-1c8b-40be-85a0-3aad427e3a17-metrics-certs") pod "router-default-7dfcf9fd6c-vh769" (UID: "da28351c-1c8b-40be-85a0-3aad427e3a17") : secret "router-metrics-certs-default" not found
Oct 30 18:33:52.803132 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:52.803110    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/15d485f9-5a4b-4f4b-9e59-a6eb062b4c5c-samples-operator-tls podName:15d485f9-5a4b-4f4b-9e59-a6eb062b4c5c nodeName:}" failed. No retries permitted until 2026-10-30 18:34:00.803098973 +0000 UTC m=+47.294533435 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/15d485f9-5a4b-4f4b-9e59-a6eb062b4c5c-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-pm58m" (UID: "15d485f9-5a4b-4f4b-9e59-a6eb062b4c5c") : secret "samples-operator-tls" not found
Oct 30 18:33:53.215208 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:53.215096    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-4969l" event={"ID":"81de5dce-c40f-4bd5-878c-5d75a0487066","Type":"ContainerStarted","Data":"4a44a6f8753232623b2d0b379293abed868b794f09a5313b1b41ce5bf2a4af47"}
Oct 30 18:33:53.217547 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:53.217523    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-g9fhp" event={"ID":"e6b2a584-ae78-4e80-b595-9b1a5897d556","Type":"ContainerStarted","Data":"a67e8e2d43ea2ea8ed7d89c88c8914469deec8c3cbda962573bccee3c1634bd4"}
Oct 30 18:33:53.331847 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:53.331793    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-4969l" podStartSLOduration=247.445950977 podStartE2EDuration="4m14.331778677s" totalImagesPullingTime="6.8858277s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:29:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:33:53.329901012 +0000 UTC m=+39.821335475" watchObservedRunningTime="2026-10-30 18:33:53.331778677 +0000 UTC m=+39.823213139"
Oct 30 18:33:53.332159 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:53.332123    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-5bqdc" podStartSLOduration=249.575932207 podStartE2EDuration="4m15.33211101s" totalImagesPullingTime="5.756178803s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:29:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:33:52.557934196 +0000 UTC m=+39.049368658" watchObservedRunningTime="2026-10-30 18:33:53.33211101 +0000 UTC m=+39.823545505"
Oct 30 18:33:53.453061 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:53.453010    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-g9fhp" podStartSLOduration=2.9305716 podStartE2EDuration="9.452995922s" totalImagesPullingTime="6.522424322s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:33:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:33:53.451921609 +0000 UTC m=+39.943356072" watchObservedRunningTime="2026-10-30 18:33:53.452995922 +0000 UTC m=+39.944430385"
Oct 30 18:33:53.611402 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:53.611370    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/57ca8600-79e7-4f59-9d11-b1c9ec3d358a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jbsw6\" (UID: \"57ca8600-79e7-4f59-9d11-b1c9ec3d358a\") " pod="openshift-insights/insights-runtime-extractor-jbsw6"
Oct 30 18:33:53.611507 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:53.611450    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/57ca8600-79e7-4f59-9d11-b1c9ec3d358a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jbsw6\" (UID: \"57ca8600-79e7-4f59-9d11-b1c9ec3d358a\") " pod="openshift-insights/insights-runtime-extractor-jbsw6"
Oct 30 18:33:53.611572 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:53.611535    2574 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Oct 30 18:33:53.611625 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:53.611605    2574 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 30 18:33:53.611669 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:53.611628    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/57ca8600-79e7-4f59-9d11-b1c9ec3d358a-kube-rbac-proxy-cm podName:57ca8600-79e7-4f59-9d11-b1c9ec3d358a nodeName:}" failed. No retries permitted until 2026-10-30 18:33:55.611606349 +0000 UTC m=+42.103040805 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/57ca8600-79e7-4f59-9d11-b1c9ec3d358a-kube-rbac-proxy-cm") pod "insights-runtime-extractor-jbsw6" (UID: "57ca8600-79e7-4f59-9d11-b1c9ec3d358a") : configmap "kube-rbac-proxy" not found
Oct 30 18:33:53.611669 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:53.611660    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/57ca8600-79e7-4f59-9d11-b1c9ec3d358a-insights-runtime-extractor-tls podName:57ca8600-79e7-4f59-9d11-b1c9ec3d358a nodeName:}" failed. No retries permitted until 2026-10-30 18:33:55.611647565 +0000 UTC m=+42.103082018 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/57ca8600-79e7-4f59-9d11-b1c9ec3d358a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jbsw6" (UID: "57ca8600-79e7-4f59-9d11-b1c9ec3d358a") : secret "insights-runtime-extractor-tls" not found
Oct 30 18:33:53.662649 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:53.662624    2574 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-ht9zn"]
Oct 30 18:33:53.677011 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:53.676984    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-ht9zn"
Oct 30 18:33:53.727577 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:53.727552    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-ht9zn"]
Oct 30 18:33:53.804925 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:53.804886    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Oct 30 18:33:53.804925 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:53.804901    2574 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Oct 30 18:33:53.805117 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:53.805085    2574 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-phb6k\""
Oct 30 18:33:53.813605 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:53.813561    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jt475\" (UniqueName: \"kubernetes.io/projected/a3cf021c-2c20-4687-9b3a-cfa65ce1ee8b-kube-api-access-jt475\") pod \"migrator-66df5d496b-ht9zn\" (UID: \"a3cf021c-2c20-4687-9b3a-cfa65ce1ee8b\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-ht9zn"
Oct 30 18:33:53.914845 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:53.914764    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jt475\" (UniqueName: \"kubernetes.io/projected/a3cf021c-2c20-4687-9b3a-cfa65ce1ee8b-kube-api-access-jt475\") pod \"migrator-66df5d496b-ht9zn\" (UID: \"a3cf021c-2c20-4687-9b3a-cfa65ce1ee8b\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-ht9zn"
Oct 30 18:33:53.985583 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:53.985558    2574 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jt475\" (UniqueName: \"kubernetes.io/projected/a3cf021c-2c20-4687-9b3a-cfa65ce1ee8b-kube-api-access-jt475\") pod \"migrator-66df5d496b-ht9zn\" (UID: \"a3cf021c-2c20-4687-9b3a-cfa65ce1ee8b\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-ht9zn"
Oct 30 18:33:54.010828 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:54.010789    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-ht9zn"
Oct 30 18:33:54.223312 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:54.223195    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-8xskh" event={"ID":"613020c0-f3e4-4650-896f-748a40f0ab8b","Type":"ContainerStarted","Data":"050960a77231256df9a53864bf6d84a95d64cb29adcf725d94a502f496231dbb"}
Oct 30 18:33:54.223734 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:54.223316    2574 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-8xskh"
Oct 30 18:33:54.339900 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:54.339845    2574 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-8xskh" podStartSLOduration=35.427481006 podStartE2EDuration="40.339825896s" totalImagesPullingTime="4.91234489s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:33:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:33:54.338780349 +0000 UTC m=+40.830214814" watchObservedRunningTime="2026-10-30 18:33:54.339825896 +0000 UTC m=+40.831260358"
Oct 30 18:33:54.452393 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:54.452363    2574 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-ht9zn"]
Oct 30 18:33:54.458595 ip-10-0-134-46 kubenswrapper[2574]: W1030 18:33:54.458564    2574 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3cf021c_2c20_4687_9b3a_cfa65ce1ee8b.slice/crio-83e431df5ea6349b7b60e71173b402e4ef22fd518c8f8b81c8eccd92d1179b40 WatchSource:0}: Error finding container 83e431df5ea6349b7b60e71173b402e4ef22fd518c8f8b81c8eccd92d1179b40: Status 404 returned error can't find the container with id 83e431df5ea6349b7b60e71173b402e4ef22fd518c8f8b81c8eccd92d1179b40
Oct 30 18:33:55.227406 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:55.227373    2574 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-ht9zn" event={"ID":"a3cf021c-2c20-4687-9b3a-cfa65ce1ee8b","Type":"ContainerStarted","Data":"83e431df5ea6349b7b60e71173b402e4ef22fd518c8f8b81c8eccd92d1179b40"}
Oct 30 18:33:55.230719 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:55.230701    2574 kubelet.go:2727] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7cd56b8dbc-7x4rb"
Oct 30 18:33:55.230818 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:55.230734    2574 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7cd56b8dbc-7x4rb"
Oct 30 18:33:55.230999 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:55.230988    2574 scope.go:122] "RemoveContainer" containerID="b7e8a752d2699b7e5514e278ff1b7c9fd161b532812a42ac3605f0cdaad0c160"
Oct 30 18:33:55.231138 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:55.231122    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-7x4rb_openshift-console-operator(382e8b31-a7fa-43f3-9f15-4c61627b0c04)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-7x4rb" podUID="382e8b31-a7fa-43f3-9f15-4c61627b0c04"
Oct 30 18:33:55.629771 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:55.629730    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/57ca8600-79e7-4f59-9d11-b1c9ec3d358a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jbsw6\" (UID: \"57ca8600-79e7-4f59-9d11-b1c9ec3d358a\") " pod="openshift-insights/insights-runtime-extractor-jbsw6"
Oct 30 18:33:55.629955 ip-10-0-134-46 kubenswrapper[2574]: I1030 18:33:55.629832    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/57ca8600-79e7-4f59-9d11-b1c9ec3d358a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jbsw6\" (UID: \"57ca8600-79e7-4f59-9d11-b1c9ec3d358a\") " pod="openshift-insights/insights-runtime-extractor-jbsw6"
Oct 30 18:33:55.629955 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:55.629854    2574 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Oct 30 18:33:55.630029 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:55.629972    2574 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 30 18:33:55.630029 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:55.629994    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/57ca8600-79e7-4f59-9d11-b1c9ec3d358a-kube-rbac-proxy-cm podName:57ca8600-79e7-4f59-9d11-b1c9ec3d358a nodeName:}" failed. No retries permitted until 2026-10-30 18:33:59.629974787 +0000 UTC m=+46.121409242 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/57ca8600-79e7-4f59-9d11-b1c9ec3d358a-kube-rbac-proxy-cm") pod "insights-runtime-extractor-jbsw6" (UID: "57ca8600-79e7-4f59-9d11-b1c9ec3d358a") : configmap "kube-rbac-proxy" not found
Oct 30 18:33:55.630029 ip-10-0-134-46 kubenswrapper[2574]: E1030 18:33:55.630028    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/57ca8600-79e7-4f59-9d11-b1c9ec3d358a-insights-runtime-extractor-tls podName:57ca8600-79e7-4f59-9d11-b1c9ec3d358a nodeName:}" failed. No retries permitted until 2026-10-30 18:33:59.630015148 +0000 UTC m=+46.121449601 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/57ca8600-79e7-4f59-9d11-b1c9ec3d358a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jbsw6" (UID: "57ca8600-79e7-4f59-9d11-b1c9ec3d358a") : secret "insights-runtime-extractor-tls" not found
Oct 30 18:33:55.691039 ip-10-0-134-46 systemd[1]: Stopping Kubernetes Kubelet...
Oct 30 18:33:55.707544 ip-10-0-134-46 systemd[1]: kubelet.service: Deactivated successfully.
Oct 30 18:33:55.707742 ip-10-0-134-46 systemd[1]: Stopped Kubernetes Kubelet.
Oct 30 18:33:55.707797 ip-10-0-134-46 systemd[1]: kubelet.service: Consumed 3.009s CPU time.
Oct 30 18:33:55.720823 ip-10-0-134-46 systemd[1]: Starting Kubernetes Kubelet...
Oct 30 18:33:55.796249 ip-10-0-134-46 kubenswrapper[4888]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 18:33:55.796249 ip-10-0-134-46 kubenswrapper[4888]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Oct 30 18:33:55.796249 ip-10-0-134-46 kubenswrapper[4888]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 18:33:55.796249 ip-10-0-134-46 kubenswrapper[4888]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 18:33:55.798552 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.798477    4888 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 18:33:55.798552 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.798551    4888 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 18:33:55.798552 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.798554    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 18:33:55.798658 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.798558    4888 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 18:33:55.798658 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.798560    4888 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 18:33:55.798658 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.798564    4888 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 18:33:55.798658 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.798567    4888 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 18:33:55.798658 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.798570    4888 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 18:33:55.798658 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.798573    4888 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 18:33:55.798658 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.798575    4888 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 18:33:55.798658 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.798578    4888 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 18:33:55.798658 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.798580    4888 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 18:33:55.798658 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.798583    4888 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 18:33:55.798658 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.798586    4888 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 18:33:55.798658 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.798588    4888 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 18:33:55.798658 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.798591    4888 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 18:33:55.798658 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.798593    4888 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 18:33:55.798658 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.798596    4888 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 18:33:55.798658 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.798598    4888 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 18:33:55.798658 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.798601    4888 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 18:33:55.798658 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.798607    4888 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 18:33:55.798658 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.798610    4888 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 18:33:55.798658 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.798612    4888 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 18:33:55.799140 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.798615    4888 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 18:33:55.799140 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.798618    4888 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 18:33:55.799140 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.798621    4888 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 18:33:55.799140 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.798624    4888 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 18:33:55.799140 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.798627    4888 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 18:33:55.799140 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.798629    4888 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 18:33:55.799140 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.798632    4888 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 18:33:55.799140 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.798635    4888 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 18:33:55.799140 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.798637    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 18:33:55.799140 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.798640    4888 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 18:33:55.799140 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.798643    4888 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 18:33:55.799140 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.798645    4888 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 18:33:55.799140 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.798648    4888 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 18:33:55.799140 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.798650    4888 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 18:33:55.799140 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.798653    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 18:33:55.799140 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.798655    4888 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 18:33:55.799140 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.798659    4888 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 18:33:55.799140 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.798662    4888 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 18:33:55.799140 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.798664    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 18:33:55.799140 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.798667    4888 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 18:33:55.799693 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.798670    4888 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 18:33:55.799693 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.798673    4888 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 18:33:55.799693 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.798676    4888 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 18:33:55.799693 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.798678    4888 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 18:33:55.799693 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.798681    4888 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 18:33:55.799693 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.798684    4888 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 18:33:55.799693 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.798686    4888 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 18:33:55.799693 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.798689    4888 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 18:33:55.799693 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.798692    4888 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 18:33:55.799693 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.798694    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 18:33:55.799693 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.798697    4888 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 18:33:55.799693 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.798699    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 18:33:55.799693 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.798701    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 18:33:55.799693 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.798704    4888 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 18:33:55.799693 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.798707    4888 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 18:33:55.799693 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.798709    4888 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 18:33:55.799693 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.798712    4888 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 18:33:55.799693 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.798714    4888 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 18:33:55.799693 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.798717    4888 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 18:33:55.800143 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.798719    4888 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 18:33:55.800143 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.798722    4888 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 18:33:55.800143 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.798724    4888 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 18:33:55.800143 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.798727    4888 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 18:33:55.800143 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.798731    4888 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 18:33:55.800143 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.798735    4888 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 18:33:55.800143 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.798738    4888 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 18:33:55.800143 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.798741    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 18:33:55.800143 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.798743    4888 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 18:33:55.800143 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.798746    4888 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 18:33:55.800143 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.798749    4888 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 18:33:55.800143 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.798753    4888 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 18:33:55.800143 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.798757    4888 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 18:33:55.800143 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.798760    4888 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 18:33:55.800143 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.798763    4888 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 18:33:55.800143 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.798765    4888 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 18:33:55.800143 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.798768    4888 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 18:33:55.800143 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.798770    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 18:33:55.800143 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.798773    4888 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 18:33:55.800752 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.798775    4888 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 18:33:55.800752 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.798777    4888 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 18:33:55.800752 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.798780    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 18:33:55.800752 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.798783    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 18:33:55.800752 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.798785    4888 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 18:33:55.800752 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.798788    4888 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 18:33:55.800752 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.798791    4888 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 18:33:55.800752 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.798794    4888 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 18:33:55.800752 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.798797    4888 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 18:33:55.800752 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.798800    4888 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 18:33:55.800752 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.798802    4888 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 18:33:55.800752 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.798805    4888 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 18:33:55.800752 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.798807    4888 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 18:33:55.800752 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.798810    4888 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 18:33:55.800752 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.798812    4888 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 18:33:55.800752 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.798815    4888 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 18:33:55.800752 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.798817    4888 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 18:33:55.800752 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.798819    4888 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 18:33:55.800752 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.798822    4888 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 18:33:55.800752 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.798824    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 18:33:55.801230 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.798827    4888 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 18:33:55.801230 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.798829    4888 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 18:33:55.801230 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.798832    4888 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 18:33:55.801230 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.798834    4888 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 18:33:55.801230 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.798837    4888 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 18:33:55.801230 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.799605    4888 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 18:33:55.801230 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.799613    4888 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 18:33:55.801230 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.799616    4888 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 18:33:55.801230 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.799619    4888 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 18:33:55.801230 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.799621    4888 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 18:33:55.801230 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.799624    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 18:33:55.801230 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.799627    4888 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 18:33:55.801230 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.799629    4888 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 18:33:55.801230 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.799632    4888 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 18:33:55.801230 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.799634    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 18:33:55.801230 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.799637    4888 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 18:33:55.801230 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.799640    4888 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 18:33:55.801230 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.799642    4888 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 18:33:55.801230 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.799645    4888 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 18:33:55.801230 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.799648    4888 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 18:33:55.801752 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.799651    4888 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 18:33:55.801752 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.799654    4888 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 18:33:55.801752 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.799656    4888 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 18:33:55.801752 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.799659    4888 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 18:33:55.801752 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.799661    4888 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 18:33:55.801752 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.799664    4888 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 18:33:55.801752 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.799666    4888 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 18:33:55.801752 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.799669    4888 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 18:33:55.801752 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.799671    4888 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 18:33:55.801752 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.799673    4888 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 18:33:55.801752 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.799676    4888 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 18:33:55.801752 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.799678    4888 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 18:33:55.801752 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.799680    4888 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 18:33:55.801752 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.799683    4888 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 18:33:55.801752 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.799685    4888 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 18:33:55.801752 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.799688    4888 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 18:33:55.801752 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.799690    4888 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 18:33:55.801752 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.799693    4888 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 18:33:55.801752 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.799696    4888 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 18:33:55.801752 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.799699    4888 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 18:33:55.802237 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.799701    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 18:33:55.802237 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.799704    4888 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 18:33:55.802237 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.799706    4888 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 18:33:55.802237 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.799709    4888 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 18:33:55.802237 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.799712    4888 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 18:33:55.802237 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.799714    4888 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 18:33:55.802237 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.799717    4888 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 18:33:55.802237 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.799720    4888 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 18:33:55.802237 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.799722    4888 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 18:33:55.802237 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.799725    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 18:33:55.802237 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.799727    4888 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 18:33:55.802237 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.799732    4888 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 18:33:55.802237 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.799735    4888 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 18:33:55.802237 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.799738    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 18:33:55.802237 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.799741    4888 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 18:33:55.802237 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.799744    4888 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 18:33:55.802237 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.799746    4888 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 18:33:55.802237 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.799749    4888 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 18:33:55.802237 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.799753    4888 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 18:33:55.802734 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.799756    4888 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 18:33:55.802734 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.799759    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 18:33:55.802734 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.799762    4888 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 18:33:55.802734 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.799765    4888 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 18:33:55.802734 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.799768    4888 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 18:33:55.802734 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.799770    4888 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 18:33:55.802734 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.799773    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 18:33:55.802734 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.799775    4888 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 18:33:55.802734 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.799778    4888 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 18:33:55.802734 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.799780    4888 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 18:33:55.802734 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.799783    4888 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 18:33:55.802734 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.799785    4888 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 18:33:55.802734 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.799788    4888 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 18:33:55.802734 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.799791    4888 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 18:33:55.802734 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.799793    4888 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 18:33:55.802734 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.799796    4888 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 18:33:55.802734 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.799798    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 18:33:55.802734 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.799801    4888 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 18:33:55.802734 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.799803    4888 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 18:33:55.803207 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.799806    4888 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 18:33:55.803207 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.799809    4888 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 18:33:55.803207 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.799811    4888 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 18:33:55.803207 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.799814    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 18:33:55.803207 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.799816    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 18:33:55.803207 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.799820    4888 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 18:33:55.803207 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.799823    4888 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 18:33:55.803207 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.799825    4888 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 18:33:55.803207 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.799827    4888 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 18:33:55.803207 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.799830    4888 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 18:33:55.803207 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.799832    4888 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 18:33:55.803207 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.799835    4888 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 18:33:55.803207 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.799837    4888 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 18:33:55.803207 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.799840    4888 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 18:33:55.803207 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.799842    4888 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 18:33:55.803207 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.799845    4888 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 18:33:55.803207 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.799847    4888 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 18:33:55.803207 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.799850    4888 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 18:33:55.803207 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.799852    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 18:33:55.803713 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.799855    4888 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 18:33:55.803713 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.799857    4888 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 18:33:55.803713 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.799860    4888 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 18:33:55.803713 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.799862    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 18:33:55.803713 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.799865    4888 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 18:33:55.803713 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.799867    4888 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 18:33:55.803713 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.799870    4888 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 18:33:55.803713 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.799873    4888 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 18:33:55.803713 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.799875    4888 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 18:33:55.803713 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.799877    4888 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 18:33:55.803713 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.799880    4888 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 18:33:55.803713 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.799884    4888 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 18:33:55.803713 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.799888    4888 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 18:33:55.803713 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.799890    4888 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 18:33:55.803713 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.800368    4888 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 18:33:55.803713 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.800374    4888 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 18:33:55.803713 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.800378    4888 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 18:33:55.803713 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.800381    4888 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 18:33:55.803713 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.800385    4888 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 18:33:55.804184 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.800388    4888 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 18:33:55.804184 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.800391    4888 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 18:33:55.804184 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.800394    4888 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 18:33:55.804184 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.800396    4888 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 18:33:55.804184 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.800399    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 18:33:55.804184 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.800401    4888 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 18:33:55.804184 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.800405    4888 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 18:33:55.804184 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.800408    4888 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 18:33:55.804184 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.800410    4888 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 18:33:55.804184 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.800413    4888 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 18:33:55.804184 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.800415    4888 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 18:33:55.804184 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.800418    4888 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 18:33:55.804184 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.800420    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 18:33:55.804184 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.800423    4888 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 18:33:55.804184 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.800426    4888 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 18:33:55.804184 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.800428    4888 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 18:33:55.804184 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.800431    4888 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 18:33:55.804184 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.800434    4888 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 18:33:55.804184 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.800436    4888 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 18:33:55.804184 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.800439    4888 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 18:33:55.804719 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.800442    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 18:33:55.804719 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.800445    4888 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 18:33:55.804719 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.800447    4888 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 18:33:55.804719 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.800450    4888 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 18:33:55.804719 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.800452    4888 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 18:33:55.804719 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.800455    4888 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 18:33:55.804719 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.800457    4888 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 18:33:55.804719 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.800460    4888 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 18:33:55.804719 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.800462    4888 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 18:33:55.804719 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.800465    4888 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 18:33:55.804719 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.800467    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 18:33:55.804719 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.800471    4888 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 18:33:55.804719 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.800474    4888 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 18:33:55.804719 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.800477    4888 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 18:33:55.804719 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.800479    4888 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 18:33:55.804719 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.800482    4888 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 18:33:55.804719 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.800484    4888 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 18:33:55.804719 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.800487    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 18:33:55.804719 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.800492    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 18:33:55.805171 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.800495    4888 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 18:33:55.805171 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.800497    4888 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 18:33:55.805171 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.800500    4888 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 18:33:55.805171 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.800502    4888 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 18:33:55.805171 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.800505    4888 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 18:33:55.805171 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.800507    4888 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 18:33:55.805171 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.800510    4888 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 18:33:55.805171 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.800512    4888 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 18:33:55.805171 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.800515    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 18:33:55.805171 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.800517    4888 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 18:33:55.805171 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.800522    4888 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 18:33:55.805171 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.800526    4888 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 18:33:55.805171 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.800529    4888 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 18:33:55.805171 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.800533    4888 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 18:33:55.805171 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.800535    4888 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 18:33:55.805171 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.800538    4888 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 18:33:55.805171 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.800541    4888 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 18:33:55.805171 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.800543    4888 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 18:33:55.805171 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.800546    4888 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 18:33:55.805668 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.800548    4888 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 18:33:55.805668 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.800552    4888 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 18:33:55.805668 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.800554    4888 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 18:33:55.805668 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.800558    4888 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 18:33:55.805668 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.800561    4888 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 18:33:55.805668 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.800565    4888 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 18:33:55.805668 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.800568    4888 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 18:33:55.805668 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.800570    4888 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 18:33:55.805668 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.800573    4888 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 18:33:55.805668 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.800575    4888 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 18:33:55.805668 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.800578    4888 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 18:33:55.805668 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.800580    4888 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 18:33:55.805668 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.800584    4888 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 18:33:55.805668 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.800586    4888 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 18:33:55.805668 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.800589    4888 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 18:33:55.805668 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.800592    4888 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 18:33:55.805668 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.800594    4888 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 18:33:55.805668 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.800597    4888 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 18:33:55.805668 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.800599    4888 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 18:33:55.806343 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.800602    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 18:33:55.806343 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.800604    4888 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 18:33:55.806343 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.800606    4888 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 18:33:55.806343 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.800609    4888 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 18:33:55.806343 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.800611    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 18:33:55.806343 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.800614    4888 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 18:33:55.806343 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.800617    4888 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 18:33:55.806343 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.800620    4888 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 18:33:55.806343 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.800624    4888 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 18:33:55.806343 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.800626    4888 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 18:33:55.806343 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.800629    4888 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 18:33:55.806343 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.800631    4888 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 18:33:55.806343 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.800634    4888 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 18:33:55.806343 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.800636    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 18:33:55.806343 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.800639    4888 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 18:33:55.806343 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.800641    4888 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 18:33:55.806343 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.800644    4888 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 18:33:55.806343 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.800646    4888 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 18:33:55.806343 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.800649    4888 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 18:33:55.806343 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.800652    4888 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 18:33:55.807073 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.800654    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 18:33:55.807073 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.800656    4888 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 18:33:55.807073 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.800659    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 18:33:55.807073 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.800661    4888 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 18:33:55.807073 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.800762    4888 flags.go:64] FLAG: --address="0.0.0.0"
Oct 30 18:33:55.807073 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.800774    4888 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Oct 30 18:33:55.807073 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.800783    4888 flags.go:64] FLAG: --anonymous-auth="true"
Oct 30 18:33:55.807073 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.800790    4888 flags.go:64] FLAG: --application-metrics-count-limit="100"
Oct 30 18:33:55.807073 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.800794    4888 flags.go:64] FLAG: --authentication-token-webhook="false"
Oct 30 18:33:55.807073 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.800798    4888 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Oct 30 18:33:55.807073 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.800802    4888 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Oct 30 18:33:55.807073 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.800806    4888 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Oct 30 18:33:55.807073 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.800809    4888 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Oct 30 18:33:55.807073 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.800812    4888 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Oct 30 18:33:55.807073 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.800815    4888 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Oct 30 18:33:55.807073 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.800818    4888 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Oct 30 18:33:55.807073 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.800821    4888 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Oct 30 18:33:55.807073 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.800824    4888 flags.go:64] FLAG: --cgroup-root=""
Oct 30 18:33:55.807073 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.800826    4888 flags.go:64] FLAG: --cgroups-per-qos="true"
Oct 30 18:33:55.807073 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.800829    4888 flags.go:64] FLAG: --client-ca-file=""
Oct 30 18:33:55.807073 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.800832    4888 flags.go:64] FLAG: --cloud-provider="external"
Oct 30 18:33:55.807073 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.800836    4888 flags.go:64] FLAG: --cluster-dns="[]"
Oct 30 18:33:55.807073 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.800840    4888 flags.go:64] FLAG: --cluster-domain=""
Oct 30 18:33:55.807669 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.800843    4888 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Oct 30 18:33:55.807669 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.800846    4888 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Oct 30 18:33:55.807669 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.800849    4888 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Oct 30 18:33:55.807669 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.800852    4888 flags.go:64] FLAG: --container-log-max-files="5"
Oct 30 18:33:55.807669 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.800856    4888 flags.go:64] FLAG: --container-log-max-size="10Mi"
Oct 30 18:33:55.807669 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.800859    4888 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Oct 30 18:33:55.807669 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.800862    4888 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Oct 30 18:33:55.807669 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.800866    4888 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Oct 30 18:33:55.807669 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.800869    4888 flags.go:64] FLAG: --contention-profiling="false"
Oct 30 18:33:55.807669 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.800872    4888 flags.go:64] FLAG: --cpu-cfs-quota="true"
Oct 30 18:33:55.807669 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.800875    4888 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Oct 30 18:33:55.807669 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.800878    4888 flags.go:64] FLAG: --cpu-manager-policy="none"
Oct 30 18:33:55.807669 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.800881    4888 flags.go:64] FLAG: --cpu-manager-policy-options=""
Oct 30 18:33:55.807669 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.800885    4888 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Oct 30 18:33:55.807669 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.800890    4888 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Oct 30 18:33:55.807669 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.800893    4888 flags.go:64] FLAG: --enable-debugging-handlers="true"
Oct 30 18:33:55.807669 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.800896    4888 flags.go:64] FLAG: --enable-load-reader="false"
Oct 30 18:33:55.807669 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.800899    4888 flags.go:64] FLAG: --enable-server="true"
Oct 30 18:33:55.807669 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.800902    4888 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Oct 30 18:33:55.807669 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.800906    4888 flags.go:64] FLAG: --event-burst="100"
Oct 30 18:33:55.807669 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.800909    4888 flags.go:64] FLAG: --event-qps="50"
Oct 30 18:33:55.807669 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.800912    4888 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Oct 30 18:33:55.807669 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.800914    4888 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Oct 30 18:33:55.807669 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.800917    4888 flags.go:64] FLAG: --eviction-hard=""
Oct 30 18:33:55.808254 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.800921    4888 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Oct 30 18:33:55.808254 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.800924    4888 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Oct 30 18:33:55.808254 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.800927    4888 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Oct 30 18:33:55.808254 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.800930    4888 flags.go:64] FLAG: --eviction-soft=""
Oct 30 18:33:55.808254 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.800934    4888 flags.go:64] FLAG: --eviction-soft-grace-period=""
Oct 30 18:33:55.808254 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.800936    4888 flags.go:64] FLAG: --exit-on-lock-contention="false"
Oct 30 18:33:55.808254 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.800939    4888 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Oct 30 18:33:55.808254 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.800943    4888 flags.go:64] FLAG: --experimental-mounter-path=""
Oct 30 18:33:55.808254 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.800945    4888 flags.go:64] FLAG: --fail-cgroupv1="true"
Oct 30 18:33:55.808254 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.800949    4888 flags.go:64] FLAG: --fail-swap-on="true"
Oct 30 18:33:55.808254 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.800951    4888 flags.go:64] FLAG: --feature-gates=""
Oct 30 18:33:55.808254 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.800958    4888 flags.go:64] FLAG: --file-check-frequency="20s"
Oct 30 18:33:55.808254 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.800961    4888 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Oct 30 18:33:55.808254 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.800964    4888 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Oct 30 18:33:55.808254 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.800967    4888 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Oct 30 18:33:55.808254 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.800972    4888 flags.go:64] FLAG: --healthz-port="10248"
Oct 30 18:33:55.808254 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.800975    4888 flags.go:64] FLAG: --help="false"
Oct 30 18:33:55.808254 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.800977    4888 flags.go:64] FLAG: --hostname-override="ip-10-0-134-46.us-west-1.compute.internal"
Oct 30 18:33:55.808254 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.800981    4888 flags.go:64] FLAG: --housekeeping-interval="10s"
Oct 30 18:33:55.808254 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.800984    4888 flags.go:64] FLAG: --http-check-frequency="20s"
Oct 30 18:33:55.808254 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.800986    4888 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Oct 30 18:33:55.808254 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.800990    4888 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Oct 30 18:33:55.808254 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.800994    4888 flags.go:64] FLAG: --image-gc-high-threshold="85"
Oct 30 18:33:55.808816 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.800997    4888 flags.go:64] FLAG: --image-gc-low-threshold="80"
Oct 30 18:33:55.808816 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.800999    4888 flags.go:64] FLAG: --image-service-endpoint=""
Oct 30 18:33:55.808816 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.801002    4888 flags.go:64] FLAG: --kernel-memcg-notification="false"
Oct 30 18:33:55.808816 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.801005    4888 flags.go:64] FLAG: --kube-api-burst="100"
Oct 30 18:33:55.808816 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.801008    4888 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Oct 30 18:33:55.808816 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.801018    4888 flags.go:64] FLAG: --kube-api-qps="50"
Oct 30 18:33:55.808816 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.801021    4888 flags.go:64] FLAG: --kube-reserved=""
Oct 30 18:33:55.808816 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.801024    4888 flags.go:64] FLAG: --kube-reserved-cgroup=""
Oct 30 18:33:55.808816 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.801027    4888 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Oct 30 18:33:55.808816 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.801030    4888 flags.go:64] FLAG: --kubelet-cgroups=""
Oct 30 18:33:55.808816 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.801033    4888 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Oct 30 18:33:55.808816 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.801036    4888 flags.go:64] FLAG: --lock-file=""
Oct 30 18:33:55.808816 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.801038    4888 flags.go:64] FLAG: --log-cadvisor-usage="false"
Oct 30 18:33:55.808816 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.801041    4888 flags.go:64] FLAG: --log-flush-frequency="5s"
Oct 30 18:33:55.808816 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.801044    4888 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Oct 30 18:33:55.808816 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.801050    4888 flags.go:64] FLAG: --log-json-split-stream="false"
Oct 30 18:33:55.808816 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.801053    4888 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Oct 30 18:33:55.808816 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.801061    4888 flags.go:64] FLAG: --log-text-split-stream="false"
Oct 30 18:33:55.808816 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.801065    4888 flags.go:64] FLAG: --logging-format="text"
Oct 30 18:33:55.808816 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.801067    4888 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Oct 30 18:33:55.808816 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.801071    4888 flags.go:64] FLAG: --make-iptables-util-chains="true"
Oct 30 18:33:55.808816 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.801073    4888 flags.go:64] FLAG: --manifest-url=""
Oct 30 18:33:55.808816 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.801076    4888 flags.go:64] FLAG: --manifest-url-header=""
Oct 30 18:33:55.808816 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.801080    4888 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Oct 30 18:33:55.808816 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.801084    4888 flags.go:64] FLAG: --max-open-files="1000000"
Oct 30 18:33:55.809438 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.801088    4888 flags.go:64] FLAG: --max-pods="110"
Oct 30 18:33:55.809438 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.801091    4888 flags.go:64] FLAG: --maximum-dead-containers="-1"
Oct 30 18:33:55.809438 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.801094    4888 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Oct 30 18:33:55.809438 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.801097    4888 flags.go:64] FLAG: --memory-manager-policy="None"
Oct 30 18:33:55.809438 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.801100    4888 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Oct 30 18:33:55.809438 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.801102    4888 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Oct 30 18:33:55.809438 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.801106    4888 flags.go:64] FLAG: --node-ip="0.0.0.0"
Oct 30 18:33:55.809438 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.801109    4888 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Oct 30 18:33:55.809438 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.801116    4888 flags.go:64] FLAG: --node-status-max-images="50"
Oct 30 18:33:55.809438 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.801119    4888 flags.go:64] FLAG: --node-status-update-frequency="10s"
Oct 30 18:33:55.809438 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.801122    4888 flags.go:64] FLAG: --oom-score-adj="-999"
Oct 30 18:33:55.809438 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.801125    4888 flags.go:64] FLAG: --pod-cidr=""
Oct 30 18:33:55.809438 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.801127    4888 flags.go:64] FLAG: --pod-manifest-path=""
Oct 30 18:33:55.809438 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.801130    4888 flags.go:64] FLAG: --pod-max-pids="-1"
Oct 30 18:33:55.809438 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.801133    4888 flags.go:64] FLAG: --pods-per-core="0"
Oct 30 18:33:55.809438 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.801136    4888 flags.go:64] FLAG: --port="10250"
Oct 30 18:33:55.809438 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.801139    4888 flags.go:64] FLAG: --protect-kernel-defaults="false"
Oct 30 18:33:55.809438 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.801142    4888 flags.go:64] FLAG: --provider-id="aws:///us-west-1a/i-0459d68fab3e5d86f"
Oct 30 18:33:55.809438 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.801145    4888 flags.go:64] FLAG: --qos-reserved=""
Oct 30 18:33:55.809438 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.801148    4888 flags.go:64] FLAG: --read-only-port="10255"
Oct 30 18:33:55.809438 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.801150    4888 flags.go:64] FLAG: --register-node="true"
Oct 30 18:33:55.809438 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.801153    4888 flags.go:64] FLAG: --register-with-taints=""
Oct 30 18:33:55.809438 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.801157    4888 flags.go:64] FLAG: --registry-burst="10"
Oct 30 18:33:55.809438 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.801159    4888 flags.go:64] FLAG: --registry-qps="5"
Oct 30 18:33:55.809438 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.801162    4888 flags.go:64] FLAG: --reserved-cpus=""
Oct 30 18:33:55.809438 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.801172    4888 flags.go:64] FLAG: --reserved-memory=""
Oct 30 18:33:55.810102 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.801177    4888 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Oct 30 18:33:55.810102 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.801180    4888 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Oct 30 18:33:55.810102 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.801183    4888 flags.go:64] FLAG: --rotate-certificates="false"
Oct 30 18:33:55.810102 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.801185    4888 flags.go:64] FLAG: --rotate-server-certificates="false"
Oct 30 18:33:55.810102 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.801188    4888 flags.go:64] FLAG: --runonce="false"
Oct 30 18:33:55.810102 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.801192    4888 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Oct 30 18:33:55.810102 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.801195    4888 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Oct 30 18:33:55.810102 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.801199    4888 flags.go:64] FLAG: --seccomp-default="false"
Oct 30 18:33:55.810102 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.801201    4888 flags.go:64] FLAG: --serialize-image-pulls="true"
Oct 30 18:33:55.810102 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.801204    4888 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Oct 30 18:33:55.810102 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.801207    4888 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Oct 30 18:33:55.810102 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.801210    4888 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Oct 30 18:33:55.810102 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.801214    4888 flags.go:64] FLAG: --storage-driver-password="root"
Oct 30 18:33:55.810102 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.801217    4888 flags.go:64] FLAG: --storage-driver-secure="false"
Oct 30 18:33:55.810102 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.801220    4888 flags.go:64] FLAG: --storage-driver-table="stats"
Oct 30 18:33:55.810102 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.801222    4888 flags.go:64] FLAG: --storage-driver-user="root"
Oct 30 18:33:55.810102 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.801225    4888 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Oct 30 18:33:55.810102 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.801228    4888 flags.go:64] FLAG: --sync-frequency="1m0s"
Oct 30 18:33:55.810102 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.801231    4888 flags.go:64] FLAG: --system-cgroups=""
Oct 30 18:33:55.810102 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.801234    4888 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Oct 30 18:33:55.810102 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.801239    4888 flags.go:64] FLAG: --system-reserved-cgroup=""
Oct 30 18:33:55.810102 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.801242    4888 flags.go:64] FLAG: --tls-cert-file=""
Oct 30 18:33:55.810102 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.801245    4888 flags.go:64] FLAG: --tls-cipher-suites="[]"
Oct 30 18:33:55.810102 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.801252    4888 flags.go:64] FLAG: --tls-min-version=""
Oct 30 18:33:55.810102 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.801255    4888 flags.go:64] FLAG: --tls-private-key-file=""
Oct 30 18:33:55.810725 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.801258    4888 flags.go:64] FLAG: --topology-manager-policy="none"
Oct 30 18:33:55.810725 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.801261    4888 flags.go:64] FLAG: --topology-manager-policy-options=""
Oct 30 18:33:55.810725 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.801263    4888 flags.go:64] FLAG: --topology-manager-scope="container"
Oct 30 18:33:55.810725 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.801266    4888 flags.go:64] FLAG: --v="2"
Oct 30 18:33:55.810725 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.801270    4888 flags.go:64] FLAG: --version="false"
Oct 30 18:33:55.810725 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.801274    4888 flags.go:64] FLAG: --vmodule=""
Oct 30 18:33:55.810725 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.801293    4888 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Oct 30 18:33:55.810725 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.801303    4888 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Oct 30 18:33:55.810725 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801477    4888 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 18:33:55.810725 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801481    4888 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 18:33:55.810725 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801484    4888 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 18:33:55.810725 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801487    4888 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 18:33:55.810725 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801490    4888 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 18:33:55.810725 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801493    4888 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 18:33:55.810725 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801495    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 18:33:55.810725 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801498    4888 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 18:33:55.810725 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801500    4888 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 18:33:55.810725 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801503    4888 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 18:33:55.810725 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801505    4888 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 18:33:55.810725 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801508    4888 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 18:33:55.810725 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801511    4888 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 18:33:55.811224 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801513    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 18:33:55.811224 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801516    4888 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 18:33:55.811224 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801518    4888 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 18:33:55.811224 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801521    4888 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 18:33:55.811224 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801523    4888 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 18:33:55.811224 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801526    4888 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 18:33:55.811224 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801528    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 18:33:55.811224 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801531    4888 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 18:33:55.811224 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801534    4888 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 18:33:55.811224 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801536    4888 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 18:33:55.811224 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801539    4888 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 18:33:55.811224 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801541    4888 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 18:33:55.811224 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801543    4888 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 18:33:55.811224 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801546    4888 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 18:33:55.811224 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801548    4888 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 18:33:55.811224 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801551    4888 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 18:33:55.811224 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801553    4888 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 18:33:55.811224 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801556    4888 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 18:33:55.811224 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801558    4888 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 18:33:55.811800 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801568    4888 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 18:33:55.811800 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801571    4888 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 18:33:55.811800 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801574    4888 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 18:33:55.811800 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801576    4888 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 18:33:55.811800 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801579    4888 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 18:33:55.811800 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801581    4888 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 18:33:55.811800 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801584    4888 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 18:33:55.811800 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801587    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 18:33:55.811800 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801590    4888 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 18:33:55.811800 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801593    4888 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 18:33:55.811800 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801595    4888 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 18:33:55.811800 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801598    4888 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 18:33:55.811800 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801600    4888 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 18:33:55.811800 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801603    4888 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 18:33:55.811800 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801605    4888 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 18:33:55.811800 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801607    4888 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 18:33:55.811800 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801610    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 18:33:55.811800 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801612    4888 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 18:33:55.811800 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801615    4888 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 18:33:55.811800 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801617    4888 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 18:33:55.812445 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801619    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 18:33:55.812445 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801622    4888 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 18:33:55.812445 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801624    4888 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 18:33:55.812445 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801627    4888 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 18:33:55.812445 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801629    4888 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 18:33:55.812445 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801632    4888 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 18:33:55.812445 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801634    4888 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 18:33:55.812445 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801637    4888 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 18:33:55.812445 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801639    4888 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 18:33:55.812445 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801641    4888 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 18:33:55.812445 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801644    4888 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 18:33:55.812445 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801646    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 18:33:55.812445 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801650    4888 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 18:33:55.812445 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801660    4888 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 18:33:55.812445 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801663    4888 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 18:33:55.812445 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801666    4888 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 18:33:55.812445 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801668    4888 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 18:33:55.812445 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801671    4888 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 18:33:55.812445 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801673    4888 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 18:33:55.812973 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801677    4888 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 18:33:55.812973 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801681    4888 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 18:33:55.812973 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801684    4888 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 18:33:55.812973 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801687    4888 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 18:33:55.812973 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801689    4888 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 18:33:55.812973 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801692    4888 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 18:33:55.812973 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801694    4888 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 18:33:55.812973 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801696    4888 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 18:33:55.812973 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801699    4888 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 18:33:55.812973 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801701    4888 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 18:33:55.812973 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801704    4888 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 18:33:55.812973 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801706    4888 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 18:33:55.812973 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801708    4888 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 18:33:55.812973 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801711    4888 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 18:33:55.812973 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801713    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 18:33:55.812973 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801716    4888 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 18:33:55.812973 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801718    4888 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 18:33:55.812973 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801721    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 18:33:55.812973 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801723    4888 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 18:33:55.813479 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801725    4888 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 18:33:55.813479 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801728    4888 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 18:33:55.813479 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801730    4888 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 18:33:55.813479 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801733    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 18:33:55.813479 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801735    4888 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 18:33:55.813479 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801738    4888 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 18:33:55.813479 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801740    4888 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 18:33:55.813479 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801742    4888 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 18:33:55.813479 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801748    4888 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 18:33:55.813479 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801750    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 18:33:55.813479 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801753    4888 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 18:33:55.813479 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801755    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 18:33:55.813479 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801758    4888 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 18:33:55.813479 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801760    4888 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 18:33:55.813479 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801763    4888 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 18:33:55.813479 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801765    4888 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 18:33:55.813863 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.801793    4888 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 18:33:55.813863 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801846    4888 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 18:33:55.813863 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801850    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 18:33:55.813863 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801852    4888 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 18:33:55.813863 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801855    4888 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 18:33:55.813863 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801858    4888 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 18:33:55.813863 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801860    4888 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 18:33:55.813863 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801863    4888 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 18:33:55.813863 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801865    4888 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 18:33:55.813863 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801868    4888 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 18:33:55.813863 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801870    4888 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 18:33:55.813863 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801873    4888 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 18:33:55.813863 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801875    4888 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 18:33:55.813863 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801878    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 18:33:55.813863 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801882    4888 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 18:33:55.813863 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801884    4888 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 18:33:55.814316 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801887    4888 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 18:33:55.814316 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801889    4888 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 18:33:55.814316 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801892    4888 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 18:33:55.814316 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801894    4888 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 18:33:55.814316 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801896    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 18:33:55.814316 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801899    4888 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 18:33:55.814316 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801901    4888 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 18:33:55.814316 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801904    4888 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 18:33:55.814316 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801906    4888 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 18:33:55.814316 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801910    4888 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 18:33:55.814316 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801913    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 18:33:55.814316 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801915    4888 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 18:33:55.814316 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801918    4888 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 18:33:55.814316 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801920    4888 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 18:33:55.814316 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801923    4888 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 18:33:55.814316 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801925    4888 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 18:33:55.814316 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801928    4888 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 18:33:55.814316 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801930    4888 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 18:33:55.814316 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801933    4888 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 18:33:55.814874 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801936    4888 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 18:33:55.814874 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801938    4888 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 18:33:55.814874 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801941    4888 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 18:33:55.814874 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801943    4888 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 18:33:55.814874 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801946    4888 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 18:33:55.814874 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801948    4888 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 18:33:55.814874 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801951    4888 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 18:33:55.814874 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801953    4888 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 18:33:55.814874 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801956    4888 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 18:33:55.814874 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801958    4888 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 18:33:55.814874 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801961    4888 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 18:33:55.814874 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801965    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 18:33:55.814874 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801968    4888 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 18:33:55.814874 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801970    4888 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 18:33:55.814874 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801973    4888 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 18:33:55.814874 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801976    4888 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 18:33:55.814874 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801978    4888 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 18:33:55.814874 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801980    4888 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 18:33:55.814874 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801983    4888 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 18:33:55.814874 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801985    4888 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 18:33:55.815419 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801988    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 18:33:55.815419 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801990    4888 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 18:33:55.815419 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801994    4888 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 18:33:55.815419 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801997    4888 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 18:33:55.815419 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.801999    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 18:33:55.815419 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.802002    4888 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 18:33:55.815419 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.802004    4888 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 18:33:55.815419 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.802006    4888 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 18:33:55.815419 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.802009    4888 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 18:33:55.815419 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.802011    4888 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 18:33:55.815419 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.802014    4888 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 18:33:55.815419 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.802016    4888 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 18:33:55.815419 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.802019    4888 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 18:33:55.815419 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.802021    4888 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 18:33:55.815419 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.802023    4888 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 18:33:55.815419 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.802026    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 18:33:55.815419 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.802031    4888 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 18:33:55.815419 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.802035    4888 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 18:33:55.815419 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.802038    4888 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 18:33:55.815878 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.802041    4888 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 18:33:55.815878 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.802043    4888 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 18:33:55.815878 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.802046    4888 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 18:33:55.815878 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.802048    4888 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 18:33:55.815878 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.802055    4888 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 18:33:55.815878 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.802058    4888 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 18:33:55.815878 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.802061    4888 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 18:33:55.815878 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.802064    4888 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 18:33:55.815878 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.802066    4888 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 18:33:55.815878 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.802069    4888 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 18:33:55.815878 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.802071    4888 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 18:33:55.815878 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.802074    4888 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 18:33:55.815878 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.802076    4888 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 18:33:55.815878 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.802079    4888 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 18:33:55.815878 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.802081    4888 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 18:33:55.815878 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.802085    4888 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 18:33:55.815878 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.802087    4888 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 18:33:55.815878 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.802090    4888 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 18:33:55.815878 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.802093    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 18:33:55.815878 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.802095    4888 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 18:33:55.816379 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.802098    4888 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 18:33:55.816379 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.802100    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 18:33:55.816379 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.802103    4888 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 18:33:55.816379 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.802105    4888 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 18:33:55.816379 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.802108    4888 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 18:33:55.816379 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.802110    4888 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 18:33:55.816379 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.802113    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 18:33:55.816379 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.802115    4888 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 18:33:55.816379 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.802119    4888 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 18:33:55.816379 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.802121    4888 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 18:33:55.816379 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.802124    4888 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 18:33:55.816379 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.802126    4888 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 18:33:55.816379 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.802129    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 18:33:55.816379 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.807253    4888 server.go:525] "Kubelet version" kubeletVersion="v1.35.9"
Oct 30 18:33:55.816379 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.807268    4888 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Oct 30 18:33:55.816379 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.807311    4888 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Oct 30 18:33:55.816379 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.807317    4888 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Oct 30 18:33:55.816379 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807376    4888 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 18:33:55.816379 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807381    4888 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 18:33:55.816823 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807385    4888 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 18:33:55.816823 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807390    4888 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 18:33:55.816823 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807393    4888 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 18:33:55.816823 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807396    4888 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 18:33:55.816823 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807399    4888 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 18:33:55.816823 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807402    4888 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 18:33:55.816823 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807405    4888 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 18:33:55.816823 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807408    4888 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 18:33:55.816823 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807411    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 18:33:55.816823 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807414    4888 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 18:33:55.816823 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807416    4888 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 18:33:55.816823 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807419    4888 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 18:33:55.816823 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807421    4888 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 18:33:55.816823 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807424    4888 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 18:33:55.816823 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807426    4888 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 18:33:55.816823 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807429    4888 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 18:33:55.816823 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807431    4888 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 18:33:55.816823 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807434    4888 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 18:33:55.816823 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807436    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 18:33:55.816823 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807438    4888 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 18:33:55.817384 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807441    4888 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 18:33:55.817384 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807443    4888 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 18:33:55.817384 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807446    4888 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 18:33:55.817384 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807448    4888 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 18:33:55.817384 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807451    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 18:33:55.817384 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807454    4888 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 18:33:55.817384 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807456    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 18:33:55.817384 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807459    4888 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 18:33:55.817384 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807461    4888 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 18:33:55.817384 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807464    4888 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 18:33:55.817384 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807467    4888 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 18:33:55.817384 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807469    4888 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 18:33:55.817384 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807472    4888 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 18:33:55.817384 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807474    4888 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 18:33:55.817384 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807477    4888 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 18:33:55.817384 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807479    4888 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 18:33:55.817384 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807482    4888 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 18:33:55.817384 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807485    4888 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 18:33:55.817384 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807487    4888 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 18:33:55.817833 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807490    4888 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 18:33:55.817833 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807493    4888 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 18:33:55.817833 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807495    4888 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 18:33:55.817833 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807498    4888 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 18:33:55.817833 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807500    4888 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 18:33:55.817833 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807509    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 18:33:55.817833 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807512    4888 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 18:33:55.817833 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807514    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 18:33:55.817833 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807517    4888 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 18:33:55.817833 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807519    4888 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 18:33:55.817833 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807522    4888 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 18:33:55.817833 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807524    4888 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 18:33:55.817833 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807527    4888 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 18:33:55.817833 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807529    4888 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 18:33:55.817833 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807531    4888 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 18:33:55.817833 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807534    4888 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 18:33:55.817833 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807536    4888 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 18:33:55.817833 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807539    4888 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 18:33:55.817833 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807541    4888 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 18:33:55.817833 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807544    4888 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 18:33:55.818358 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807546    4888 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 18:33:55.818358 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807549    4888 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 18:33:55.818358 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807551    4888 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 18:33:55.818358 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807554    4888 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 18:33:55.818358 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807556    4888 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 18:33:55.818358 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807559    4888 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 18:33:55.818358 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807562    4888 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 18:33:55.818358 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807564    4888 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 18:33:55.818358 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807567    4888 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 18:33:55.818358 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807569    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 18:33:55.818358 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807573    4888 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 18:33:55.818358 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807577    4888 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 18:33:55.818358 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807580    4888 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 18:33:55.818358 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807583    4888 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 18:33:55.818358 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807585    4888 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 18:33:55.818358 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807588    4888 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 18:33:55.818358 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807590    4888 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 18:33:55.818358 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807592    4888 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 18:33:55.818358 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807600    4888 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 18:33:55.818835 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807603    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 18:33:55.818835 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807606    4888 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 18:33:55.818835 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807608    4888 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 18:33:55.818835 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807611    4888 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 18:33:55.818835 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807613    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 18:33:55.818835 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807615    4888 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 18:33:55.818835 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807618    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 18:33:55.818835 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807621    4888 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 18:33:55.818835 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807623    4888 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 18:33:55.818835 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807625    4888 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 18:33:55.818835 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807628    4888 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 18:33:55.818835 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807630    4888 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 18:33:55.818835 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807632    4888 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 18:33:55.818835 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807635    4888 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 18:33:55.818835 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807637    4888 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 18:33:55.818835 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807640    4888 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 18:33:55.818835 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807642    4888 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 18:33:55.818835 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807645    4888 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 18:33:55.818835 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807647    4888 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 18:33:55.819296 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807650    4888 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 18:33:55.819296 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807652    4888 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 18:33:55.819296 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807655    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 18:33:55.819296 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807658    4888 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 18:33:55.819296 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807660    4888 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 18:33:55.819296 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807663    4888 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 18:33:55.819296 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807665    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 18:33:55.819296 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.807692    4888 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 18:33:55.819296 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807822    4888 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 18:33:55.819296 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807826    4888 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 18:33:55.819296 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807829    4888 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 18:33:55.819296 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807831    4888 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 18:33:55.819296 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807834    4888 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 18:33:55.819296 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807837    4888 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 18:33:55.819296 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807841    4888 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 18:33:55.819296 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807845    4888 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 18:33:55.819688 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807848    4888 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 18:33:55.819688 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807851    4888 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 18:33:55.819688 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807854    4888 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 18:33:55.819688 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807856    4888 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 18:33:55.819688 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807859    4888 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 18:33:55.819688 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807861    4888 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 18:33:55.819688 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807863    4888 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 18:33:55.819688 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807866    4888 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 18:33:55.819688 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807869    4888 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 18:33:55.819688 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807871    4888 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 18:33:55.819688 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807873    4888 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 18:33:55.819688 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807876    4888 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 18:33:55.819688 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807880    4888 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 18:33:55.819688 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807883    4888 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 18:33:55.819688 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807886    4888 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 18:33:55.819688 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807888    4888 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 18:33:55.819688 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807890    4888 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 18:33:55.819688 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807893    4888 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 18:33:55.819688 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807895    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 18:33:55.819688 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807898    4888 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 18:33:55.820158 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807900    4888 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 18:33:55.820158 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807903    4888 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 18:33:55.820158 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807905    4888 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 18:33:55.820158 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807908    4888 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 18:33:55.820158 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807910    4888 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 18:33:55.820158 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807913    4888 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 18:33:55.820158 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807915    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 18:33:55.820158 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807918    4888 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 18:33:55.820158 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807920    4888 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 18:33:55.820158 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807923    4888 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 18:33:55.820158 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807926    4888 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 18:33:55.820158 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807928    4888 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 18:33:55.820158 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807930    4888 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 18:33:55.820158 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807933    4888 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 18:33:55.820158 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807936    4888 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 18:33:55.820158 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807940    4888 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 18:33:55.820158 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807942    4888 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 18:33:55.820158 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807945    4888 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 18:33:55.820158 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807948    4888 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 18:33:55.820634 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807950    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 18:33:55.820634 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807953    4888 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 18:33:55.820634 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807955    4888 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 18:33:55.820634 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807957    4888 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 18:33:55.820634 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807960    4888 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 18:33:55.820634 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807963    4888 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 18:33:55.820634 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807966    4888 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 18:33:55.820634 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807968    4888 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 18:33:55.820634 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807971    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 18:33:55.820634 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807973    4888 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 18:33:55.820634 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807976    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 18:33:55.820634 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807978    4888 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 18:33:55.820634 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807980    4888 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 18:33:55.820634 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807983    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 18:33:55.820634 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807985    4888 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 18:33:55.820634 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807987    4888 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 18:33:55.820634 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807990    4888 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 18:33:55.820634 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807992    4888 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 18:33:55.820634 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807995    4888 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 18:33:55.820634 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.807997    4888 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 18:33:55.821100 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.808000    4888 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 18:33:55.821100 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.808002    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 18:33:55.821100 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.808005    4888 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 18:33:55.821100 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.808007    4888 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 18:33:55.821100 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.808010    4888 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 18:33:55.821100 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.808013    4888 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 18:33:55.821100 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.808015    4888 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 18:33:55.821100 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.808018    4888 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 18:33:55.821100 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.808020    4888 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 18:33:55.821100 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.808022    4888 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 18:33:55.821100 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.808025    4888 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 18:33:55.821100 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.808027    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 18:33:55.821100 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.808030    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 18:33:55.821100 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.808032    4888 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 18:33:55.821100 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.808035    4888 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 18:33:55.821100 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.808037    4888 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 18:33:55.821100 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.808039    4888 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 18:33:55.821100 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.808043    4888 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 18:33:55.821100 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.808045    4888 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 18:33:55.821100 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.808047    4888 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 18:33:55.821779 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.808050    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 18:33:55.821779 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.808052    4888 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 18:33:55.821779 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.808054    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 18:33:55.821779 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.808057    4888 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 18:33:55.821779 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.808059    4888 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 18:33:55.821779 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.808062    4888 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 18:33:55.821779 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.808064    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 18:33:55.821779 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.808067    4888 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 18:33:55.821779 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.808069    4888 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 18:33:55.821779 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.808072    4888 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 18:33:55.821779 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.808074    4888 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 18:33:55.821779 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.808077    4888 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 18:33:55.821779 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.808079    4888 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 18:33:55.821779 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.808081    4888 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 18:33:55.821779 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.808084    4888 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 18:33:55.821779 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.808086    4888 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 18:33:55.821779 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.808089    4888 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 18:33:55.821779 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.808092    4888 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 18:33:55.821779 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.808094    4888 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 18:33:55.822228 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.808117    4888 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 18:33:55.822228 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.808147    4888 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 18:33:55.822228 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.808151    4888 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 18:33:55.822228 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.808154    4888 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 18:33:55.822228 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.808156    4888 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 18:33:55.822228 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.808159    4888 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 18:33:55.822228 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.808161    4888 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 18:33:55.822228 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.808164    4888 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 18:33:55.822228 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.808167    4888 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 18:33:55.822228 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.808170    4888 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 18:33:55.822228 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.808174    4888 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 18:33:55.822228 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.808177    4888 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 18:33:55.822228 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.808180    4888 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 18:33:55.822228 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.808182    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 18:33:55.822228 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.808185    4888 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 18:33:55.822228 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.808187    4888 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 18:33:55.822675 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.808190    4888 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 18:33:55.822675 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.808192    4888 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 18:33:55.822675 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.808195    4888 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 18:33:55.822675 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.808197    4888 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 18:33:55.822675 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.808200    4888 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 18:33:55.822675 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.808202    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 18:33:55.822675 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.808205    4888 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 18:33:55.822675 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.808207    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 18:33:55.822675 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.808210    4888 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 18:33:55.822675 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.808212    4888 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 18:33:55.822675 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.808215    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 18:33:55.822675 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.808218    4888 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 18:33:55.822675 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.808220    4888 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 18:33:55.822675 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.808222    4888 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 18:33:55.822675 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.808225    4888 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 18:33:55.822675 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.808229    4888 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 18:33:55.822675 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.808232    4888 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 18:33:55.822675 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.808234    4888 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 18:33:55.822675 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.808237    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 18:33:55.822675 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.808239    4888 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 18:33:55.823154 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.808242    4888 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 18:33:55.823154 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.808244    4888 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 18:33:55.823154 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.808247    4888 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 18:33:55.823154 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.808249    4888 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 18:33:55.823154 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.808252    4888 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 18:33:55.823154 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.808254    4888 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 18:33:55.823154 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.808258    4888 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 18:33:55.823154 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.808261    4888 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 18:33:55.823154 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.808264    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 18:33:55.823154 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.808266    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 18:33:55.823154 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.808269    4888 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 18:33:55.823154 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.808272    4888 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 18:33:55.823154 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.808274    4888 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 18:33:55.823154 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.808277    4888 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 18:33:55.823154 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.808293    4888 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 18:33:55.823154 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.808296    4888 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 18:33:55.823154 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.808298    4888 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 18:33:55.823154 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.808301    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 18:33:55.823154 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.808303    4888 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 18:33:55.823154 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.808306    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 18:33:55.823711 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.808309    4888 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 18:33:55.823711 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.808311    4888 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 18:33:55.823711 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.808314    4888 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 18:33:55.823711 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.808316    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 18:33:55.823711 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.808319    4888 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 18:33:55.823711 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.808322    4888 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 18:33:55.823711 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.808324    4888 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 18:33:55.823711 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.808328    4888 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 18:33:55.823711 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.808331    4888 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 18:33:55.823711 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.808334    4888 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 18:33:55.823711 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.808336    4888 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 18:33:55.823711 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.808339    4888 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 18:33:55.823711 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.808342    4888 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 18:33:55.823711 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.808344    4888 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 18:33:55.823711 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.808347    4888 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 18:33:55.823711 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.808350    4888 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 18:33:55.823711 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.808352    4888 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 18:33:55.823711 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.808355    4888 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 18:33:55.823711 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.808358    4888 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 18:33:55.824518 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.808361    4888 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 18:33:55.824518 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.808363    4888 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 18:33:55.824518 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.808366    4888 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 18:33:55.824518 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.808370    4888 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 18:33:55.824518 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.808374    4888 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 18:33:55.824518 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.808377    4888 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 18:33:55.824518 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.808379    4888 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 18:33:55.824518 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.808382    4888 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 18:33:55.824518 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.808385    4888 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 18:33:55.824518 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.808387    4888 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 18:33:55.824518 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.808389    4888 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 18:33:55.824518 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.808392    4888 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 18:33:55.824518 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.808395    4888 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 18:33:55.824518 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.808397    4888 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 18:33:55.824518 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.808400    4888 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 18:33:55.824518 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.808402    4888 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 18:33:55.824518 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.808405    4888 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 18:33:55.824518 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.808407    4888 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 18:33:55.824518 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.808410    4888 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 18:33:55.825103 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.808412    4888 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 18:33:55.825103 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.808416    4888 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 18:33:55.825103 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.808418    4888 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 18:33:55.825103 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.808422    4888 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 18:33:55.825103 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.808424    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 18:33:55.825103 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.808427    4888 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 18:33:55.825103 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.808429    4888 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 18:33:55.825103 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.808432    4888 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 18:33:55.825103 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.808435    4888 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 18:33:55.825103 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.808437    4888 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 18:33:55.825103 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.808440    4888 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 18:33:55.825103 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.808442    4888 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 18:33:55.825103 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:33:55.808444    4888 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 18:33:55.825103 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.808496    4888 server.go:957] "Client rotation is on, will bootstrap in background"
Oct 30 18:33:55.825103 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.809420    4888 bootstrap.go:85] "Current kubeconfig file contents are still valid, no bootstrap necessary"
Oct 30 18:33:55.825103 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.809466    4888 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-client-current.pem"
Oct 30 18:33:55.825103 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.809632    4888 server.go:1014] "Starting client certificate rotation"
Oct 30 18:33:55.825103 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.809789    4888 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Oct 30 18:33:55.825574 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.809835    4888 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-10-29 18:28:14 +0000 UTC" deadline="2028-04-02 19:52:42.113661888 +0000 UTC"
Oct 30 18:33:55.825574 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.809885    4888 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12481h18m46.303778865s"
Oct 30 18:33:55.825574 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.810067    4888 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 18:33:55.825574 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.811164    4888 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 18:33:55.825574 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.812433    4888 log.go:25] "Validated CRI v1 runtime API"
Oct 30 18:33:55.825574 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.814084    4888 log.go:25] "Validated CRI v1 image API"
Oct 30 18:33:55.825574 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.814850    4888 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Oct 30 18:33:55.825574 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.816684    4888 fs.go:135] Filesystem UUIDs: map[2c0c17c3-049f-444e-8104-bc73b9e8fb70:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 9f3017e7-a4ab-4bd0-9554-505960a37d8e:/dev/nvme0n1p3]
Oct 30 18:33:55.825792 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.816698    4888 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/010820c82deec18c7b4002e46b65e8dab8cc3c9c8cd27ac759462efc6c4c32ac/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/010820c82deec18c7b4002e46b65e8dab8cc3c9c8cd27ac759462efc6c4c32ac/userdata/shm major:0 minor:83 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/02e1780cae1ffe8664994cc74406d074948af8b6ea7d829904b28cc97b45939f/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/02e1780cae1ffe8664994cc74406d074948af8b6ea7d829904b28cc97b45939f/userdata/shm major:0 minor:244 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/0698cafab13d74adaa3fd8179e2b7ec62ff9ce956db4148cafedadbb6d4d553c/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/0698cafab13d74adaa3fd8179e2b7ec62ff9ce956db4148cafedadbb6d4d553c/userdata/shm major:0 minor:82 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/110c3deb893a6eff9717f921ccf7dc3f3e56cf7408a56a85d1d6b3e8a95633a4/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/110c3deb893a6eff9717f921ccf7dc3f3e56cf7408a56a85d1d6b3e8a95633a4/userdata/shm major:0 minor:246 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/1d584626e578f70bd2bd96914c22b5060fe64a2954330b204e9b4c0683253984/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/1d584626e578f70bd2bd96914c22b5060fe64a2954330b204e9b4c0683253984/userdata/shm major:0 minor:210 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/1fb8715703bb95f8a132c554ff1169dc975c64de24b0bc642f7bf41a12efe875/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/1fb8715703bb95f8a132c554ff1169dc975c64de24b0bc642f7bf41a12efe875/userdata/shm major:0 minor:234 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/264171f78a701c6a4af300c38636f113df3b276804fb65397c6d59c290a3b89f/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/264171f78a701c6a4af300c38636f113df3b276804fb65397c6d59c290a3b89f/userdata/shm major:0 minor:231 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/3dacffc196bf6205ffae4eaaf96fc7405b8753485def2dab566ce25b8106f9ae/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/3dacffc196bf6205ffae4eaaf96fc7405b8753485def2dab566ce25b8106f9ae/userdata/shm major:0 minor:236 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/41306bd73702bf41bacd24f69e9da18eaa60d7a3d5c5782b2c004af7846ac805/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/41306bd73702bf41bacd24f69e9da18eaa60d7a3d5c5782b2c004af7846ac805/userdata/shm major:0 minor:290 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/41566f398fb6e7ad1be747e3b375979733d9965f00edb0dd1f8a0328234a824c/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/41566f398fb6e7ad1be747e3b375979733d9965f00edb0dd1f8a0328234a824c/userdata/shm major:0 minor:242 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/83e431df5ea6349b7b60e71173b402e4ef22fd518c8f8b81c8eccd92d1179b40/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/83e431df5ea6349b7b60e71173b402e4ef22fd518c8f8b81c8eccd92d1179b40/userdata/shm major:0 minor:340 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/8f3fb203f2d1c8bf14c057244ee2331e339db7c0ab0f9a744f96acb79a075483/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/8f3fb203f2d1c8bf14c057244ee2331e339db7c0ab0f9a744f96acb79a075483/userdata/shm major:0 minor:90 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/93eecdc0633397bbec713dfdc4aff8aeb6399c863290c63bbd8a0da3fb70b0fc/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/93eecdc0633397bbec713dfdc4aff8aeb6399c863290c63bbd8a0da3fb70b0fc/userdata/shm major:0 minor:230 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/94eb3352db48f6acf5fa74ee0bcc8ac26482c3360cb4db2fabe3ff668bd86c13/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/94eb3352db48f6acf5fa74ee0bcc8ac26482c3360cb4db2fabe3ff668bd86c13/userdata/shm major:0 minor:47 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/9766c5bb2416907e5b0313e177e5075e6f6480fc8b492fcfa2d287d15133f64e/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/9766c5bb2416907e5b0313e177e5075e6f6480fc8b492fcfa2d287d15133f64e/userdata/shm major:0 minor:68 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/a0d10b0331467cc7d179bb0e0cb083d789035c89166c8c5f594f142864c45f4c/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/a0d10b0331467cc7d179bb0e0cb083d789035c89166c8c5f594f142864c45f4c/userdata/shm major:0 minor:72 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/a6818d98d464d242de00bec5b3468e251eaafeecc0329e9b18c563db7d7b4ba8/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/a6818d98d464d242de00bec5b3468e251eaafeecc0329e9b18c563db7d7b4ba8/userdata/shm major:0 minor:78 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/add4754d17d14d035566b3612bce0fd00489041c05d84daf0553c4e983484ba6/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/add4754d17d14d035566b3612bce0fd00489041c05d84daf0553c4e983484ba6/userdata/shm major:0 minor:94 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/b3f8d0948c4c12d4822539070451f179a0b9a930adb6cdfd15c3c8b13a7f4395/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/b3f8d0948c4c12d4822539070451f179a0b9a930adb6cdfd15c3c8b13a7f4395/userdata/shm major:0 minor:238 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/e1430cbb57146c9fef763300e771803f7b3cb39a5af4e3bbf5c2b1ea173d02d7/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/e1430cbb57146c9fef763300e771803f7b3cb39a5af4e3bbf5c2b1ea173d02d7/userdata/shm major:0 minor:97 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/ecaf79502997d6f1f864f0f6e6427641cd2d8cc464f7f8ec8279c05e8bbc14b3/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/ecaf79502997d6f1f864f0f6e6427641cd2d8cc464f7f8ec8279c05e8bbc14b3/userdata/shm major:0 minor:51 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/ecf8942e95e312a77dfd631bc979e0c4fb823eb197c2536e76ed89c513f0cd6e/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/ecf8942e95e312a77dfd631bc979e0c4fb823eb197c2536e76ed89c513f0cd6e/userdata/shm major:0 minor:227 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/f5a68d540ea6105c69282b72fcef5f4df6de3fef4565d6e65827425d667c1633/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/f5a68d540ea6105c69282b72fcef5f4df6de3fef4565d6e65827425d667c1633/userdata/shm major:0 minor:287 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/09c13759-50d4-40e6-b854-4691822b8c8a/volumes/kubernetes.io~projected/kube-api-access-k5crl:{mountpoint:/var/lib/kubelet/pods/09c13759-50d4-40e6-b854-4691822b8c8a/volumes/kubernetes.io~projected/kube-api-access-k5crl major:0 minor:221 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/09c13759-50d4-40e6-b854-4691822b8c8a/volumes/kubernetes.io~secret/serving-cert:{mountpoint:/var/lib/kubelet/pods/09c13759-50d4-40e6-b854-4691822b8c8a/volumes/kubernetes.io~secret/serving-cert major:0 minor:207 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/15d485f9-5a4b-4f4b-9e59-a6eb062b4c5c/volumes/kubernetes.io~projected/kube-api-access-fzrkt:{mountpoint:/var/lib/kubelet/pods/15d485f9-5a4b-4f4b-9e59-a6eb062b4c5c/volumes/kubernetes.io~projected/kube-api-access-fzrkt major:0 minor:220 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/2e49faaa-a718-44c8-9237-5a41169faee7/volumes/kubernetes.io~projected/kube-api-access-6llht:{mountpoint:/var/lib/kubelet/pods/2e49faaa-a718-44c8-9237-5a41169faee7/volumes/kubernetes.io~projected/kube-api-access-6llht major:0 minor:61 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/36a04851-81db-40cd-97ef-8e2e00a08604/volumes/kubernetes.io~projected/kube-api-access-rw8g9:{mountpoint:/var/lib/kubelet/pods/36a04851-81db-40cd-97ef-8e2e00a08604/volumes/kubernetes.io~projected/kube-api-access-rw8g9 major:0 minor:225 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/36a04851-81db-40cd-97ef-8e2e00a08604/volumes/kubernetes.io~secret/serving-cert:{mountpoint:/var/lib/kubelet/pods/36a04851-81db-40cd-97ef-8e2e00a08604/volumes/kubernetes.io~secret/serving-cert major:0 minor:205 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/382e8b31-a7fa-43f3-9f15-4c61627b0c04/volumes/kubernetes.io~projected/kube-api-access-m4j2q:{mountpoint:/var/lib/kubelet/pods/382e8b31-a7fa-43f3-9f15-4c61627b0c04/volumes/kubernetes.io~projected/kube-api-access-m4j2q major:0 minor:219 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/382e8b31-a7fa-43f3-9f15-4c61627b0c04/volumes/kubernetes.io~secret/serving-cert:{mountpoint:/var/lib/kubelet/pods/382e8b31-a7fa-43f3-9f15-4c61627b0c04/volumes/kubernetes.io~secret/serving-cert major:0 minor:203 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3f3fe764-2652-46c9-8e54-7882c0e6bcf1/volume-subpaths/run-systemd/ovnkube-controller/6:{mountpoint:/var/lib/kubelet/pods/3f3fe764-2652-46c9-8e54-7882c0e6bcf1/volume-subpaths/run-systemd/ovnkube-controller/6 major:0 minor:26 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3f3fe764-2652-46c9-8e54-7882c0e6bcf1/volumes/kubernetes.io~projected/kube-api-access-wzxwr:{mountpoint:/var/lib/kubelet/pods/3f3fe764-2652-46c9-8e54-7882c0e6bcf1/volumes/kubernetes.io~projected/kube-api-access-wzxwr major:0 minor:64 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3f3fe764-2652-46c9-8e54-7882c0e6bcf1/volumes/kubernetes.io~secret/ovn-node-metrics-cert:{mountpoint:/var/lib/kubelet/pods/3f3fe764-2652-46c9-8e54-7882c0e6bcf1/volumes/kubernetes.io~secret/ovn-node-metrics-cert major:0 minor:58 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/46a80637-6ccf-4a56-9904-f5e2fe5784ad/volumes/kubernetes.io~projected/kube-api-access-x2978:{mountpoint:/var/lib/kubelet/pods/46a80637-6ccf-4a56-9904-f5e2fe5784ad/volumes/kubernetes.io~projected/kube-api-access-x2978 major:0 minor:59 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/46d9ac39-ee45-4e43-bda7-3931c3e599cf/volumes/kubernetes.io~secret/original-pull-secret:{mountpoint:/var/lib/kubelet/pods/46d9ac39-ee45-4e43-bda7-3931c3e599cf/volumes/kubernetes.io~secret/original-pull-secret major:0 minor:280 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/57ca8600-79e7-4f59-9d11-b1c9ec3d358a/volumes/kubernetes.io~projected/kube-api-access-wrmcg:{mountpoint:/var/lib/kubelet/pods/57ca8600-79e7-4f59-9d11-b1c9ec3d358a/volumes/kubernetes.io~projected/kube-api-access-wrmcg major:0 minor:312 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/613020c0-f3e4-4650-896f-748a40f0ab8b/volumes/kubernetes.io~projected/kube-api-access-c45qr:{mountpoint:/var/lib/kubelet/pods/613020c0-f3e4-4650-896f-748a40f0ab8b/volumes/kubernetes.io~projected/kube-api-access-c45qr major:0 minor:289 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/63090f48-9143-4118-9906-d759c0aff52c/volumes/kubernetes.io~projected/kube-api-access-bgxbg:{mountpoint:/var/lib/kubelet/pods/63090f48-9143-4118-9906-d759c0aff52c/volumes/kubernetes.io~projected/kube-api-access-bgxbg major:0 minor:63 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/68195e19-b1ff-4148-9fae-3545f2997ff5/volumes/kubernetes.io~projected/kube-api-access-4rztt:{mountpoint:/var/lib/kubelet/pods/68195e19-b1ff-4148-9fae-3545f2997ff5/volumes/kubernetes.io~projected/kube-api-access-4rztt major:0 minor:212 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7c09088b-3a07-426d-8804-f0f5bac4d27c/volumes/kubernetes.io~projected/kube-api-access-cvxhl:{mountpoint:/var/lib/kubelet/pods/7c09088b-3a07-426d-8804-f0f5bac4d27c/volumes/kubernetes.io~projected/kube-api-access-cvxhl major:0 minor:214 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7c09088b-3a07-426d-8804-f0f5bac4d27c/volumes/kubernetes.io~secret/serving-cert:{mountpoint:/var/lib/kubelet/pods/7c09088b-3a07-426d-8804-f0f5bac4d27c/volumes/kubernetes.io~secret/serving-cert major:0 minor:204 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/81de5dce-c40f-4bd5-878c-5d75a0487066/volumes/kubernetes.io~projected/kube-api-access-9j2qw:{mountpoint:/var/lib/kubelet/pods/81de5dce-c40f-4bd5-878c-5d75a0487066/volumes/kubernetes.io~projected/kube-api-access-9j2qw major:0 minor:222 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8789ec5b-226e-4551-bf68-3ac585006266/volumes/kubernetes.io~projected/bound-sa-token:{mountpoint:/var/lib/kubelet/pods/8789ec5b-226e-4551-bf68-3ac585006266/volumes/kubernetes.io~projected/bound-sa-token major:0 minor:226 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8789ec5b-226e-4551-bf68-3ac585006266/volumes/kubernetes.io~projected/kube-api-access-vf948:{mountpoint:/var/lib/kubelet/pods/8789ec5b-226e-4551-bf68-3ac585006266/volumes/kubernetes.io~projected/kube-api-access-vf948 major:0 minor:213 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8789ec5b-226e-4551-bf68-3ac585006266/volumes/kubernetes.io~secret/image-registry-private-configuration:{mountpoint:/var/lib/kubelet/pods/8789ec5b-226e-4551-bf68-3ac585006266/volumes/kubernetes.io~secret/image-registry-private-configuration major:0 minor:208 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8789ec5b-226e-4551-bf68-3ac585006266/volumes/kubernetes.io~secret/installation-pull-secrets:{mountpoint:/var/lib/kubelet/pods/8789ec5b-226e-4551-bf68-3ac585006266/volumes/kubernetes.io~secret/installation-pull-secrets major:0 minor:206 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8ecf2b62-8053-4bba-a852-c6f6119d3288/volumes/kubernetes.io~secret/agent-certs:{mountpoint:/var/lib/kubelet/pods/8ecf2b62-8053-4bba-a852-c6f6119d3288/volumes/kubernetes.io~secret/agent-certs major:0 minor:56 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/97b19aa3-42be-4def-8d7a-8e3516852e79/volumes/kubernetes.io~projected/kube-api-access-tc2fp:{mountpoint:/var/lib/kubelet/pods/97b19aa3-42be-4def-8d7a-8e3516852e79/volumes/kubernetes.io~projected/kube-api-access-tc2fp major:0 minor:241 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351/volumes/kubernetes.io~projected/bound-sa-token:{mountpoint:/var/lib/kubelet/pods/9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351/volumes/kubernetes.io~projected/bound-sa-token major:0 minor:216 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351/volumes/kubernetes.io~projected/kube-api-access-vxw47:{mountpoint:/var/lib/kubelet/pods/9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351/volumes/kubernetes.io~projected/kube-api-access-vxw47 major:0 minor:224 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351/volumes/kubernetes.io~secret/image-registry-private-configuration:{mountpoint:/var/lib/kubelet/pods/9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351/volumes/kubernetes.io~secret/image-registry-private-configuration major:0 minor:201 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351/volumes/kubernetes.io~secret/installation-pull-secrets:{mountpoint:/var/lib/kubelet/pods/9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351/volumes/kubernetes.io~secret/installation-pull-secrets major:0 minor:209 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a0823e8f-bbdd-477b-95ac-dad05d23cefa/volumes/kubernetes.io~projected/kube-api-access-t56q5:{mountpoint:/var/lib/kubelet/pods/a0823e8f-bbdd-477b-95ac-dad05d23cefa/volumes/kubernetes.io~projected/kube-api-access-t56q5 major:0 minor:60 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a3cf021c-2c20-4687-9b3a-cfa65ce1ee8b/volumes/kubernetes.io~projected/kube-api-access-jt475:{mountpoint:/var/lib/kubelet/pods/a3cf021c-2c20-4687-9b3a-cfa65ce1ee8b/volumes/kubernetes.io~projected/kube-api-access-jt475 major:0 minor:335 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/aa3f3b62-a1ab-4113-bef3-ed1b290e47c1/volumes/kubernetes.io~projected/kube-api-access-rvrxn:{mountpoint:/var/lib/kubelet/pods/aa3f3b62-a1ab-4113-bef3-ed1b290e47c1/volumes/kubernetes.io~projected/kube-api-access-rvrxn major:0 minor:66 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ad78ad05-b0af-4adf-a6bc-0f993b8a6288/volumes/kubernetes.io~projected/kube-api-access-5f2ls:{mountpoint:/var/lib/kubelet/pods/ad78ad05-b0af-4adf-a6bc-0f993b8a6288/volumes/kubernetes.io~projected/kube-api-access-5f2ls major:0 minor:67 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d81cef55-16ee-4677-ae12-86de3c9eb216/volumes/kubernetes.io~projected/kube-api-access-gzhnk:{mountpoint:/var/lib/kubelet/pods/d81cef55-16ee-4677-ae12-86de3c9eb216/volumes/kubernetes.io~projected/kube-api-access-gzhnk major:0 minor:65 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/da28351c-1c8b-40be-85a0-3aad427e3a17/volumes/kubernetes.io~projected/kube-api-access-cftdb:{mountpoint:/var/lib/kubelet/pods/da28351c-1c8b-40be-85a0-3aad427e3a17/volumes/kubernetes.io~projected/kube-api-access-cftdb major:0 minor:223 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/da28351c-1c8b-40be-85a0-3aad427e3a17/volumes/kubernetes.io~secret/default-certificate:{mountpoint:/var/lib/kubelet/pods/da28351c-1c8b-40be-85a0-3aad427e3a17/volumes/kubernetes.io~secret/default-certificate major:0 minor:202 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/da28351c-1c8b-40be-85a0-3aad427e3a17/volumes/kubernetes.io~secret/stats-auth:{mountpoint:/var/lib/kubelet/pods/da28351c-1c8b-40be-85a0-3aad427e3a17/volumes/kubernetes.io~secret/stats-auth major:0 minor:197 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/dedb8b54-d0ce-434c-9461-db90a189c121/volumes/kubernetes.io~projected/kube-api-access-btt6c:{mountpoint:/var/lib/kubelet/pods/dedb8b54-d0ce-434c-9461-db90a189c121/volumes/kubernetes.io~projected/kube-api-access-btt6c major:0 minor:217 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/df9e5074-3bb1-4817-8855-b045cee61004/volumes/kubernetes.io~projected/kube-api-access-nk8r7:{mountpoint:/var/lib/kubelet/pods/df9e5074-3bb1-4817-8855-b045cee61004/volumes/kubernetes.io~projected/kube-api-access-nk8r7 major:0 minor:45 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e6b2a584-ae78-4e80-b595-9b1a5897d556/volumes/kubernetes.io~projected/kube-api-access-24cp9:{mountpoint:/var/lib/kubelet/pods/e6b2a584-ae78-4e80-b595-9b1a5897d556/volumes/kubernetes.io~projected/kube-api-access-24cp9 major:0 minor:240 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e6b2a584-ae78-4e80-b595-9b1a5897d556/volumes/kubernetes.io~secret/cert:{mountpoint:/var/lib/kubelet/pods/e6b2a584-ae78-4e80-b595-9b1a5897d556/volumes/kubernetes.io~secret/cert major:0 minor:229 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e77c4479-d6e5-48a8-ac82-102a9f1a2c4a/volumes/kubernetes.io~projected/kube-api-access-x4hfg:{mountpoint:/var/lib/kubelet/pods/e77c4479-d6e5-48a8-ac82-102a9f1a2c4a/volumes/kubernetes.io~projected/kube-api-access-x4hfg major:0 minor:218 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f2a80d89-61d1-4694-8734-004ce5cfb22b/volumes/kubernetes.io~empty-dir/etc-tuned:{mountpoint:/var/lib/kubelet/pods/f2a80d89-61d1-4694-8734-004ce5cfb22b/volumes/kubernetes.io~empty-dir/etc-tuned major:0 minor:55 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f2a80d89-61d1-4694-8734-004ce5cfb22b/volumes/kubernetes.io~empty-dir/tmp:{mountpoint:/var/lib/kubelet/pods/f2a80d89-61d1-4694-8734-004ce5cfb22b/volumes/kubernetes.io~empty-dir/tmp major:0 minor:57 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f2a80d89-61d1-4694-8734-004ce5cfb22b/volumes/kubernetes.io~projected/kube-api-access-8wvhh:{mountpoint:/var/lib/kubelet/pods/f2a80d89-61d1-4694-8734-004ce5cfb22b/volumes/kubernetes.io~projected/kube-api-access-8wvhh major:0 minor:62 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f4ac17da-6b5b-4441-9b4e-ebe696e06507/volumes/kubernetes.io~projected/kube-api-access-kk8qr:{mountpoint:/var/lib/kubelet/pods/f4ac17da-6b5b-4441-9b4e-ebe696e06507/volumes/kubernetes.io~projected/kube-api-access-kk8qr major:0 minor:215 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0} overlay_0-100:{mountpoint:/var/lib/containers/storage/overlay/36792b9b84fae77302c55f99115a15e43943f814412f91ea169629ba72271093/merged major:0 minor:100 fsType:overlay blockSize:0} overlay_0-102:{mountpoint:/var/lib/containers/storage/overlay/71168058a0a0542c20f43edddf0ee98e68758ebb8478eab46656d07f58c265c7/merged major:0 minor:102 fsType:overlay blockSize:0} overlay_0-104:{mountpoint:/var/lib/containers/storage/overlay/f9242d7ebda7b589c20d8b67991d137b79f9def6ff9b525e11124931b7b6c0d9/merged major:0 minor:104 fsType:overlay blockSize:0} overlay_0-109:{mountpoint:/var/lib/containers/storage/overlay/968ae2256fe6ebdede65b2a5389dd982307acf77c88893a66f0a5c848033bc0a/merged major:0 minor:109 fsType:overlay blockSize:0} overlay_0-114:{mountpoint:/var/lib/containers/storage/overlay/0a525f68fdebc825a4cf5ef02cc51cb4d169839719d0ce225ff2930810e01cad/merged major:0 minor:114 fsType:overlay blockSize:0} overlay_0-119:{mountpoint:/var/lib/containers/storage/overlay/5d65e5ea5a180c49f7dfdfca76c672bb4dd633d07246feca4383c468cddc433c/merged major:0 minor:119 fsType:overlay blockSize:0} overlay_0-124:{mountpoint:/var/lib/containers/storage/overlay/864db7f4120808068adfc60f87b1f4ace172d55c95dee460f60a75751d4941c8/merged major:0 minor:124 fsType:overlay blockSize:0} overlay_0-127:{mountpoint:/var/lib/containers/storage/overlay/8c26433c5e2aacc09f266c7ebfe4bf7bf684aa89c323ae1ce0e8e065de713ffd/merged major:0 minor:127 fsType:overlay blockSize:0} overlay_0-129:{mountpoint:/var/lib/containers/storage/overlay/bd0532f22a583174bba6bd9bf9bb64e261baa6b5afcaef6fb1348c4aae1adf05/merged major:0 minor:129 fsType:overlay blockSize:0} overlay_0-134:{mountpoint:/var/lib/containers/storage/overlay/3971a281542b22565bdf6395c35026781e48079b04e82bdf745a24b9f5a4a3fc/merged major:0 minor:134 fsType:overlay blockSize:0} overlay_0-144:{mountpoint:/var/lib/containers/storage/overlay/becf77bd61c16feb155ded5fe6020d78b6e6771775803c2f5d207298f78bfbf1/merged major:0 minor:144 fsType:overlay blockSize:0} overlay_0-146:{mountpoint:/var/lib/containers/storage/overlay/c3aa2bbb115ca3c6c2552697ee743da8688de3192627036970566f10b842f397/merged major:0 minor:146 fsType:overlay blockSize:0} overlay_0-152:{mountpoint:/var/lib/containers/storage/overlay/4949519a4e3fa216de2514465e49f9d34a988806118ccfe068609b3e058f0ba0/merged major:0 minor:152 fsType:overlay blockSize:0} overlay_0-157:{mountpoint:/var/lib/containers/storage/overlay/32da3ee7d15c8350ce37e262ae249939b101d09a4a38cd06ae271d53b6fddbd2/merged major:0 minor:157 fsType:overlay blockSize:0} overlay_0-162:{mountpoint:/var/lib/containers/storage/overlay/f2def6a4c840f594b51e309e69cb9066d3d6a6f1f8317ded390fcd6aef20b467/merged major:0 minor:162 fsType:overlay blockSize:0} overlay_0-167:{mountpoint:/var/lib/containers/storage/overlay/19eed5df4ef1f750bd37e07578fa6b2b0bfd3046d77f1fe72dbd4223229fa512/merged major:0 minor:167 fsType:overlay blockSize:0} overlay_0-172:{mountpoint:/var/lib/containers/storage/overlay/7b1306bbf6f6efb657e21ed086569432c3aa059247f91046e35854200fa5e1fe/merged major:0 minor:172 fsType:overlay blockSize:0} overlay_0-177:{mountpoint:/var/lib/containers/storage/overlay/36be7a0ac419d835b9de4b4dc40afa953de3b09feca1a35c281b6667b0d1c6c9/merged major:0 minor:177 fsType:overlay blockSize:0} overlay_0-182:{mountpoint:/var/lib/containers/storage/overlay/e09fa482e5b7ef00387c9a06289c6d28d4b47d9a8f9235b0f97f7aef768b366a/merged major:0 minor:182 fsType:overlay blockSize:0} overlay_0-187:{mountpoint:/var/lib/containers/storage/overlay/c260a14082b410d5980dac1eb4fae11da6befc34d566de5fe7aa3800c4c627e3/merged major:0 minor:187 fsType:overlay blockSize:0} overlay_0-192:{mountpoint:/var/lib/containers/storage/overlay/4226c1e683af64d62ab885ff9f11de1de43c0ed09bfadc6b07ff0a56469c3f2d/merged major:0 minor:192 fsType:overlay blockSize:0} overlay_0-248:{mountpoint:/var/lib/containers/storage/overlay/727522196e32ea9ca1c3cb2a769a722d06210da4c2bb032dd7c3bf1e8a808964/merged major:0 minor:248 fsType:overlay blockSize:0} overlay_0-250:{mountpoint:/var/lib/containers/storage/overlay/f970dc8edafe9d7fb5f49d50889875ddfc9a5c6b9542f06b7fa31156ec673806/merged major:0 minor:250 fsType:overlay blockSize:0} overlay_0-252:{mountpoint:/var/lib/containers/storage/overlay/809fde7ed127089cc5f003ae19d1f4019c98663be470361b8ea750610c6f3674/merged major:0 minor:252 fsType:overlay blockSize:0} overlay_0-254:{mountpoint:/var/lib/containers/storage/overlay/d3bb9547272a61e9689c27f96edfc97c72ed6451f91c0a16e5872edb81d0535f/merged major:0 minor:254 fsType:overlay blockSize:0} overlay_0-256:{mountpoint:/var/lib/containers/storage/overlay/f16a5cd756fcc0258bac87b5ffb533188eb5811ff7ef1918fa3414bff8e42238/merged major:0 minor:256 fsType:overlay blockSize:0} overlay_0-262:{mountpoint:/var/lib/containers/storage/overlay/1e0add126dc0c55fbb27ecd887b087d420d7760b512acfbb31de47962a9ef4aa/merged major:0 minor:262 fsType:overlay blockSize:0} overlay_0-264:{mountpoint:/var/lib/containers/storage/overlay/5f11c5b0d19e495bed191ac7e7575a77ca976126fb1449ec71efff7b405c70c9/merged major:0 minor:264 fsType:overlay blockSize:0} overlay_0-266:{mountpoint:/var/lib/containers/storage/overlay/9357a1c16da8dbbe7eef77811978e51aa4da3db50ec81f7473c50197573f3b2e/merged major:0 minor:266 fsType:overlay blockSize:0} overlay_0-268:{mountpoint:/var/lib/containers/storage/overlay/f17b6237f715944c7499aa7c5a1d42d9a4e953f40bc464c26009d7859ffa5e66/merged major:0 minor:268 fsType:overlay blockSize:0} overlay_0-270:{mountpoint:/var/lib/containers/storage/overlay/d96eb5ffd1fb13d8f60374a4454b226edee108a6d756311191b9e38e76859df8/merged major:0 minor:270 fsType:overlay blockSize:0} overlay_0-271:{mountpoint:/var/lib/containers/storage/overlay/18aff1c821d5c95cf5b5e0d3f77f3a3f216320b14887ce74a282b01b733e046b/merged major:0 minor:271 fsType:overlay blockSize:0} overlay_0-278:{mountpoint:/var/lib/containers/storage/overlay/12d2bb6ded5baae41b82444535d40232ceb0f15bf7e8cf9375edcbb32cdfa06f/merged major:0 minor:278 fsType:overlay blockSize:0} overlay_0-285:{mountpoint:/var/lib/containers/storage/overlay/ccd38fbaef9f62eca74623bb58f6837a9b9ad1c992e39a70d24ff3c926c32346/merged major:0 minor:285 fsType:overlay blockSize:0} overlay_0-292:{mountpoint:/var/lib/containers/storage/overlay/2db56ffbe60c6e3e83f339313d9fd4c8caf6ae15ca21affed03fdf3406789a86/merged major:0 minor:292 fsType:overlay blockSize:0} overlay_0-294:{mountpoint:/var/lib/containers/storage/overlay/02c75c0ed2da7057bab8b86d97932700622787b61aab65b05f17ac826aa311ad/merged major:0 minor:294 fsType:overlay blockSize:0} overlay_0-296:{mountpoint:/var/lib/containers/storage/overlay/c7b7387b3d4082c2bc2a589cbe78ede6a342c850fd1b92f5b02f5bd7441748e6/merged major:0 minor:296 fsType:overlay blockSize:0} overlay_0-298:{mountpoint:/var/lib/containers/storage/overlay/341b1b58d0bd4c5dcae3dd3dd83e3e012506764a935e2e910918b8ad2e0756ea/merged major:0 minor:298 fsType:overlay blockSize:0} overlay_0-304:{mountpoint:/var/lib/containers/storage/overlay/8e8afea56b168bf1b2289e5c58546e1adf17733f19716bcb2b65ce9887243bbb/merged major:0 minor:304 fsType:overlay blockSize:0} overlay_0-310:{mountpoint:/var/lib/containers/storage/overlay/bfd26735576218006fe3729ebf97cbb40e7cf49115870b83d1409ab9fc1bd19d/merged major:0 minor:310 fsType:overlay blockSize:0} overlay_0-313:{mountpoint:/var/lib/containers/storage/overlay/60cc6e1f2ca0485c4d8bd9833c0ae4636a5f96ef338892ad11c68303e214bc16/merged major:0 minor:313 fsType:overlay blockSize:0} overlay_0-323:{mountpoint:/var/lib/containers/storage/overlay/9f162b84b0e018a43c65307edd6b3e5b5b8d59915b7c2581658d2ffa4d6a2285/merged major:0 minor:323 fsType:overlay blockSize:0} overlay_0-329:{mountpoint:/var/lib/containers/storage/overlay/3c8473ea58d21e560561d4ce4589f70d4cc454513fcbe99b32b9873f47fac953/merged major:0 minor:329 fsType:overlay blockSize:0} overlay_0-342:{mountpoint:/var/lib/containers/storage/overlay/7e466feae008d9cc2b5fcfcfb7f808b1a6222ab25e67c889fc01e1512a9a845d/merged major:0 minor:342 fsType:overlay blockSize:0} overlay_0-344:{mountpoint:/var/lib/containers/storage/overlay/b9775fe7935dcb6faf0ab7b9b7008af0cc480e3f80e4fdd7a1bf1d9b4e1337ef/merged major:0 minor:344 fsType:overlay blockSize:0} overlay_0-49:{mountpoint:/var/lib/containers/storage/overlay/92f2190280c49a27c0c653a4643ec422bcb34bddb67c8d0247fd18c9017eafcd/merged major:0 minor:49 fsType:overlay blockSize:0} overlay_0-53:{mountpoint:/var/lib/containers/storage/overlay/b912f1f155eac13f2b7a03d546dd19ae7e02914245f9f09b9b0a92836e67ac3e/merged major:0 minor:53 fsType:overlay blockSize:0} overlay_0-70:{mountpoint:/var/lib/containers/storage/overlay/eb51dfbb218d5051adedf4d5af692b48a2d61c61cf8fca6c4d2b9e01cce4756a/merged major:0 minor:70 fsType:overlay blockSize:0} overlay_0-74:{mountpoint:/var/lib/containers/storage/overlay/4af236c5423c476a291b1e6cfe7ece4b5b457c6acae77c5d5e95ba72505921f7/merged major:0 minor:74 fsType:overlay blockSize:0} overlay_0-76:{mountpoint:/var/lib/containers/storage/overlay/5bf14c05170a1c3671fa6b2f52eab5f9505b84aea2e4b29995880dad8f02ef8e/merged major:0 minor:76 fsType:overlay blockSize:0} overlay_0-80:{mountpoint:/var/lib/containers/storage/overlay/ea70c80eb812201d3f80135e9f9a65a0c877eff559556a272d2f30bf90a66bec/merged major:0 minor:80 fsType:overlay blockSize:0} overlay_0-85:{mountpoint:/var/lib/containers/storage/overlay/b1a7c8d1fcb91bd49d4f330eee8b8f98043c6607042f6611eb2fdec1bfb712fd/merged major:0 minor:85 fsType:overlay blockSize:0} overlay_0-88:{mountpoint:/var/lib/containers/storage/overlay/06f7e0aee52baf151dede221201644d5d92fd72c283ed2b75c0919df7f3684bf/merged major:0 minor:88 fsType:overlay blockSize:0} overlay_0-92:{mountpoint:/var/lib/containers/storage/overlay/f84aa59f2340e8662c9333a04c622cf65ce4a2548338cc7ed09fc7b183bfc673/merged major:0 minor:92 fsType:overlay blockSize:0} overlay_0-96:{mountpoint:/var/lib/containers/storage/overlay/fb04212db48992d762951bd38f19285c5c266b1bb9581571e01131fc8e5d2fce/merged major:0 minor:96 fsType:overlay blockSize:0}]
Oct 30 18:33:55.825988 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.825485    4888 manager.go:217] Machine: {Timestamp:2026-10-30 18:33:55.824389328 +0000 UTC m=+0.071795346 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100023 MemoryCapacity:32812011520 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec209f7cf0c716387e6716ee829baada SystemUUID:ec209f7c-f0c7-1638-7e67-16ee829baada BootID:d8d2fdad-954d-46d6-a103-8aed2454c617 Filesystems:[{Device:/var/lib/kubelet/pods/8789ec5b-226e-4551-bf68-3ac585006266/volumes/kubernetes.io~secret/image-registry-private-configuration DeviceMajor:0 DeviceMinor:208 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/613020c0-f3e4-4650-896f-748a40f0ab8b/volumes/kubernetes.io~projected/kube-api-access-c45qr DeviceMajor:0 DeviceMinor:289 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/ad78ad05-b0af-4adf-a6bc-0f993b8a6288/volumes/kubernetes.io~projected/kube-api-access-5f2ls DeviceMajor:0 DeviceMinor:67 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-157 DeviceMajor:0 DeviceMinor:157 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/81de5dce-c40f-4bd5-878c-5d75a0487066/volumes/kubernetes.io~projected/kube-api-access-9j2qw DeviceMajor:0 DeviceMinor:222 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-80 DeviceMajor:0 DeviceMinor:80 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-162 DeviceMajor:0 DeviceMinor:162 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-310 DeviceMajor:0 DeviceMinor:310 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/aa3f3b62-a1ab-4113-bef3-ed1b290e47c1/volumes/kubernetes.io~projected/kube-api-access-rvrxn DeviceMajor:0 DeviceMinor:66 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/8789ec5b-226e-4551-bf68-3ac585006266/volumes/kubernetes.io~projected/kube-api-access-vf948 DeviceMajor:0 DeviceMinor:213 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-344 DeviceMajor:0 DeviceMinor:344 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16406003712 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/da28351c-1c8b-40be-85a0-3aad427e3a17/volumes/kubernetes.io~secret/stats-auth DeviceMajor:0 DeviceMinor:197 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/09c13759-50d4-40e6-b854-4691822b8c8a/volumes/kubernetes.io~projected/kube-api-access-k5crl DeviceMajor:0 DeviceMinor:221 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-250 DeviceMajor:0 DeviceMinor:250 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-266 DeviceMajor:0 DeviceMinor:266 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-127 DeviceMajor:0 DeviceMinor:127 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/3f3fe764-2652-46c9-8e54-7882c0e6bcf1/volume-subpaths/run-systemd/ovnkube-controller/6 DeviceMajor:0 DeviceMinor:26 Capacity:6562402304 Type:vfs Inodes:819200 HasInodes:true} {Device:/run/containers/storage/overlay-containers/ecf8942e95e312a77dfd631bc979e0c4fb823eb197c2536e76ed89c513f0cd6e/userdata/shm DeviceMajor:0 DeviceMinor:227 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-296 DeviceMajor:0 DeviceMinor:296 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6562402304 Type:vfs Inodes:819200 HasInodes:true} {Device:overlay_0-92 DeviceMajor:0 DeviceMinor:92 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/e6b2a584-ae78-4e80-b595-9b1a5897d556/volumes/kubernetes.io~projected/kube-api-access-24cp9 DeviceMajor:0 DeviceMinor:240 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-252 DeviceMajor:0 DeviceMinor:252 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/46d9ac39-ee45-4e43-bda7-3931c3e599cf/volumes/kubernetes.io~secret/original-pull-secret DeviceMajor:0 DeviceMinor:280 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-298 DeviceMajor:0 DeviceMinor:298 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-104 DeviceMajor:0 DeviceMinor:104 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-134 DeviceMajor:0 DeviceMinor:134 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-187 DeviceMajor:0 DeviceMinor:187 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/7c09088b-3a07-426d-8804-f0f5bac4d27c/volumes/kubernetes.io~secret/serving-cert DeviceMajor:0 DeviceMinor:204 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/run/containers/storage/overlay-containers/41566f398fb6e7ad1be747e3b375979733d9965f00edb0dd1f8a0328234a824c/userdata/shm DeviceMajor:0 DeviceMinor:242 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-256 DeviceMajor:0 DeviceMinor:256 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-49 DeviceMajor:0 DeviceMinor:49 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/3f3fe764-2652-46c9-8e54-7882c0e6bcf1/volumes/kubernetes.io~projected/kube-api-access-wzxwr DeviceMajor:0 DeviceMinor:64 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-74 DeviceMajor:0 DeviceMinor:74 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351/volumes/kubernetes.io~secret/installation-pull-secrets DeviceMajor:0 DeviceMinor:209 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/run/containers/storage/overlay-containers/ecaf79502997d6f1f864f0f6e6427641cd2d8cc464f7f8ec8279c05e8bbc14b3/userdata/shm DeviceMajor:0 DeviceMinor:51 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-53 DeviceMajor:0 DeviceMinor:53 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-96 DeviceMajor:0 DeviceMinor:96 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351/volumes/kubernetes.io~projected/kube-api-access-vxw47 DeviceMajor:0 DeviceMinor:224 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/run/containers/storage/overlay-containers/f5a68d540ea6105c69282b72fcef5f4df6de3fef4565d6e65827425d667c1633/userdata/shm DeviceMajor:0 DeviceMinor:287 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-313 DeviceMajor:0 DeviceMinor:313 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/63090f48-9143-4118-9906-d759c0aff52c/volumes/kubernetes.io~projected/kube-api-access-bgxbg DeviceMajor:0 DeviceMinor:63 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/run/containers/storage/overlay-containers/9766c5bb2416907e5b0313e177e5075e6f6480fc8b492fcfa2d287d15133f64e/userdata/shm DeviceMajor:0 DeviceMinor:68 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:/run/containers/storage/overlay-containers/a0d10b0331467cc7d179bb0e0cb083d789035c89166c8c5f594f142864c45f4c/userdata/shm DeviceMajor:0 DeviceMinor:72 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/f2a80d89-61d1-4694-8734-004ce5cfb22b/volumes/kubernetes.io~empty-dir/etc-tuned DeviceMajor:0 DeviceMinor:55 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/2e49faaa-a718-44c8-9237-5a41169faee7/volumes/kubernetes.io~projected/kube-api-access-6llht DeviceMajor:0 DeviceMinor:61 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/run/containers/storage/overlay-containers/010820c82deec18c7b4002e46b65e8dab8cc3c9c8cd27ac759462efc6c4c32ac/userdata/shm DeviceMajor:0 DeviceMinor:83 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-192 DeviceMajor:0 DeviceMinor:192 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-264 DeviceMajor:0 DeviceMinor:264 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-177 DeviceMajor:0 DeviceMinor:177 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-262 DeviceMajor:0 DeviceMinor:262 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/57ca8600-79e7-4f59-9d11-b1c9ec3d358a/volumes/kubernetes.io~projected/kube-api-access-wrmcg DeviceMajor:0 DeviceMinor:312 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/var/lib/kubelet/pods/d81cef55-16ee-4677-ae12-86de3c9eb216/volumes/kubernetes.io~projected/kube-api-access-gzhnk DeviceMajor:0 DeviceMinor:65 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-167 DeviceMajor:0 DeviceMinor:167 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/382e8b31-a7fa-43f3-9f15-4c61627b0c04/volumes/kubernetes.io~projected/kube-api-access-m4j2q DeviceMajor:0 DeviceMinor:219 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/36a04851-81db-40cd-97ef-8e2e00a08604/volumes/kubernetes.io~projected/kube-api-access-rw8g9 DeviceMajor:0 DeviceMinor:225 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/run/containers/storage/overlay-containers/264171f78a701c6a4af300c38636f113df3b276804fb65397c6d59c290a3b89f/userdata/shm DeviceMajor:0 DeviceMinor:231 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-76 DeviceMajor:0 DeviceMinor:76 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/02e1780cae1ffe8664994cc74406d074948af8b6ea7d829904b28cc97b45939f/userdata/shm DeviceMajor:0 DeviceMinor:244 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-342 DeviceMajor:0 DeviceMinor:342 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/df9e5074-3bb1-4817-8855-b045cee61004/volumes/kubernetes.io~projected/kube-api-access-nk8r7 DeviceMajor:0 DeviceMinor:45 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-285 DeviceMajor:0 DeviceMinor:285 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-85 DeviceMajor:0 DeviceMinor:85 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-88 DeviceMajor:0 DeviceMinor:88 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/382e8b31-a7fa-43f3-9f15-4c61627b0c04/volumes/kubernetes.io~secret/serving-cert DeviceMajor:0 DeviceMinor:203 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/68195e19-b1ff-4148-9fae-3545f2997ff5/volumes/kubernetes.io~projected/kube-api-access-4rztt DeviceMajor:0 DeviceMinor:212 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/f2a80d89-61d1-4694-8734-004ce5cfb22b/volumes/kubernetes.io~projected/kube-api-access-8wvhh DeviceMajor:0 DeviceMinor:62 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/run/containers/storage/overlay-containers/e1430cbb57146c9fef763300e771803f7b3cb39a5af4e3bbf5c2b1ea173d02d7/userdata/shm DeviceMajor:0 DeviceMinor:97 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351/volumes/kubernetes.io~secret/image-registry-private-configuration DeviceMajor:0 DeviceMinor:201 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/da28351c-1c8b-40be-85a0-3aad427e3a17/volumes/kubernetes.io~secret/default-certificate DeviceMajor:0 DeviceMinor:202 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/a3cf021c-2c20-4687-9b3a-cfa65ce1ee8b/volumes/kubernetes.io~projected/kube-api-access-jt475 DeviceMajor:0 DeviceMinor:335 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16406007808 Type:vfs Inodes:1048576 HasInodes:true} {Device:overlay_0-109 DeviceMajor:0 DeviceMinor:109 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/8789ec5b-226e-4551-bf68-3ac585006266/volumes/kubernetes.io~secret/installation-pull-secrets DeviceMajor:0 DeviceMinor:206 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-271 DeviceMajor:0 DeviceMinor:271 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-278 DeviceMajor:0 DeviceMinor:278 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-292 DeviceMajor:0 DeviceMinor:292 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/8789ec5b-226e-4551-bf68-3ac585006266/volumes/kubernetes.io~projected/bound-sa-token DeviceMajor:0 DeviceMinor:226 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/3f3fe764-2652-46c9-8e54-7882c0e6bcf1/volumes/kubernetes.io~secret/ovn-node-metrics-cert DeviceMajor:0 DeviceMinor:58 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/run/containers/storage/overlay-containers/a6818d98d464d242de00bec5b3468e251eaafeecc0329e9b18c563db7d7b4ba8/userdata/shm DeviceMajor:0 DeviceMinor:78 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-114 DeviceMajor:0 DeviceMinor:114 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/7c09088b-3a07-426d-8804-f0f5bac4d27c/volumes/kubernetes.io~projected/kube-api-access-cvxhl DeviceMajor:0 DeviceMinor:214 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/dedb8b54-d0ce-434c-9461-db90a189c121/volumes/kubernetes.io~projected/kube-api-access-btt6c DeviceMajor:0 DeviceMinor:217 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/da28351c-1c8b-40be-85a0-3aad427e3a17/volumes/kubernetes.io~projected/kube-api-access-cftdb DeviceMajor:0 DeviceMinor:223 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/run/containers/storage/overlay-containers/93eecdc0633397bbec713dfdc4aff8aeb6399c863290c63bbd8a0da3fb70b0fc/userdata/shm DeviceMajor:0 DeviceMinor:230 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:/run/containers/storage/overlay-containers/110c3deb893a6eff9717f921ccf7dc3f3e56cf7408a56a85d1d6b3e8a95633a4/userdata/shm DeviceMajor:0 DeviceMinor:246 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-270 DeviceMajor:0 DeviceMinor:270 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-119 DeviceMajor:0 DeviceMinor:119 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-172 DeviceMajor:0 DeviceMinor:172 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-304 DeviceMajor:0 DeviceMinor:304 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-329 DeviceMajor:0 DeviceMinor:329 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/run/containers/storage/overlay-containers/8f3fb203f2d1c8bf14c057244ee2331e339db7c0ab0f9a744f96acb79a075483/userdata/shm DeviceMajor:0 DeviceMinor:90 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:/run/containers/storage/overlay-containers/3dacffc196bf6205ffae4eaaf96fc7405b8753485def2dab566ce25b8106f9ae/userdata/shm DeviceMajor:0 DeviceMinor:236 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:/run/containers/storage/overlay-containers/b3f8d0948c4c12d4822539070451f179a0b9a930adb6cdfd15c3c8b13a7f4395/userdata/shm DeviceMajor:0 DeviceMinor:238 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/8ecf2b62-8053-4bba-a852-c6f6119d3288/volumes/kubernetes.io~secret/agent-certs DeviceMajor:0 DeviceMinor:56 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/f2a80d89-61d1-4694-8734-004ce5cfb22b/volumes/kubernetes.io~empty-dir/tmp DeviceMajor:0 DeviceMinor:57 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-182 DeviceMajor:0 DeviceMinor:182 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/83e431df5ea6349b7b60e71173b402e4ef22fd518c8f8b81c8eccd92d1179b40/userdata/shm DeviceMajor:0 DeviceMinor:340 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:/run/containers/storage/overlay-containers/94eb3352db48f6acf5fa74ee0bcc8ac26482c3360cb4db2fabe3ff668bd86c13/userdata/shm DeviceMajor:0 DeviceMinor:47 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-100 DeviceMajor:0 DeviceMinor:100 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-129 DeviceMajor:0 DeviceMinor:129 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-144 DeviceMajor:0 DeviceMinor:144 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-152 DeviceMajor:0 DeviceMinor:152 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/f4ac17da-6b5b-4441-9b4e-ebe696e06507/volumes/kubernetes.io~projected/kube-api-access-kk8qr DeviceMajor:0 DeviceMinor:215 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351/volumes/kubernetes.io~projected/bound-sa-token DeviceMajor:0 DeviceMinor:216 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/e77c4479-d6e5-48a8-ac82-102a9f1a2c4a/volumes/kubernetes.io~projected/kube-api-access-x4hfg DeviceMajor:0 DeviceMinor:218 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-146 DeviceMajor:0 DeviceMinor:146 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-248 DeviceMajor:0 DeviceMinor:248 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-254 DeviceMajor:0 DeviceMinor:254 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/46a80637-6ccf-4a56-9904-f5e2fe5784ad/volumes/kubernetes.io~projected/kube-api-access-x2978 DeviceMajor:0 DeviceMinor:59 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/36a04851-81db-40cd-97ef-8e2e00a08604/volumes/kubernetes.io~secret/serving-cert DeviceMajor:0 DeviceMinor:205 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/run/containers/storage/overlay-containers/1fb8715703bb95f8a132c554ff1169dc975c64de24b0bc642f7bf41a12efe875/userdata/shm DeviceMajor:0 DeviceMinor:234 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-268 DeviceMajor:0 DeviceMinor:268 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/0698cafab13d74adaa3fd8179e2b7ec62ff9ce956db4148cafedadbb6d4d553c/userdata/shm DeviceMajor:0 DeviceMinor:82 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:/run/containers/storage/overlay-containers/add4754d17d14d035566b3612bce0fd00489041c05d84daf0553c4e983484ba6/userdata/shm DeviceMajor:0 DeviceMinor:94 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-323 DeviceMajor:0 DeviceMinor:323 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/a0823e8f-bbdd-477b-95ac-dad05d23cefa/volumes/kubernetes.io~projected/kube-api-access-t56q5 DeviceMajor:0 DeviceMinor:60 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-102 DeviceMajor:0 DeviceMinor:102 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-124 DeviceMajor:0 DeviceMinor:124 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-70 DeviceMajor:0 DeviceMinor:70 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/09c13759-50d4-40e6-b854-4691822b8c8a/volumes/kubernetes.io~secret/serving-cert DeviceMajor:0 DeviceMinor:207 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/run/containers/storage/overlay-containers/1d584626e578f70bd2bd96914c22b5060fe64a2954330b204e9b4c0683253984/userdata/shm DeviceMajor:0 DeviceMinor:210 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/15d485f9-5a4b-4f4b-9e59-a6eb062b4c5c/volumes/kubernetes.io~projected/kube-api-access-fzrkt DeviceMajor:0 DeviceMinor:220 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/e6b2a584-ae78-4e80-b595-9b1a5897d556/volumes/kubernetes.io~secret/cert DeviceMajor:0 DeviceMinor:229 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/var/lib/kubelet/pods/97b19aa3-42be-4def-8d7a-8e3516852e79/volumes/kubernetes.io~projected/kube-api-access-tc2fp DeviceMajor:0 DeviceMinor:241 Capacity:31633412096 Type:vfs Inodes:4005372 HasInodes:true} {Device:/run/containers/storage/overlay-containers/41306bd73702bf41bacd24f69e9da18eaa60d7a3d5c5782b2c004af7846ac805/userdata/shm DeviceMajor:0 DeviceMinor:290 Capacity:67108864 Type:vfs Inodes:4005372 HasInodes:true} {Device:overlay_0-294 DeviceMajor:0 DeviceMinor:294 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:02e1780cae1ffe8 MacAddress:ce:b4:d1:93:dc:1e Speed:10000 Mtu:8901} {Name:110c3deb893a6ef MacAddress:42:15:36:12:da:74 Speed:10000 Mtu:8901} {Name:1d584626e578f70 MacAddress:ce:08:70:32:d1:25 Speed:10000 Mtu:8901} {Name:1fb8715703bb95f MacAddress:8e:cd:62:bf:0d:a2 Speed:10000 Mtu:8901} {Name:264171f78a701c6 MacAddress:ce:9b:d7:7e:8e:8f Speed:10000 Mtu:8901} {Name:3dacffc196bf620 MacAddress:22:c0:1b:e8:56:7a Speed:10000 Mtu:8901} {Name:41306bd73702bf4 MacAddress:8e:f1:93:28:3d:81 Speed:10000 Mtu:8901} {Name:41566f398fb6e7a MacAddress:72:b6:ff:65:ce:1b Speed:10000 Mtu:8901} {Name:83e431df5ea6349 MacAddress:26:28:b3:2e:22:fb Speed:10000 Mtu:8901} {Name:93eecdc0633397b MacAddress:02:a2:80:20:51:11 Speed:10000 Mtu:8901} {Name:b3f8d0948c4c12d MacAddress:5e:12:ed:b0:db:b7 Speed:10000 Mtu:8901} {Name:br-ex MacAddress:02:2f:7f:1b:a1:51 Speed:0 Mtu:9001} {Name:br-int MacAddress:3e:5e:51:3b:97:18 Speed:0 Mtu:8901} {Name:ecf8942e95e312a MacAddress:16:ad:f8:8d:a2:53 Speed:10000 Mtu:8901} {Name:ens5 MacAddress:02:2f:7f:1b:a1:51 Speed:0 Mtu:9001} {Name:f5a68d540ea6105 MacAddress:1a:8f:84:38:0d:b2 Speed:10000 Mtu:8901} {Name:genev_sys_6081 MacAddress:b6:c3:ad:c9:1a:dd Speed:0 Mtu:65000} {Name:ovn-k8s-mp0 MacAddress:0a:58:0a:84:00:02 Speed:0 Mtu:8901} {Name:ovs-system MacAddress:ce:cf:b9:73:19:f0 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812011520 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Oct 30 18:33:55.825988 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.825826    4888 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Oct 30 18:33:55.825988 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.825866    4888 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Oct 30 18:33:55.826245 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.826108    4888 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Oct 30 18:33:55.826245 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.826130    4888 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-134-46.us-west-1.compute.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Oct 30 18:33:55.826328 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.826248    4888 topology_manager.go:143] "Creating topology manager with none policy"
Oct 30 18:33:55.826328 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.826255    4888 container_manager_linux.go:311] "Creating device plugin manager"
Oct 30 18:33:55.826328 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.826264    4888 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 30 18:33:55.826328 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.826276    4888 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 30 18:33:55.826328 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.826313    4888 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager"
Oct 30 18:33:55.826487 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.826475    4888 state_mem.go:41] "Initialized" logger="CPUManager state memory"
Oct 30 18:33:55.826584 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.826575    4888 server.go:1235] "Using root directory" path="/var/lib/kubelet"
Oct 30 18:33:55.826614 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.826611    4888 kubelet.go:493] "Attempting to sync node with API server"
Oct 30 18:33:55.826645 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.826624    4888 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests"
Oct 30 18:33:55.826645 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.826634    4888 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Oct 30 18:33:55.826645 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.826643    4888 kubelet.go:405] "Adding apiserver pod source"
Oct 30 18:33:55.826724 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.826652    4888 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Oct 30 18:33:55.827511 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.827496    4888 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint"
Oct 30 18:33:55.827586 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.827512    4888 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 30 18:33:55.828034 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.828008    4888 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1"
Oct 30 18:33:55.839595 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.839576    4888 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint"
Oct 30 18:33:55.839595 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.839588    4888 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 30 18:33:55.839707 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.839690    4888 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-server-current.pem"
Oct 30 18:33:55.839889 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.839878    4888 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Oct 30 18:33:55.839924 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.839908    4888 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Oct 30 18:33:55.840024 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.840015    4888 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Oct 30 18:33:55.840054 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.840031    4888 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Oct 30 18:33:55.840054 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.840037    4888 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Oct 30 18:33:55.840054 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.840043    4888 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Oct 30 18:33:55.840054 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.840050    4888 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Oct 30 18:33:55.840152 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.840060    4888 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Oct 30 18:33:55.840152 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.840069    4888 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Oct 30 18:33:55.840152 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.840077    4888 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Oct 30 18:33:55.840152 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.840083    4888 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Oct 30 18:33:55.840152 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.840088    4888 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Oct 30 18:33:55.840152 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.840102    4888 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Oct 30 18:33:55.840152 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.840111    4888 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Oct 30 18:33:55.840152 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.840136    4888 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Oct 30 18:33:55.840152 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.840142    4888 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Oct 30 18:33:55.843616 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.843600    4888 server.go:1263] "Started kubelet"
Oct 30 18:33:55.843713 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.843681    4888 server.go:182] "Starting to listen" address="0.0.0.0" port=10250
Oct 30 18:33:55.844136 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.843706    4888 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Oct 30 18:33:55.844136 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.843866    4888 server_v1.go:49] "podresources" method="list" useActivePods=true
Oct 30 18:33:55.844114 ip-10-0-134-46 systemd[1]: Started Kubernetes Kubelet.
Oct 30 18:33:55.844390 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.844165    4888 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Oct 30 18:33:55.844953 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.844937    4888 server.go:317] "Adding debug handlers to kubelet server"
Oct 30 18:33:55.849568 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.849553    4888 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer"
Oct 30 18:33:55.849836 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.849815    4888 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Oct 30 18:33:55.850580 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.850557    4888 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-29 18:28:15 +0000 UTC" deadline="2028-06-05 16:31:11.053784179 +0000 UTC"
Oct 30 18:33:55.850672 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.850580    4888 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14013h57m15.203207983s"
Oct 30 18:33:55.850672 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:33:55.850613    4888 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-134-46.us-west-1.compute.internal\" not found"
Oct 30 18:33:55.850672 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.850663    4888 volume_manager.go:309] "The desired_state_of_world populator starts"
Oct 30 18:33:55.850810 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.850675    4888 volume_manager.go:311] "Starting Kubelet Volume Manager"
Oct 30 18:33:55.850810 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.850779    4888 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Oct 30 18:33:55.851445 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.851424    4888 factory.go:55] Registering systemd factory
Oct 30 18:33:55.851445 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.851447    4888 factory.go:223] Registration of the systemd container factory successfully
Oct 30 18:33:55.854306 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.851777    4888 factory.go:153] Registering CRI-O factory
Oct 30 18:33:55.854306 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.851800    4888 factory.go:223] Registration of the crio container factory successfully
Oct 30 18:33:55.854306 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.851859    4888 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Oct 30 18:33:55.854306 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.851878    4888 factory.go:103] Registering Raw factory
Oct 30 18:33:55.854306 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.851895    4888 manager.go:1199] Started watching for new ooms in manager
Oct 30 18:33:55.854306 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.852315    4888 manager.go:319] Starting recovery of all containers
Oct 30 18:33:55.854306 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.853354    4888 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="aa3f3b62-a1ab-4113-bef3-ed1b290e47c1" volumeName="kubernetes.io/configmap/aa3f3b62-a1ab-4113-bef3-ed1b290e47c1-cni-binary-copy" seLinuxMountContext=""
Oct 30 18:33:55.854306 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.853388    4888 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="aa3f3b62-a1ab-4113-bef3-ed1b290e47c1" volumeName="kubernetes.io/projected/aa3f3b62-a1ab-4113-bef3-ed1b290e47c1-kube-api-access-rvrxn" seLinuxMountContext=""
Oct 30 18:33:55.854306 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.853403    4888 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e77c4479-d6e5-48a8-ac82-102a9f1a2c4a" volumeName="kubernetes.io/projected/e77c4479-d6e5-48a8-ac82-102a9f1a2c4a-kube-api-access-x4hfg" seLinuxMountContext=""
Oct 30 18:33:55.854306 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.853420    4888 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="382e8b31-a7fa-43f3-9f15-4c61627b0c04" volumeName="kubernetes.io/configmap/382e8b31-a7fa-43f3-9f15-4c61627b0c04-config" seLinuxMountContext=""
Oct 30 18:33:55.854306 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.853433    4888 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="613020c0-f3e4-4650-896f-748a40f0ab8b" volumeName="kubernetes.io/projected/613020c0-f3e4-4650-896f-748a40f0ab8b-kube-api-access-c45qr" seLinuxMountContext=""
Oct 30 18:33:55.854306 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.853445    4888 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="63090f48-9143-4118-9906-d759c0aff52c" volumeName="kubernetes.io/projected/63090f48-9143-4118-9906-d759c0aff52c-kube-api-access-bgxbg" seLinuxMountContext=""
Oct 30 18:33:55.854306 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.853458    4888 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8789ec5b-226e-4551-bf68-3ac585006266" volumeName="kubernetes.io/secret/8789ec5b-226e-4551-bf68-3ac585006266-image-registry-private-configuration" seLinuxMountContext=""
Oct 30 18:33:55.854306 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.853470    4888 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ad78ad05-b0af-4adf-a6bc-0f993b8a6288" volumeName="kubernetes.io/projected/ad78ad05-b0af-4adf-a6bc-0f993b8a6288-kube-api-access-5f2ls" seLinuxMountContext=""
Oct 30 18:33:55.854306 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.853485    4888 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8789ec5b-226e-4551-bf68-3ac585006266" volumeName="kubernetes.io/projected/8789ec5b-226e-4551-bf68-3ac585006266-bound-sa-token" seLinuxMountContext=""
Oct 30 18:33:55.854306 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.853496    4888 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="46a80637-6ccf-4a56-9904-f5e2fe5784ad" volumeName="kubernetes.io/configmap/46a80637-6ccf-4a56-9904-f5e2fe5784ad-serviceca" seLinuxMountContext=""
Oct 30 18:33:55.854306 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.853508    4888 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7c09088b-3a07-426d-8804-f0f5bac4d27c" volumeName="kubernetes.io/empty-dir/7c09088b-3a07-426d-8804-f0f5bac4d27c-tmp" seLinuxMountContext=""
Oct 30 18:33:55.854306 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.853523    4888 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8ecf2b62-8053-4bba-a852-c6f6119d3288" volumeName="kubernetes.io/configmap/8ecf2b62-8053-4bba-a852-c6f6119d3288-konnectivity-ca" seLinuxMountContext=""
Oct 30 18:33:55.854306 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.853536    4888 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a0823e8f-bbdd-477b-95ac-dad05d23cefa" volumeName="kubernetes.io/projected/a0823e8f-bbdd-477b-95ac-dad05d23cefa-kube-api-access-t56q5" seLinuxMountContext=""
Oct 30 18:33:55.854306 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.853549    4888 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f2a80d89-61d1-4694-8734-004ce5cfb22b" volumeName="kubernetes.io/projected/f2a80d89-61d1-4694-8734-004ce5cfb22b-kube-api-access-8wvhh" seLinuxMountContext=""
Oct 30 18:33:55.854306 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.853565    4888 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f4ac17da-6b5b-4441-9b4e-ebe696e06507" volumeName="kubernetes.io/projected/f4ac17da-6b5b-4441-9b4e-ebe696e06507-kube-api-access-kk8qr" seLinuxMountContext=""
Oct 30 18:33:55.854306 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.853577    4888 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="15d485f9-5a4b-4f4b-9e59-a6eb062b4c5c" volumeName="kubernetes.io/configmap/15d485f9-5a4b-4f4b-9e59-a6eb062b4c5c-samples-operator-config" seLinuxMountContext=""
Oct 30 18:33:55.854306 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.853590    4888 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2e49faaa-a718-44c8-9237-5a41169faee7" volumeName="kubernetes.io/configmap/2e49faaa-a718-44c8-9237-5a41169faee7-cni-sysctl-allowlist" seLinuxMountContext=""
Oct 30 18:33:55.854306 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.853620    4888 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="46d9ac39-ee45-4e43-bda7-3931c3e599cf" volumeName="kubernetes.io/secret/46d9ac39-ee45-4e43-bda7-3931c3e599cf-original-pull-secret" seLinuxMountContext=""
Oct 30 18:33:55.854306 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.853633    4888 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8789ec5b-226e-4551-bf68-3ac585006266" volumeName="kubernetes.io/empty-dir/8789ec5b-226e-4551-bf68-3ac585006266-ca-trust-extracted" seLinuxMountContext=""
Oct 30 18:33:55.854306 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.853645    4888 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8789ec5b-226e-4551-bf68-3ac585006266" volumeName="kubernetes.io/secret/8789ec5b-226e-4551-bf68-3ac585006266-installation-pull-secrets" seLinuxMountContext=""
Oct 30 18:33:55.854306 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.853658    4888 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2e49faaa-a718-44c8-9237-5a41169faee7" volumeName="kubernetes.io/configmap/2e49faaa-a718-44c8-9237-5a41169faee7-whereabouts-flatfile-configmap" seLinuxMountContext=""
Oct 30 18:33:55.854306 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.853669    4888 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7c09088b-3a07-426d-8804-f0f5bac4d27c" volumeName="kubernetes.io/configmap/7c09088b-3a07-426d-8804-f0f5bac4d27c-service-ca-bundle" seLinuxMountContext=""
Oct 30 18:33:55.854306 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.853682    4888 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8789ec5b-226e-4551-bf68-3ac585006266" volumeName="kubernetes.io/configmap/8789ec5b-226e-4551-bf68-3ac585006266-trusted-ca" seLinuxMountContext=""
Oct 30 18:33:55.854306 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.853695    4888 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="da28351c-1c8b-40be-85a0-3aad427e3a17" volumeName="kubernetes.io/projected/da28351c-1c8b-40be-85a0-3aad427e3a17-kube-api-access-cftdb" seLinuxMountContext=""
Oct 30 18:33:55.854306 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.853708    4888 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351" volumeName="kubernetes.io/secret/9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351-installation-pull-secrets" seLinuxMountContext=""
Oct 30 18:33:55.854306 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.853737    4888 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="da28351c-1c8b-40be-85a0-3aad427e3a17" volumeName="kubernetes.io/secret/da28351c-1c8b-40be-85a0-3aad427e3a17-stats-auth" seLinuxMountContext=""
Oct 30 18:33:55.854306 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.853749    4888 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="36a04851-81db-40cd-97ef-8e2e00a08604" volumeName="kubernetes.io/configmap/36a04851-81db-40cd-97ef-8e2e00a08604-config" seLinuxMountContext=""
Oct 30 18:33:55.854306 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.853761    4888 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="68195e19-b1ff-4148-9fae-3545f2997ff5" volumeName="kubernetes.io/configmap/68195e19-b1ff-4148-9fae-3545f2997ff5-telemetry-config" seLinuxMountContext=""
Oct 30 18:33:55.854306 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.853789    4888 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7c09088b-3a07-426d-8804-f0f5bac4d27c" volumeName="kubernetes.io/secret/7c09088b-3a07-426d-8804-f0f5bac4d27c-serving-cert" seLinuxMountContext=""
Oct 30 18:33:55.854306 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.853801    4888 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351" volumeName="kubernetes.io/empty-dir/9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351-ca-trust-extracted" seLinuxMountContext=""
Oct 30 18:33:55.854306 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.853814    4888 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="aa3f3b62-a1ab-4113-bef3-ed1b290e47c1" volumeName="kubernetes.io/configmap/aa3f3b62-a1ab-4113-bef3-ed1b290e47c1-multus-daemon-config" seLinuxMountContext=""
Oct 30 18:33:55.854306 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.853826    4888 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d81cef55-16ee-4677-ae12-86de3c9eb216" volumeName="kubernetes.io/projected/d81cef55-16ee-4677-ae12-86de3c9eb216-kube-api-access-gzhnk" seLinuxMountContext=""
Oct 30 18:33:55.854306 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.853838    4888 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="dedb8b54-d0ce-434c-9461-db90a189c121" volumeName="kubernetes.io/projected/dedb8b54-d0ce-434c-9461-db90a189c121-kube-api-access-btt6c" seLinuxMountContext=""
Oct 30 18:33:55.854306 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.853851    4888 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="97b19aa3-42be-4def-8d7a-8e3516852e79" volumeName="kubernetes.io/projected/97b19aa3-42be-4def-8d7a-8e3516852e79-kube-api-access-tc2fp" seLinuxMountContext=""
Oct 30 18:33:55.854306 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.853863    4888 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="09c13759-50d4-40e6-b854-4691822b8c8a" volumeName="kubernetes.io/secret/09c13759-50d4-40e6-b854-4691822b8c8a-serving-cert" seLinuxMountContext=""
Oct 30 18:33:55.854306 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.853875    4888 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3f3fe764-2652-46c9-8e54-7882c0e6bcf1" volumeName="kubernetes.io/projected/3f3fe764-2652-46c9-8e54-7882c0e6bcf1-kube-api-access-wzxwr" seLinuxMountContext=""
Oct 30 18:33:55.854306 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.853888    4888 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3f3fe764-2652-46c9-8e54-7882c0e6bcf1" volumeName="kubernetes.io/secret/3f3fe764-2652-46c9-8e54-7882c0e6bcf1-ovn-node-metrics-cert" seLinuxMountContext=""
Oct 30 18:33:55.854306 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.853900    4888 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="68195e19-b1ff-4148-9fae-3545f2997ff5" volumeName="kubernetes.io/projected/68195e19-b1ff-4148-9fae-3545f2997ff5-kube-api-access-4rztt" seLinuxMountContext=""
Oct 30 18:33:55.854306 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.853913    4888 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7c09088b-3a07-426d-8804-f0f5bac4d27c" volumeName="kubernetes.io/projected/7c09088b-3a07-426d-8804-f0f5bac4d27c-kube-api-access-cvxhl" seLinuxMountContext=""
Oct 30 18:33:55.854306 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.853925    4888 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8ecf2b62-8053-4bba-a852-c6f6119d3288" volumeName="kubernetes.io/secret/8ecf2b62-8053-4bba-a852-c6f6119d3288-agent-certs" seLinuxMountContext=""
Oct 30 18:33:55.854306 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.853938    4888 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="da28351c-1c8b-40be-85a0-3aad427e3a17" volumeName="kubernetes.io/secret/da28351c-1c8b-40be-85a0-3aad427e3a17-default-certificate" seLinuxMountContext=""
Oct 30 18:33:55.854306 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.853951    4888 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="09c13759-50d4-40e6-b854-4691822b8c8a" volumeName="kubernetes.io/configmap/09c13759-50d4-40e6-b854-4691822b8c8a-config" seLinuxMountContext=""
Oct 30 18:33:55.854306 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.853972    4888 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="382e8b31-a7fa-43f3-9f15-4c61627b0c04" volumeName="kubernetes.io/projected/382e8b31-a7fa-43f3-9f15-4c61627b0c04-kube-api-access-m4j2q" seLinuxMountContext=""
Oct 30 18:33:55.854306 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.853984    4888 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="57ca8600-79e7-4f59-9d11-b1c9ec3d358a" volumeName="kubernetes.io/projected/57ca8600-79e7-4f59-9d11-b1c9ec3d358a-kube-api-access-wrmcg" seLinuxMountContext=""
Oct 30 18:33:55.854306 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.853997    4888 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8789ec5b-226e-4551-bf68-3ac585006266" volumeName="kubernetes.io/projected/8789ec5b-226e-4551-bf68-3ac585006266-kube-api-access-vf948" seLinuxMountContext=""
Oct 30 18:33:55.854306 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.854009    4888 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351" volumeName="kubernetes.io/configmap/9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351-registry-certificates" seLinuxMountContext=""
Oct 30 18:33:55.854306 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.854022    4888 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f2a80d89-61d1-4694-8734-004ce5cfb22b" volumeName="kubernetes.io/empty-dir/f2a80d89-61d1-4694-8734-004ce5cfb22b-etc-tuned" seLinuxMountContext=""
Oct 30 18:33:55.854306 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.854035    4888 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="57ca8600-79e7-4f59-9d11-b1c9ec3d358a" volumeName="kubernetes.io/empty-dir/57ca8600-79e7-4f59-9d11-b1c9ec3d358a-data-volume" seLinuxMountContext=""
Oct 30 18:33:55.854306 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.854049    4888 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d81cef55-16ee-4677-ae12-86de3c9eb216" volumeName="kubernetes.io/empty-dir/d81cef55-16ee-4677-ae12-86de3c9eb216-tmp-dir" seLinuxMountContext=""
Oct 30 18:33:55.854306 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.854062    4888 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351" volumeName="kubernetes.io/projected/9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351-bound-sa-token" seLinuxMountContext=""
Oct 30 18:33:55.854306 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.854074    4888 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3f3fe764-2652-46c9-8e54-7882c0e6bcf1" volumeName="kubernetes.io/configmap/3f3fe764-2652-46c9-8e54-7882c0e6bcf1-env-overrides" seLinuxMountContext=""
Oct 30 18:33:55.854306 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.854086    4888 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="382e8b31-a7fa-43f3-9f15-4c61627b0c04" volumeName="kubernetes.io/configmap/382e8b31-a7fa-43f3-9f15-4c61627b0c04-trusted-ca" seLinuxMountContext=""
Oct 30 18:33:55.854306 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.854098    4888 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="09c13759-50d4-40e6-b854-4691822b8c8a" volumeName="kubernetes.io/projected/09c13759-50d4-40e6-b854-4691822b8c8a-kube-api-access-k5crl" seLinuxMountContext=""
Oct 30 18:33:55.854306 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.854109    4888 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="36a04851-81db-40cd-97ef-8e2e00a08604" volumeName="kubernetes.io/projected/36a04851-81db-40cd-97ef-8e2e00a08604-kube-api-access-rw8g9" seLinuxMountContext=""
Oct 30 18:33:55.854306 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.854122    4888 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="71fce796-0032-481b-80b3-66fbb731452c" volumeName="kubernetes.io/configmap/71fce796-0032-481b-80b3-66fbb731452c-nginx-conf" seLinuxMountContext=""
Oct 30 18:33:55.854306 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.854134    4888 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351" volumeName="kubernetes.io/secret/9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351-image-registry-private-configuration" seLinuxMountContext=""
Oct 30 18:33:55.854306 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.854155    4888 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="df9e5074-3bb1-4817-8855-b045cee61004" volumeName="kubernetes.io/projected/df9e5074-3bb1-4817-8855-b045cee61004-kube-api-access-nk8r7" seLinuxMountContext=""
Oct 30 18:33:55.854306 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.854168    4888 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2e49faaa-a718-44c8-9237-5a41169faee7" volumeName="kubernetes.io/projected/2e49faaa-a718-44c8-9237-5a41169faee7-kube-api-access-6llht" seLinuxMountContext=""
Oct 30 18:33:55.854306 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.854180    4888 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="63090f48-9143-4118-9906-d759c0aff52c" volumeName="kubernetes.io/configmap/63090f48-9143-4118-9906-d759c0aff52c-iptables-alerter-script" seLinuxMountContext=""
Oct 30 18:33:55.854306 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.854195    4888 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7c09088b-3a07-426d-8804-f0f5bac4d27c" volumeName="kubernetes.io/empty-dir/7c09088b-3a07-426d-8804-f0f5bac4d27c-snapshots" seLinuxMountContext=""
Oct 30 18:33:55.854306 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.854208    4888 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="15d485f9-5a4b-4f4b-9e59-a6eb062b4c5c" volumeName="kubernetes.io/projected/15d485f9-5a4b-4f4b-9e59-a6eb062b4c5c-kube-api-access-fzrkt" seLinuxMountContext=""
Oct 30 18:33:55.854306 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.854221    4888 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3f3fe764-2652-46c9-8e54-7882c0e6bcf1" volumeName="kubernetes.io/configmap/3f3fe764-2652-46c9-8e54-7882c0e6bcf1-ovnkube-script-lib" seLinuxMountContext=""
Oct 30 18:33:55.854306 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.854233    4888 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="81de5dce-c40f-4bd5-878c-5d75a0487066" volumeName="kubernetes.io/projected/81de5dce-c40f-4bd5-878c-5d75a0487066-kube-api-access-9j2qw" seLinuxMountContext=""
Oct 30 18:33:55.854306 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.854246    4888 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8789ec5b-226e-4551-bf68-3ac585006266" volumeName="kubernetes.io/configmap/8789ec5b-226e-4551-bf68-3ac585006266-registry-certificates" seLinuxMountContext=""
Oct 30 18:33:55.854306 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.854258    4888 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e6b2a584-ae78-4e80-b595-9b1a5897d556" volumeName="kubernetes.io/projected/e6b2a584-ae78-4e80-b595-9b1a5897d556-kube-api-access-24cp9" seLinuxMountContext=""
Oct 30 18:33:55.854306 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.854271    4888 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f2a80d89-61d1-4694-8734-004ce5cfb22b" volumeName="kubernetes.io/empty-dir/f2a80d89-61d1-4694-8734-004ce5cfb22b-tmp" seLinuxMountContext=""
Oct 30 18:33:55.854306 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.854301    4888 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="46a80637-6ccf-4a56-9904-f5e2fe5784ad" volumeName="kubernetes.io/projected/46a80637-6ccf-4a56-9904-f5e2fe5784ad-kube-api-access-x2978" seLinuxMountContext=""
Oct 30 18:33:55.854306 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.854315    4888 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2e49faaa-a718-44c8-9237-5a41169faee7" volumeName="kubernetes.io/configmap/2e49faaa-a718-44c8-9237-5a41169faee7-cni-binary-copy" seLinuxMountContext=""
Oct 30 18:33:55.854306 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.854327    4888 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="36a04851-81db-40cd-97ef-8e2e00a08604" volumeName="kubernetes.io/secret/36a04851-81db-40cd-97ef-8e2e00a08604-serving-cert" seLinuxMountContext=""
Oct 30 18:33:55.854306 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.854339    4888 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="382e8b31-a7fa-43f3-9f15-4c61627b0c04" volumeName="kubernetes.io/secret/382e8b31-a7fa-43f3-9f15-4c61627b0c04-serving-cert" seLinuxMountContext=""
Oct 30 18:33:55.854306 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.854353    4888 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3f3fe764-2652-46c9-8e54-7882c0e6bcf1" volumeName="kubernetes.io/configmap/3f3fe764-2652-46c9-8e54-7882c0e6bcf1-ovnkube-config" seLinuxMountContext=""
Oct 30 18:33:55.854306 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.854365    4888 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7c09088b-3a07-426d-8804-f0f5bac4d27c" volumeName="kubernetes.io/configmap/7c09088b-3a07-426d-8804-f0f5bac4d27c-trusted-ca-bundle" seLinuxMountContext=""
Oct 30 18:33:55.857951 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.854379    4888 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351" volumeName="kubernetes.io/configmap/9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351-trusted-ca" seLinuxMountContext=""
Oct 30 18:33:55.857951 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.854391    4888 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e6b2a584-ae78-4e80-b595-9b1a5897d556" volumeName="kubernetes.io/secret/e6b2a584-ae78-4e80-b595-9b1a5897d556-cert" seLinuxMountContext=""
Oct 30 18:33:55.857951 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.854410    4888 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="97b19aa3-42be-4def-8d7a-8e3516852e79" volumeName="kubernetes.io/empty-dir/97b19aa3-42be-4def-8d7a-8e3516852e79-tmp-dir" seLinuxMountContext=""
Oct 30 18:33:55.857951 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.854422    4888 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351" volumeName="kubernetes.io/projected/9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351-kube-api-access-vxw47" seLinuxMountContext=""
Oct 30 18:33:55.857951 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.854435    4888 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="97b19aa3-42be-4def-8d7a-8e3516852e79" volumeName="kubernetes.io/configmap/97b19aa3-42be-4def-8d7a-8e3516852e79-config-volume" seLinuxMountContext=""
Oct 30 18:33:55.857951 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.854446    4888 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a3cf021c-2c20-4687-9b3a-cfa65ce1ee8b" volumeName="kubernetes.io/projected/a3cf021c-2c20-4687-9b3a-cfa65ce1ee8b-kube-api-access-jt475" seLinuxMountContext=""
Oct 30 18:33:55.857951 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.854459    4888 reconstruct.go:97] "Volume reconstruction finished"
Oct 30 18:33:55.857951 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.854468    4888 reconciler.go:29] "Reconciler: start to sync state"
Oct 30 18:33:55.857951 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.857418    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-7x4rb_382e8b31-a7fa-43f3-9f15-4c61627b0c04/console-operator/1.log"
Oct 30 18:33:55.860394 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.860376    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-7x4rb_382e8b31-a7fa-43f3-9f15-4c61627b0c04/console-operator/1.log"
Oct 30 18:33:55.869349 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.869326    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xfcdl_3f3fe764-2652-46c9-8e54-7882c0e6bcf1/ovn-acl-logging/0.log"
Oct 30 18:33:55.873881 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.873860    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xfcdl_3f3fe764-2652-46c9-8e54-7882c0e6bcf1/ovn-acl-logging/0.log"
Oct 30 18:33:55.880800 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.880769    4888 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Oct 30 18:33:55.882268 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.882246    4888 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Oct 30 18:33:55.882382 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.882275    4888 status_manager.go:249] "Starting to sync pod status with apiserver"
Oct 30 18:33:55.882382 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.882337    4888 kubelet.go:2521] "Starting kubelet main sync loop"
Oct 30 18:33:55.882460 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:33:55.882381    4888 kubelet.go:2545] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]"
Oct 30 18:33:55.899533 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.899491    4888 generic.go:356] "Generic (PLEG): container finished" podID="2e49faaa-a718-44c8-9237-5a41169faee7" containerID="5b0d439590091c2234cd6b4627899cc9c5f5e50d164316600d14a4d65f8bf752" exitCode=0
Oct 30 18:33:55.899533 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.899530    4888 generic.go:356] "Generic (PLEG): container finished" podID="2e49faaa-a718-44c8-9237-5a41169faee7" containerID="4b5e52c31b6f486d330744e5a02a1e5e7cda001334c2de6ec0339b16fff4cb92" exitCode=0
Oct 30 18:33:55.899695 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.899541    4888 generic.go:356] "Generic (PLEG): container finished" podID="2e49faaa-a718-44c8-9237-5a41169faee7" containerID="5b63a8c037044e4cf0d2e567fbf7d272e9c6563846302d3a5e865b84758cfd72" exitCode=0
Oct 30 18:33:55.899695 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.899549    4888 generic.go:356] "Generic (PLEG): container finished" podID="2e49faaa-a718-44c8-9237-5a41169faee7" containerID="6b12d406f46c96167c2e4e52af445a7bae2edd96f117a61e1a2063485532dcf0" exitCode=0
Oct 30 18:33:55.899695 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.899556    4888 generic.go:356] "Generic (PLEG): container finished" podID="2e49faaa-a718-44c8-9237-5a41169faee7" containerID="d26b775e7cc6ed33fca99f6766d410d23291f151fee83292926e7f8b3980b3f7" exitCode=0
Oct 30 18:33:55.899695 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.899565    4888 generic.go:356] "Generic (PLEG): container finished" podID="2e49faaa-a718-44c8-9237-5a41169faee7" containerID="d9dfb149b80f5deffc4eabb8b6be243f2d902b34f56f7490fe6da8d321310c36" exitCode=0
Oct 30 18:33:55.915832 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.915802    4888 generic.go:356] "Generic (PLEG): container finished" podID="563184d807a5bf00cb82141d4921b071" containerID="1e65f2f361c630d925ef31c64a07e71ddb2b0a47ad1089ca0042051b32f32f1a" exitCode=0
Oct 30 18:33:55.926756 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.926731    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-7x4rb_382e8b31-a7fa-43f3-9f15-4c61627b0c04/console-operator/1.log"
Oct 30 18:33:55.926862 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.926768    4888 generic.go:356] "Generic (PLEG): container finished" podID="382e8b31-a7fa-43f3-9f15-4c61627b0c04" containerID="b7e8a752d2699b7e5514e278ff1b7c9fd161b532812a42ac3605f0cdaad0c160" exitCode=255
Oct 30 18:33:55.932765 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.932743    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xfcdl_3f3fe764-2652-46c9-8e54-7882c0e6bcf1/ovn-acl-logging/0.log"
Oct 30 18:33:55.933151 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.933122    4888 generic.go:356] "Generic (PLEG): container finished" podID="3f3fe764-2652-46c9-8e54-7882c0e6bcf1" containerID="c5c342c1ccf42795a343c08c13f829ff016aa630fb7262cbe75edbe1e6bffb55" exitCode=1
Oct 30 18:33:55.944257 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.944240    4888 manager.go:324] Recovery completed
Oct 30 18:33:55.951306 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:33:55.951261    4888 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-134-46.us-west-1.compute.internal\" not found"
Oct 30 18:33:55.962944 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.962930    4888 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 18:33:55.964006 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.963990    4888 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-46.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Oct 30 18:33:55.964084 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.964025    4888 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-46.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 18:33:55.964084 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.964040    4888 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-46.us-west-1.compute.internal" event="NodeHasSufficientPID"
Oct 30 18:33:55.965955 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.965940    4888 cpu_manager.go:226] "Starting" policy="none"
Oct 30 18:33:55.965955 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.965953    4888 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s"
Oct 30 18:33:55.966046 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.965958    4888 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 18:33:55.966046 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.965968    4888 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory"
Oct 30 18:33:55.966116 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.966093    4888 state_mem.go:94] "Updated default CPUSet" logger="CPUManager state checkpoint.CPUManager state memory" cpuSet=""
Oct 30 18:33:55.966116 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.966103    4888 state_mem.go:102] "Updated CPUSet assignments" logger="CPUManager state checkpoint.CPUManager state memory" assignments={}
Oct 30 18:33:55.966168 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.966119    4888 state_checkpoint.go:142] "restored state from checkpoint" logger="CPUManager state checkpoint"
Oct 30 18:33:55.966168 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.966125    4888 state_checkpoint.go:143] "defaultCPUSet" logger="CPUManager state checkpoint" defaultCpuSet=""
Oct 30 18:33:55.966168 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.966129    4888 policy_none.go:50] "Start"
Oct 30 18:33:55.966168 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.966138    4888 memory_manager.go:187] "Starting memorymanager" policy="None"
Oct 30 18:33:55.966168 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.966146    4888 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Oct 30 18:33:55.966310 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.966262    4888 state_mem.go:77] "Updated machine memory state" logger="Memory Manager state checkpoint"
Oct 30 18:33:55.966310 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.966269    4888 state_checkpoint.go:85] "State checkpoint: restored state from checkpoint" logger="Memory Manager state checkpoint"
Oct 30 18:33:55.966310 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.966274    4888 policy_none.go:44] "Start"
Oct 30 18:33:55.966765 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.966750    4888 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-46.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Oct 30 18:33:55.966815 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.966782    4888 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-46.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 18:33:55.966815 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.966798    4888 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-46.us-west-1.compute.internal" event="NodeHasSufficientPID"
Oct 30 18:33:55.970430 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.970407    4888 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"}
Oct 30 18:33:55.974849 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.974835    4888 manager.go:341] "Starting Device Plugin manager"
Oct 30 18:33:55.974940 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:33:55.974856    4888 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Oct 30 18:33:55.974940 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.974865    4888 server.go:87] "Starting device plugin registration server"
Oct 30 18:33:55.975098 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.975087    4888 eviction_manager.go:194] "Eviction manager: starting control loop"
Oct 30 18:33:55.975153 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.975098    4888 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Oct 30 18:33:55.975214 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.975190    4888 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Oct 30 18:33:55.975396 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.975272    4888 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Oct 30 18:33:55.975467 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.975405    4888 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts"
Oct 30 18:33:55.975467 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.975412    4888 plugin_manager.go:121] "Starting Kubelet Plugin Manager"
Oct 30 18:33:55.975568 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.975462    4888 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-10-30T18:33:55.975393915Z","UUID":"c80effe3-9cdc-4600-821c-67e988cadacf","Handler":null,"Name":"","Endpoint":""}
Oct 30 18:33:55.976612 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:33:55.976592    4888 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Oct 30 18:33:55.976703 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:33:55.976632    4888 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-134-46.us-west-1.compute.internal\" not found"
Oct 30 18:33:55.976861 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.976846    4888 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Oct 30 18:33:55.976929 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.976873    4888 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Oct 30 18:33:55.982507 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.982471    4888 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-46.us-west-1.compute.internal","kube-system/kube-apiserver-proxy-ip-10-0-134-46.us-west-1.compute.internal"]
Oct 30 18:33:55.982613 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.982600    4888 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 18:33:55.983404 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.983384    4888 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-46.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Oct 30 18:33:55.983496 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.983416    4888 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-46.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 18:33:55.983496 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.983430    4888 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-46.us-west-1.compute.internal" event="NodeHasSufficientPID"
Oct 30 18:33:55.983768 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.983754    4888 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 18:33:55.983908 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.983897    4888 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 18:33:55.984431 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.984415    4888 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-46.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Oct 30 18:33:55.984505 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.984441    4888 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-46.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 18:33:55.984505 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.984452    4888 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-46.us-west-1.compute.internal" event="NodeHasSufficientPID"
Oct 30 18:33:55.984586 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.984506    4888 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-46.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Oct 30 18:33:55.984586 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.984528    4888 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-46.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 18:33:55.984586 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.984539    4888 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-46.us-west-1.compute.internal" event="NodeHasSufficientPID"
Oct 30 18:33:55.984705 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:33:55.984691    4888 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-134-46.us-west-1.compute.internal\" not found" node="ip-10-0-134-46.us-west-1.compute.internal"
Oct 30 18:33:55.984825 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.984790    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-46.us-west-1.compute.internal" event={"ID":"6f66fae2621414751921d90f1913a614","Type":"ContainerStarted","Data":"ba00e5f5c5b06a6028795d0f094174837c924b8adeab8e75fc728e20ebc48ee8"}
Oct 30 18:33:55.984915 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.984833    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-46.us-west-1.compute.internal" event={"ID":"6f66fae2621414751921d90f1913a614","Type":"ContainerStarted","Data":"ecaf79502997d6f1f864f0f6e6427641cd2d8cc464f7f8ec8279c05e8bbc14b3"}
Oct 30 18:33:55.984915 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.984852    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-46.us-west-1.compute.internal" event={"ID":"563184d807a5bf00cb82141d4921b071","Type":"ContainerStarted","Data":"7b4b0c07a4e86d73f8c0a42fae0f01482d9a0319b3ce15504accc2abd1a861ee"}
Oct 30 18:33:55.984915 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.984866    4888 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 18:33:55.984915 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.984870    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-46.us-west-1.compute.internal" event={"ID":"563184d807a5bf00cb82141d4921b071","Type":"ContainerDied","Data":"1e65f2f361c630d925ef31c64a07e71ddb2b0a47ad1089ca0042051b32f32f1a"}
Oct 30 18:33:55.984915 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.984890    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-46.us-west-1.compute.internal" event={"ID":"563184d807a5bf00cb82141d4921b071","Type":"ContainerStarted","Data":"94eb3352db48f6acf5fa74ee0bcc8ac26482c3360cb4db2fabe3ff668bd86c13"}
Oct 30 18:33:55.985553 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.985537    4888 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-46.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Oct 30 18:33:55.985623 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.985561    4888 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-46.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 18:33:55.985623 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:55.985572    4888 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-46.us-west-1.compute.internal" event="NodeHasSufficientPID"
Oct 30 18:33:55.985736 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:33:55.985722    4888 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-134-46.us-west-1.compute.internal\" not found" node="ip-10-0-134-46.us-west-1.compute.internal"
Oct 30 18:33:56.041574 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:56.041550    4888 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 18:33:56.043122 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:56.043107    4888 reconstruct.go:205] "DevicePaths of reconstructed volumes updated"
Oct 30 18:33:56.045208 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:56.045190    4888 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 18:33:56.051427 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:56.051373    4888 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-46.us-west-1.compute.internal"
Oct 30 18:33:56.075545 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:56.075527    4888 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 18:33:56.076482 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:56.076468    4888 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-46.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Oct 30 18:33:56.076557 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:56.076496    4888 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-46.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 18:33:56.076557 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:56.076506    4888 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-46.us-west-1.compute.internal" event="NodeHasSufficientPID"
Oct 30 18:33:56.076557 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:56.076548    4888 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-134-46.us-west-1.compute.internal"
Oct 30 18:33:56.093855 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:56.093835    4888 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 18:33:56.096409 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:56.096394    4888 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 18:33:56.120331 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:56.120308    4888 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 18:33:56.120412 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:33:56.120358    4888 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-134-46.us-west-1.compute.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-46.us-west-1.compute.internal"
Oct 30 18:33:56.120412 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:56.120371    4888 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-46.us-west-1.compute.internal"
Oct 30 18:33:56.143490 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:56.143470    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/563184d807a5bf00cb82141d4921b071-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-46.us-west-1.compute.internal\" (UID: \"563184d807a5bf00cb82141d4921b071\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-46.us-west-1.compute.internal"
Oct 30 18:33:56.143592 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:56.143497    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/563184d807a5bf00cb82141d4921b071-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-46.us-west-1.compute.internal\" (UID: \"563184d807a5bf00cb82141d4921b071\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-46.us-west-1.compute.internal"
Oct 30 18:33:56.143592 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:56.143518    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6f66fae2621414751921d90f1913a614-config\") pod \"kube-apiserver-proxy-ip-10-0-134-46.us-west-1.compute.internal\" (UID: \"6f66fae2621414751921d90f1913a614\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-46.us-west-1.compute.internal"
Oct 30 18:33:56.206670 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:56.206649    4888 kubelet_node_status.go:126] "Node was previously registered" node="ip-10-0-134-46.us-west-1.compute.internal"
Oct 30 18:33:56.206755 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:56.206708    4888 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-134-46.us-west-1.compute.internal"
Oct 30 18:33:56.207244 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:56.207230    4888 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-46.us-west-1.compute.internal" event="NodeNotReady"
Oct 30 18:33:56.207334 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:56.207261    4888 setters.go:546] "Node became not ready" node="ip-10-0-134-46.us-west-1.compute.internal" condition={"type":"Ready","status":"False","lastHeartbeatTime":"2026-10-30T18:33:56Z","lastTransitionTime":"2026-10-30T18:33:56Z","reason":"KubeletNotReady","message":"CSINode is not yet initialized"}
Oct 30 18:33:56.225479 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:56.225459    4888 nodeinfomanager.go:471] existing CSINode "ip-10-0-134-46.us-west-1.compute.internal" is owned by different node (oldNodeID="69293c9c-e40d-4497-9363-c7c67bdc1c7c", newNodeID=""), cleaning up...
Oct 30 18:33:56.225749 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:56.225738    4888 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 18:33:56.225780 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:33:56.225769    4888 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-134-46.us-west-1.compute.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-46.us-west-1.compute.internal"
Oct 30 18:33:56.243906 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:56.243883    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/563184d807a5bf00cb82141d4921b071-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-46.us-west-1.compute.internal\" (UID: \"563184d807a5bf00cb82141d4921b071\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-46.us-west-1.compute.internal"
Oct 30 18:33:56.243960 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:56.243916    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/563184d807a5bf00cb82141d4921b071-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-46.us-west-1.compute.internal\" (UID: \"563184d807a5bf00cb82141d4921b071\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-46.us-west-1.compute.internal"
Oct 30 18:33:56.243960 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:56.243934    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6f66fae2621414751921d90f1913a614-config\") pod \"kube-apiserver-proxy-ip-10-0-134-46.us-west-1.compute.internal\" (UID: \"6f66fae2621414751921d90f1913a614\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-46.us-west-1.compute.internal"
Oct 30 18:33:56.244025 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:56.243955    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/563184d807a5bf00cb82141d4921b071-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-46.us-west-1.compute.internal\" (UID: \"563184d807a5bf00cb82141d4921b071\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-46.us-west-1.compute.internal"
Oct 30 18:33:56.244025 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:56.243972    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6f66fae2621414751921d90f1913a614-config\") pod \"kube-apiserver-proxy-ip-10-0-134-46.us-west-1.compute.internal\" (UID: \"6f66fae2621414751921d90f1913a614\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-46.us-west-1.compute.internal"
Oct 30 18:33:56.244025 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:56.243981    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/563184d807a5bf00cb82141d4921b071-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-46.us-west-1.compute.internal\" (UID: \"563184d807a5bf00cb82141d4921b071\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-46.us-west-1.compute.internal"
Oct 30 18:33:56.479336 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:56.479309    4888 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-46.us-west-1.compute.internal" event="NodeReady"
Oct 30 18:33:56.479488 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:56.479393    4888 kubelet_node_status.go:477] "Fast updating node status as it just became ready"
Oct 30 18:33:56.827538 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:56.827513    4888 apiserver.go:52] "Watching apiserver"
Oct 30 18:33:56.937627 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:56.937601    4888 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-46.us-west-1.compute.internal"
Oct 30 18:33:56.937769 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:56.937728    4888 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-46.us-west-1.compute.internal"
Oct 30 18:33:56.987630 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:56.987595    4888 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Oct 30 18:33:56.988248 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:56.988224    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-pm58m","openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-jwwq2","openshift-service-ca-operator/service-ca-operator-544d749c8c-kr5qq","openshift-cluster-node-tuning-operator/tuned-z9jmw","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7j9jw","openshift-kube-storage-version-migrator/migrator-66df5d496b-ht9zn","openshift-network-diagnostics/network-check-source-6f78d59f99-4969l","openshift-network-console/networking-console-plugin-86c478bb88-r8rgt","openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-5bqdc","openshift-ingress-canary/ingress-canary-g9fhp","kube-system/kas-connection-checker-685895c7c9-4b95q","openshift-monitoring/cluster-monitoring-operator-884576c57-gmswj","kube-system/konnectivity-agent-lmdpp","openshift-multus/network-metrics-daemon-v5phn","openshift-dns/dns-default-r2nx2","kube-system/kas-connection-checker-685895c7c9-f9ldt","openshift-ovn-kubernetes/ovnkube-node-xfcdl","kube-system/global-pull-secret-syncer-g2svp","openshift-console-operator/console-operator-7cd56b8dbc-7x4rb","openshift-image-registry/image-registry-64bf5b6b76-dpvvs","openshift-insights/insights-runtime-extractor-jbsw6","openshift-multus/multus-9ssmv","openshift-multus/multus-additional-cni-plugins-rzrjz","openshift-network-diagnostics/network-check-target-8xskh","kube-system/kas-connection-checker-685895c7c9-2grbp","kube-system/kube-apiserver-proxy-ip-10-0-134-46.us-west-1.compute.internal","openshift-insights/insights-operator-7c4986d449-xvgqc","openshift-image-registry/image-registry-74757bccd5-qmrkp","openshift-ingress/router-default-7dfcf9fd6c-vh769","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-46.us-west-1.compute.internal","openshift-image-registry/node-ca-g78lj","openshift-network-operator/iptables-alerter-rcjmv","openshift-dns/node-resolver-76f9s"]
Oct 30 18:33:56.989816 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:56.989793    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-7dfcf9fd6c-vh769"
Oct 30 18:33:56.990227 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:56.990199    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-gmswj"
Oct 30 18:33:56.991834 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:56.991634    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-74757bccd5-qmrkp"
Oct 30 18:33:56.992535 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:56.992514    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-pm58m"
Oct 30 18:33:56.995615 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:56.995597    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-86c478bb88-r8rgt"
Oct 30 18:33:56.997622 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:56.997606    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v5phn"
Oct 30 18:33:56.997932 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:56.997920    4888 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 18:33:56.998786 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:56.998769    4888 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 18:33:57.001453 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.001438    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-64bf5b6b76-dpvvs"
Oct 30 18:33:57.001898 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.001887    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-r2nx2"
Oct 30 18:33:57.002990 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.002977    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-jbsw6"
Oct 30 18:33:57.037711 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:33:57.037686    4888 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-134-46.us-west-1.compute.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-46.us-west-1.compute.internal"
Oct 30 18:33:57.037807 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:33:57.037728    4888 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-134-46.us-west-1.compute.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-46.us-west-1.compute.internal"
Oct 30 18:33:57.052481 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.052460    4888 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Oct 30 18:33:57.055149 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.055133    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"openshift-service-ca.crt\""
Oct 30 18:33:57.055231 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.055170    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"kube-root-ca.crt\""
Oct 30 18:33:57.055771 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.055755    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Oct 30 18:33:57.055989 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.055978    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Oct 30 18:33:57.057016 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.057003    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Oct 30 18:33:57.057062 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.057029    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Oct 30 18:33:57.057275 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.057263    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Oct 30 18:33:57.057535 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.057524    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-qv4ml\""
Oct 30 18:33:57.057574 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.057538    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-blpnv\""
Oct 30 18:33:57.066343 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.066325    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Oct 30 18:33:57.066682 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.066661    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Oct 30 18:33:57.066788 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.066654    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Oct 30 18:33:57.066788 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.066699    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Oct 30 18:33:57.066788 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.066704    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Oct 30 18:33:57.066911 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.066857    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-lh6d4\""
Oct 30 18:33:57.066973 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.066932    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-rqt8c\""
Oct 30 18:33:57.066973 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.066964    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Oct 30 18:33:57.067060 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.066965    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-c65fk\""
Oct 30 18:33:57.067060 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.066998    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Oct 30 18:33:57.067060 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.067025    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-b9rv9\""
Oct 30 18:33:57.067060 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.066998    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-brwnt\""
Oct 30 18:33:57.067616 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.067602    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Oct 30 18:33:57.076315 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.076278    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-config\""
Oct 30 18:33:57.076482 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.076467    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Oct 30 18:33:57.076548 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.076496    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Oct 30 18:33:57.076548 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.076510    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Oct 30 18:33:57.076548 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.076534    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Oct 30 18:33:57.076548 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.076536    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Oct 30 18:33:57.076742 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.076585    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Oct 30 18:33:57.076742 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.076614    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Oct 30 18:33:57.076742 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.076630    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Oct 30 18:33:57.076742 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.076659    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Oct 30 18:33:57.076742 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.076699    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Oct 30 18:33:57.076742 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.076736    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Oct 30 18:33:57.076962 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.076751    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Oct 30 18:33:57.076962 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.076764    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Oct 30 18:33:57.076962 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.076785    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Oct 30 18:33:57.118332 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.118248    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Oct 30 18:33:57.118332 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.118265    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Oct 30 18:33:57.118492 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.118352    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Oct 30 18:33:57.118492 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.118446    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Oct 30 18:33:57.118492 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.118450    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Oct 30 18:33:57.118492 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.118468    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Oct 30 18:33:57.120255 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.120221    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Oct 30 18:33:57.121191 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.121172    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Oct 30 18:33:57.150216 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.150190    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/15d485f9-5a4b-4f4b-9e59-a6eb062b4c5c-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-pm58m\" (UID: \"15d485f9-5a4b-4f4b-9e59-a6eb062b4c5c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-pm58m"
Oct 30 18:33:57.150359 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.150224    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cvxhl\" (UniqueName: \"kubernetes.io/projected/7c09088b-3a07-426d-8804-f0f5bac4d27c-kube-api-access-cvxhl\") pod \"insights-operator-7c4986d449-xvgqc\" (UID: \"7c09088b-3a07-426d-8804-f0f5bac4d27c\") " pod="openshift-insights/insights-operator-7c4986d449-xvgqc"
Oct 30 18:33:57.150359 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.150269    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3f3fe764-2652-46c9-8e54-7882c0e6bcf1-etc-openvswitch\") pod \"ovnkube-node-xfcdl\" (UID: \"3f3fe764-2652-46c9-8e54-7882c0e6bcf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xfcdl"
Oct 30 18:33:57.150359 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.150315    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351-registry-certificates\") pod \"image-registry-64bf5b6b76-dpvvs\" (UID: \"9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351\") " pod="openshift-image-registry/image-registry-64bf5b6b76-dpvvs"
Oct 30 18:33:57.150359 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.150333    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/36a04851-81db-40cd-97ef-8e2e00a08604-serving-cert\") pod \"service-ca-operator-544d749c8c-kr5qq\" (UID: \"36a04851-81db-40cd-97ef-8e2e00a08604\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-kr5qq"
Oct 30 18:33:57.150359 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.150351    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2e49faaa-a718-44c8-9237-5a41169faee7-cnibin\") pod \"multus-additional-cni-plugins-rzrjz\" (UID: \"2e49faaa-a718-44c8-9237-5a41169faee7\") " pod="openshift-multus/multus-additional-cni-plugins-rzrjz"
Oct 30 18:33:57.150528 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.150365    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f2a80d89-61d1-4694-8734-004ce5cfb22b-etc-sysconfig\") pod \"tuned-z9jmw\" (UID: \"f2a80d89-61d1-4694-8734-004ce5cfb22b\") " pod="openshift-cluster-node-tuning-operator/tuned-z9jmw"
Oct 30 18:33:57.150528 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.150380    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f2a80d89-61d1-4694-8734-004ce5cfb22b-sys\") pod \"tuned-z9jmw\" (UID: \"f2a80d89-61d1-4694-8734-004ce5cfb22b\") " pod="openshift-cluster-node-tuning-operator/tuned-z9jmw"
Oct 30 18:33:57.150528 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.150454    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/71fce796-0032-481b-80b3-66fbb731452c-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-r8rgt\" (UID: \"71fce796-0032-481b-80b3-66fbb731452c\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-r8rgt"
Oct 30 18:33:57.150643 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.150535    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x4hfg\" (UniqueName: \"kubernetes.io/projected/e77c4479-d6e5-48a8-ac82-102a9f1a2c4a-kube-api-access-x4hfg\") pod \"kas-connection-checker-685895c7c9-2grbp\" (UID: \"e77c4479-d6e5-48a8-ac82-102a9f1a2c4a\") " pod="kube-system/kas-connection-checker-685895c7c9-2grbp"
Oct 30 18:33:57.150643 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.150572    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/57ca8600-79e7-4f59-9d11-b1c9ec3d358a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jbsw6\" (UID: \"57ca8600-79e7-4f59-9d11-b1c9ec3d358a\") " pod="openshift-insights/insights-runtime-extractor-jbsw6"
Oct 30 18:33:57.150643 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.150588    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351-registry-certificates\") pod \"image-registry-64bf5b6b76-dpvvs\" (UID: \"9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351\") " pod="openshift-image-registry/image-registry-64bf5b6b76-dpvvs"
Oct 30 18:33:57.150643 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.150599    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/09c13759-50d4-40e6-b854-4691822b8c8a-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-5bqdc\" (UID: \"09c13759-50d4-40e6-b854-4691822b8c8a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-5bqdc"
Oct 30 18:33:57.150643 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.150627    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9j2qw\" (UniqueName: \"kubernetes.io/projected/81de5dce-c40f-4bd5-878c-5d75a0487066-kube-api-access-9j2qw\") pod \"network-check-source-6f78d59f99-4969l\" (UID: \"81de5dce-c40f-4bd5-878c-5d75a0487066\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-4969l"
Oct 30 18:33:57.150643 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.150629    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/36a04851-81db-40cd-97ef-8e2e00a08604-serving-cert\") pod \"service-ca-operator-544d749c8c-kr5qq\" (UID: \"36a04851-81db-40cd-97ef-8e2e00a08604\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-kr5qq"
Oct 30 18:33:57.150828 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.150674    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c45qr\" (UniqueName: \"kubernetes.io/projected/613020c0-f3e4-4650-896f-748a40f0ab8b-kube-api-access-c45qr\") pod \"network-check-target-8xskh\" (UID: \"613020c0-f3e4-4650-896f-748a40f0ab8b\") " pod="openshift-network-diagnostics/network-check-target-8xskh"
Oct 30 18:33:57.150828 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.150735    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f2a80d89-61d1-4694-8734-004ce5cfb22b-etc-modprobe-d\") pod \"tuned-z9jmw\" (UID: \"f2a80d89-61d1-4694-8734-004ce5cfb22b\") " pod="openshift-cluster-node-tuning-operator/tuned-z9jmw"
Oct 30 18:33:57.150828 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.150815    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7c09088b-3a07-426d-8804-f0f5bac4d27c-service-ca-bundle\") pod \"insights-operator-7c4986d449-xvgqc\" (UID: \"7c09088b-3a07-426d-8804-f0f5bac4d27c\") " pod="openshift-insights/insights-operator-7c4986d449-xvgqc"
Oct 30 18:33:57.150941 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.150819    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/09c13759-50d4-40e6-b854-4691822b8c8a-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-5bqdc\" (UID: \"09c13759-50d4-40e6-b854-4691822b8c8a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-5bqdc"
Oct 30 18:33:57.150941 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.150849    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/68195e19-b1ff-4148-9fae-3545f2997ff5-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-gmswj\" (UID: \"68195e19-b1ff-4148-9fae-3545f2997ff5\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-gmswj"
Oct 30 18:33:57.150941 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.150878    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d81cef55-16ee-4677-ae12-86de3c9eb216-hosts-file\") pod \"node-resolver-76f9s\" (UID: \"d81cef55-16ee-4677-ae12-86de3c9eb216\") " pod="openshift-dns/node-resolver-76f9s"
Oct 30 18:33:57.150941 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.150905    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/da28351c-1c8b-40be-85a0-3aad427e3a17-stats-auth\") pod \"router-default-7dfcf9fd6c-vh769\" (UID: \"da28351c-1c8b-40be-85a0-3aad427e3a17\") " pod="openshift-ingress/router-default-7dfcf9fd6c-vh769"
Oct 30 18:33:57.151071 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.150944    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/382e8b31-a7fa-43f3-9f15-4c61627b0c04-config\") pod \"console-operator-7cd56b8dbc-7x4rb\" (UID: \"382e8b31-a7fa-43f3-9f15-4c61627b0c04\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-7x4rb"
Oct 30 18:33:57.151071 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.150972    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/63090f48-9143-4118-9906-d759c0aff52c-iptables-alerter-script\") pod \"iptables-alerter-rcjmv\" (UID: \"63090f48-9143-4118-9906-d759c0aff52c\") " pod="openshift-network-operator/iptables-alerter-rcjmv"
Oct 30 18:33:57.151071 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.150997    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/da28351c-1c8b-40be-85a0-3aad427e3a17-default-certificate\") pod \"router-default-7dfcf9fd6c-vh769\" (UID: \"da28351c-1c8b-40be-85a0-3aad427e3a17\") " pod="openshift-ingress/router-default-7dfcf9fd6c-vh769"
Oct 30 18:33:57.151071 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.151021    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7c09088b-3a07-426d-8804-f0f5bac4d27c-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-xvgqc\" (UID: \"7c09088b-3a07-426d-8804-f0f5bac4d27c\") " pod="openshift-insights/insights-operator-7c4986d449-xvgqc"
Oct 30 18:33:57.151071 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.151048    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vxw47\" (UniqueName: \"kubernetes.io/projected/9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351-kube-api-access-vxw47\") pod \"image-registry-64bf5b6b76-dpvvs\" (UID: \"9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351\") " pod="openshift-image-registry/image-registry-64bf5b6b76-dpvvs"
Oct 30 18:33:57.151225 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.151077    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/57ca8600-79e7-4f59-9d11-b1c9ec3d358a-data-volume\") pod \"insights-runtime-extractor-jbsw6\" (UID: \"57ca8600-79e7-4f59-9d11-b1c9ec3d358a\") " pod="openshift-insights/insights-runtime-extractor-jbsw6"
Oct 30 18:33:57.151225 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.151128    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/da28351c-1c8b-40be-85a0-3aad427e3a17-stats-auth\") pod \"router-default-7dfcf9fd6c-vh769\" (UID: \"da28351c-1c8b-40be-85a0-3aad427e3a17\") " pod="openshift-ingress/router-default-7dfcf9fd6c-vh769"
Oct 30 18:33:57.151225 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.151152    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/382e8b31-a7fa-43f3-9f15-4c61627b0c04-config\") pod \"console-operator-7cd56b8dbc-7x4rb\" (UID: \"382e8b31-a7fa-43f3-9f15-4c61627b0c04\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-7x4rb"
Oct 30 18:33:57.151225 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.151154    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/57ca8600-79e7-4f59-9d11-b1c9ec3d358a-data-volume\") pod \"insights-runtime-extractor-jbsw6\" (UID: \"57ca8600-79e7-4f59-9d11-b1c9ec3d358a\") " pod="openshift-insights/insights-runtime-extractor-jbsw6"
Oct 30 18:33:57.151382 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.151228    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cftdb\" (UniqueName: \"kubernetes.io/projected/da28351c-1c8b-40be-85a0-3aad427e3a17-kube-api-access-cftdb\") pod \"router-default-7dfcf9fd6c-vh769\" (UID: \"da28351c-1c8b-40be-85a0-3aad427e3a17\") " pod="openshift-ingress/router-default-7dfcf9fd6c-vh769"
Oct 30 18:33:57.151382 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.151239    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/da28351c-1c8b-40be-85a0-3aad427e3a17-default-certificate\") pod \"router-default-7dfcf9fd6c-vh769\" (UID: \"da28351c-1c8b-40be-85a0-3aad427e3a17\") " pod="openshift-ingress/router-default-7dfcf9fd6c-vh769"
Oct 30 18:33:57.151382 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.151246    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ad78ad05-b0af-4adf-a6bc-0f993b8a6288-socket-dir\") pod \"aws-ebs-csi-driver-node-7j9jw\" (UID: \"ad78ad05-b0af-4adf-a6bc-0f993b8a6288\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7j9jw"
Oct 30 18:33:57.151382 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.151310    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/2e49faaa-a718-44c8-9237-5a41169faee7-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-rzrjz\" (UID: \"2e49faaa-a718-44c8-9237-5a41169faee7\") " pod="openshift-multus/multus-additional-cni-plugins-rzrjz"
Oct 30 18:33:57.151382 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.151341    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351-bound-sa-token\") pod \"image-registry-64bf5b6b76-dpvvs\" (UID: \"9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351\") " pod="openshift-image-registry/image-registry-64bf5b6b76-dpvvs"
Oct 30 18:33:57.151382 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.151379    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x2978\" (UniqueName: \"kubernetes.io/projected/46a80637-6ccf-4a56-9904-f5e2fe5784ad-kube-api-access-x2978\") pod \"node-ca-g78lj\" (UID: \"46a80637-6ccf-4a56-9904-f5e2fe5784ad\") " pod="openshift-image-registry/node-ca-g78lj"
Oct 30 18:33:57.151696 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.151405    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/aa3f3b62-a1ab-4113-bef3-ed1b290e47c1-host-var-lib-cni-bin\") pod \"multus-9ssmv\" (UID: \"aa3f3b62-a1ab-4113-bef3-ed1b290e47c1\") " pod="openshift-multus/multus-9ssmv"
Oct 30 18:33:57.151696 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.151430    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rvrxn\" (UniqueName: \"kubernetes.io/projected/aa3f3b62-a1ab-4113-bef3-ed1b290e47c1-kube-api-access-rvrxn\") pod \"multus-9ssmv\" (UID: \"aa3f3b62-a1ab-4113-bef3-ed1b290e47c1\") " pod="openshift-multus/multus-9ssmv"
Oct 30 18:33:57.151696 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.151456    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rw8g9\" (UniqueName: \"kubernetes.io/projected/36a04851-81db-40cd-97ef-8e2e00a08604-kube-api-access-rw8g9\") pod \"service-ca-operator-544d749c8c-kr5qq\" (UID: \"36a04851-81db-40cd-97ef-8e2e00a08604\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-kr5qq"
Oct 30 18:33:57.151696 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.151522    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/382e8b31-a7fa-43f3-9f15-4c61627b0c04-serving-cert\") pod \"console-operator-7cd56b8dbc-7x4rb\" (UID: \"382e8b31-a7fa-43f3-9f15-4c61627b0c04\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-7x4rb"
Oct 30 18:33:57.151696 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.151561    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ad78ad05-b0af-4adf-a6bc-0f993b8a6288-registration-dir\") pod \"aws-ebs-csi-driver-node-7j9jw\" (UID: \"ad78ad05-b0af-4adf-a6bc-0f993b8a6288\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7j9jw"
Oct 30 18:33:57.151696 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.151592    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8789ec5b-226e-4551-bf68-3ac585006266-bound-sa-token\") pod \"image-registry-74757bccd5-qmrkp\" (UID: \"8789ec5b-226e-4551-bf68-3ac585006266\") " pod="openshift-image-registry/image-registry-74757bccd5-qmrkp"
Oct 30 18:33:57.151696 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.151630    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351-registry-tls\") pod \"image-registry-64bf5b6b76-dpvvs\" (UID: \"9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351\") " pod="openshift-image-registry/image-registry-64bf5b6b76-dpvvs"
Oct 30 18:33:57.151696 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.151651    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/aa3f3b62-a1ab-4113-bef3-ed1b290e47c1-multus-socket-dir-parent\") pod \"multus-9ssmv\" (UID: \"aa3f3b62-a1ab-4113-bef3-ed1b290e47c1\") " pod="openshift-multus/multus-9ssmv"
Oct 30 18:33:57.151696 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.151672    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bgxbg\" (UniqueName: \"kubernetes.io/projected/63090f48-9143-4118-9906-d759c0aff52c-kube-api-access-bgxbg\") pod \"iptables-alerter-rcjmv\" (UID: \"63090f48-9143-4118-9906-d759c0aff52c\") " pod="openshift-network-operator/iptables-alerter-rcjmv"
Oct 30 18:33:57.151696 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.151675    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/382e8b31-a7fa-43f3-9f15-4c61627b0c04-serving-cert\") pod \"console-operator-7cd56b8dbc-7x4rb\" (UID: \"382e8b31-a7fa-43f3-9f15-4c61627b0c04\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-7x4rb"
Oct 30 18:33:57.152142 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.151706    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kk8qr\" (UniqueName: \"kubernetes.io/projected/f4ac17da-6b5b-4441-9b4e-ebe696e06507-kube-api-access-kk8qr\") pod \"volume-data-source-validator-58c9f5f46d-jwwq2\" (UID: \"f4ac17da-6b5b-4441-9b4e-ebe696e06507\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-jwwq2"
Oct 30 18:33:57.152142 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.151733    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k5crl\" (UniqueName: \"kubernetes.io/projected/09c13759-50d4-40e6-b854-4691822b8c8a-kube-api-access-k5crl\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-5bqdc\" (UID: \"09c13759-50d4-40e6-b854-4691822b8c8a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-5bqdc"
Oct 30 18:33:57.152142 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.151749    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/15d485f9-5a4b-4f4b-9e59-a6eb062b4c5c-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-pm58m\" (UID: \"15d485f9-5a4b-4f4b-9e59-a6eb062b4c5c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-pm58m"
Oct 30 18:33:57.152142 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.151771    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/aa3f3b62-a1ab-4113-bef3-ed1b290e47c1-etc-kubernetes\") pod \"multus-9ssmv\" (UID: \"aa3f3b62-a1ab-4113-bef3-ed1b290e47c1\") " pod="openshift-multus/multus-9ssmv"
Oct 30 18:33:57.152142 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.151809    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5f2ls\" (UniqueName: \"kubernetes.io/projected/ad78ad05-b0af-4adf-a6bc-0f993b8a6288-kube-api-access-5f2ls\") pod \"aws-ebs-csi-driver-node-7j9jw\" (UID: \"ad78ad05-b0af-4adf-a6bc-0f993b8a6288\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7j9jw"
Oct 30 18:33:57.152142 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.151834    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8789ec5b-226e-4551-bf68-3ac585006266-ca-trust-extracted\") pod \"image-registry-74757bccd5-qmrkp\" (UID: \"8789ec5b-226e-4551-bf68-3ac585006266\") " pod="openshift-image-registry/image-registry-74757bccd5-qmrkp"
Oct 30 18:33:57.152142 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.151861    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8789ec5b-226e-4551-bf68-3ac585006266-installation-pull-secrets\") pod \"image-registry-74757bccd5-qmrkp\" (UID: \"8789ec5b-226e-4551-bf68-3ac585006266\") " pod="openshift-image-registry/image-registry-74757bccd5-qmrkp"
Oct 30 18:33:57.152142 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.151885    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/97b19aa3-42be-4def-8d7a-8e3516852e79-config-volume\") pod \"dns-default-r2nx2\" (UID: \"97b19aa3-42be-4def-8d7a-8e3516852e79\") " pod="openshift-dns/dns-default-r2nx2"
Oct 30 18:33:57.152142 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.151939    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8789ec5b-226e-4551-bf68-3ac585006266-ca-trust-extracted\") pod \"image-registry-74757bccd5-qmrkp\" (UID: \"8789ec5b-226e-4551-bf68-3ac585006266\") " pod="openshift-image-registry/image-registry-74757bccd5-qmrkp"
Oct 30 18:33:57.152142 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.151966    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/aa3f3b62-a1ab-4113-bef3-ed1b290e47c1-host-run-netns\") pod \"multus-9ssmv\" (UID: \"aa3f3b62-a1ab-4113-bef3-ed1b290e47c1\") " pod="openshift-multus/multus-9ssmv"
Oct 30 18:33:57.152142 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.151990    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/15d485f9-5a4b-4f4b-9e59-a6eb062b4c5c-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-pm58m\" (UID: \"15d485f9-5a4b-4f4b-9e59-a6eb062b4c5c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-pm58m"
Oct 30 18:33:57.152142 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.152006    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8789ec5b-226e-4551-bf68-3ac585006266-registry-certificates\") pod \"image-registry-74757bccd5-qmrkp\" (UID: \"8789ec5b-226e-4551-bf68-3ac585006266\") " pod="openshift-image-registry/image-registry-74757bccd5-qmrkp"
Oct 30 18:33:57.152142 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.152046    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3f3fe764-2652-46c9-8e54-7882c0e6bcf1-host-run-ovn-kubernetes\") pod \"ovnkube-node-xfcdl\" (UID: \"3f3fe764-2652-46c9-8e54-7882c0e6bcf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xfcdl"
Oct 30 18:33:57.152142 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.152053    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8789ec5b-226e-4551-bf68-3ac585006266-installation-pull-secrets\") pod \"image-registry-74757bccd5-qmrkp\" (UID: \"8789ec5b-226e-4551-bf68-3ac585006266\") " pod="openshift-image-registry/image-registry-74757bccd5-qmrkp"
Oct 30 18:33:57.152548 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.152153    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8789ec5b-226e-4551-bf68-3ac585006266-registry-certificates\") pod \"image-registry-74757bccd5-qmrkp\" (UID: \"8789ec5b-226e-4551-bf68-3ac585006266\") " pod="openshift-image-registry/image-registry-74757bccd5-qmrkp"
Oct 30 18:33:57.152548 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.152152    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3f3fe764-2652-46c9-8e54-7882c0e6bcf1-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-xfcdl\" (UID: \"3f3fe764-2652-46c9-8e54-7882c0e6bcf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xfcdl"
Oct 30 18:33:57.152548 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.152189    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/3f3fe764-2652-46c9-8e54-7882c0e6bcf1-ovn-node-metrics-cert\") pod \"ovnkube-node-xfcdl\" (UID: \"3f3fe764-2652-46c9-8e54-7882c0e6bcf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xfcdl"
Oct 30 18:33:57.152548 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.152220    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/aa3f3b62-a1ab-4113-bef3-ed1b290e47c1-host-run-k8s-cni-cncf-io\") pod \"multus-9ssmv\" (UID: \"aa3f3b62-a1ab-4113-bef3-ed1b290e47c1\") " pod="openshift-multus/multus-9ssmv"
Oct 30 18:33:57.152548 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.152325    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ad78ad05-b0af-4adf-a6bc-0f993b8a6288-sys-fs\") pod \"aws-ebs-csi-driver-node-7j9jw\" (UID: \"ad78ad05-b0af-4adf-a6bc-0f993b8a6288\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7j9jw"
Oct 30 18:33:57.152548 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.152360    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/63090f48-9143-4118-9906-d759c0aff52c-host-slash\") pod \"iptables-alerter-rcjmv\" (UID: \"63090f48-9143-4118-9906-d759c0aff52c\") " pod="openshift-network-operator/iptables-alerter-rcjmv"
Oct 30 18:33:57.152548 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.152392    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f2a80d89-61d1-4694-8734-004ce5cfb22b-etc-sysctl-d\") pod \"tuned-z9jmw\" (UID: \"f2a80d89-61d1-4694-8734-004ce5cfb22b\") " pod="openshift-cluster-node-tuning-operator/tuned-z9jmw"
Oct 30 18:33:57.152548 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.152421    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vf948\" (UniqueName: \"kubernetes.io/projected/8789ec5b-226e-4551-bf68-3ac585006266-kube-api-access-vf948\") pod \"image-registry-74757bccd5-qmrkp\" (UID: \"8789ec5b-226e-4551-bf68-3ac585006266\") " pod="openshift-image-registry/image-registry-74757bccd5-qmrkp"
Oct 30 18:33:57.152548 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.152454    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351-image-registry-private-configuration\") pod \"image-registry-64bf5b6b76-dpvvs\" (UID: \"9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351\") " pod="openshift-image-registry/image-registry-64bf5b6b76-dpvvs"
Oct 30 18:33:57.152548 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.152480    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351-trusted-ca\") pod \"image-registry-64bf5b6b76-dpvvs\" (UID: \"9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351\") " pod="openshift-image-registry/image-registry-64bf5b6b76-dpvvs"
Oct 30 18:33:57.152548 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.152510    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f2a80d89-61d1-4694-8734-004ce5cfb22b-etc-sysctl-conf\") pod \"tuned-z9jmw\" (UID: \"f2a80d89-61d1-4694-8734-004ce5cfb22b\") " pod="openshift-cluster-node-tuning-operator/tuned-z9jmw"
Oct 30 18:33:57.153078 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.152552    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nk8r7\" (UniqueName: \"kubernetes.io/projected/df9e5074-3bb1-4817-8855-b045cee61004-kube-api-access-nk8r7\") pod \"kas-connection-checker-685895c7c9-f9ldt\" (UID: \"df9e5074-3bb1-4817-8855-b045cee61004\") " pod="kube-system/kas-connection-checker-685895c7c9-f9ldt"
Oct 30 18:33:57.153078 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.152581    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/3f3fe764-2652-46c9-8e54-7882c0e6bcf1-systemd-units\") pod \"ovnkube-node-xfcdl\" (UID: \"3f3fe764-2652-46c9-8e54-7882c0e6bcf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xfcdl"
Oct 30 18:33:57.153078 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.152607    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3f3fe764-2652-46c9-8e54-7882c0e6bcf1-host-slash\") pod \"ovnkube-node-xfcdl\" (UID: \"3f3fe764-2652-46c9-8e54-7882c0e6bcf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xfcdl"
Oct 30 18:33:57.153078 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.152634    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/3f3fe764-2652-46c9-8e54-7882c0e6bcf1-run-systemd\") pod \"ovnkube-node-xfcdl\" (UID: \"3f3fe764-2652-46c9-8e54-7882c0e6bcf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xfcdl"
Oct 30 18:33:57.153078 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.152645    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351-image-registry-private-configuration\") pod \"image-registry-64bf5b6b76-dpvvs\" (UID: \"9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351\") " pod="openshift-image-registry/image-registry-64bf5b6b76-dpvvs"
Oct 30 18:33:57.153078 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.152658    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/3f3fe764-2652-46c9-8e54-7882c0e6bcf1-node-log\") pod \"ovnkube-node-xfcdl\" (UID: \"3f3fe764-2652-46c9-8e54-7882c0e6bcf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xfcdl"
Oct 30 18:33:57.153078 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.152690    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/46d9ac39-ee45-4e43-bda7-3931c3e599cf-original-pull-secret\") pod \"global-pull-secret-syncer-g2svp\" (UID: \"46d9ac39-ee45-4e43-bda7-3931c3e599cf\") " pod="kube-system/global-pull-secret-syncer-g2svp"
Oct 30 18:33:57.153078 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.152726    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tc2fp\" (UniqueName: \"kubernetes.io/projected/97b19aa3-42be-4def-8d7a-8e3516852e79-kube-api-access-tc2fp\") pod \"dns-default-r2nx2\" (UID: \"97b19aa3-42be-4def-8d7a-8e3516852e79\") " pod="openshift-dns/dns-default-r2nx2"
Oct 30 18:33:57.153078 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.152775    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2e49faaa-a718-44c8-9237-5a41169faee7-tuning-conf-dir\") pod \"multus-additional-cni-plugins-rzrjz\" (UID: \"2e49faaa-a718-44c8-9237-5a41169faee7\") " pod="openshift-multus/multus-additional-cni-plugins-rzrjz"
Oct 30 18:33:57.153078 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.152794    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/97b19aa3-42be-4def-8d7a-8e3516852e79-tmp-dir\") pod \"dns-default-r2nx2\" (UID: \"97b19aa3-42be-4def-8d7a-8e3516852e79\") " pod="openshift-dns/dns-default-r2nx2"
Oct 30 18:33:57.153078 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.152794    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351-trusted-ca\") pod \"image-registry-64bf5b6b76-dpvvs\" (UID: \"9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351\") " pod="openshift-image-registry/image-registry-64bf5b6b76-dpvvs"
Oct 30 18:33:57.153078 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.152834    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/68195e19-b1ff-4148-9fae-3545f2997ff5-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-gmswj\" (UID: \"68195e19-b1ff-4148-9fae-3545f2997ff5\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-gmswj"
Oct 30 18:33:57.153078 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.152869    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2e49faaa-a718-44c8-9237-5a41169faee7-cni-binary-copy\") pod \"multus-additional-cni-plugins-rzrjz\" (UID: \"2e49faaa-a718-44c8-9237-5a41169faee7\") " pod="openshift-multus/multus-additional-cni-plugins-rzrjz"
Oct 30 18:33:57.153078 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.152875    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/97b19aa3-42be-4def-8d7a-8e3516852e79-tmp-dir\") pod \"dns-default-r2nx2\" (UID: \"97b19aa3-42be-4def-8d7a-8e3516852e79\") " pod="openshift-dns/dns-default-r2nx2"
Oct 30 18:33:57.153078 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.152904    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/3f3fe764-2652-46c9-8e54-7882c0e6bcf1-log-socket\") pod \"ovnkube-node-xfcdl\" (UID: \"3f3fe764-2652-46c9-8e54-7882c0e6bcf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xfcdl"
Oct 30 18:33:57.153078 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.152931    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8789ec5b-226e-4551-bf68-3ac585006266-image-registry-private-configuration\") pod \"image-registry-74757bccd5-qmrkp\" (UID: \"8789ec5b-226e-4551-bf68-3ac585006266\") " pod="openshift-image-registry/image-registry-74757bccd5-qmrkp"
Oct 30 18:33:57.153078 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.152956    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8789ec5b-226e-4551-bf68-3ac585006266-trusted-ca\") pod \"image-registry-74757bccd5-qmrkp\" (UID: \"8789ec5b-226e-4551-bf68-3ac585006266\") " pod="openshift-image-registry/image-registry-74757bccd5-qmrkp"
Oct 30 18:33:57.153078 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.152996    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/46a80637-6ccf-4a56-9904-f5e2fe5784ad-host\") pod \"node-ca-g78lj\" (UID: \"46a80637-6ccf-4a56-9904-f5e2fe5784ad\") " pod="openshift-image-registry/node-ca-g78lj"
Oct 30 18:33:57.153078 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.153019    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gzhnk\" (UniqueName: \"kubernetes.io/projected/d81cef55-16ee-4677-ae12-86de3c9eb216-kube-api-access-gzhnk\") pod \"node-resolver-76f9s\" (UID: \"d81cef55-16ee-4677-ae12-86de3c9eb216\") " pod="openshift-dns/node-resolver-76f9s"
Oct 30 18:33:57.153078 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.153035    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/3f3fe764-2652-46c9-8e54-7882c0e6bcf1-run-ovn\") pod \"ovnkube-node-xfcdl\" (UID: \"3f3fe764-2652-46c9-8e54-7882c0e6bcf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xfcdl"
Oct 30 18:33:57.153078 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.153039    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/68195e19-b1ff-4148-9fae-3545f2997ff5-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-gmswj\" (UID: \"68195e19-b1ff-4148-9fae-3545f2997ff5\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-gmswj"
Oct 30 18:33:57.153078 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.153058    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/3f3fe764-2652-46c9-8e54-7882c0e6bcf1-env-overrides\") pod \"ovnkube-node-xfcdl\" (UID: \"3f3fe764-2652-46c9-8e54-7882c0e6bcf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xfcdl"
Oct 30 18:33:57.153787 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.153150    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jt475\" (UniqueName: \"kubernetes.io/projected/a3cf021c-2c20-4687-9b3a-cfa65ce1ee8b-kube-api-access-jt475\") pod \"migrator-66df5d496b-ht9zn\" (UID: \"a3cf021c-2c20-4687-9b3a-cfa65ce1ee8b\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-ht9zn"
Oct 30 18:33:57.153787 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.153188    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f2a80d89-61d1-4694-8734-004ce5cfb22b-etc-tuned\") pod \"tuned-z9jmw\" (UID: \"f2a80d89-61d1-4694-8734-004ce5cfb22b\") " pod="openshift-cluster-node-tuning-operator/tuned-z9jmw"
Oct 30 18:33:57.153787 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.153197    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8789ec5b-226e-4551-bf68-3ac585006266-image-registry-private-configuration\") pod \"image-registry-74757bccd5-qmrkp\" (UID: \"8789ec5b-226e-4551-bf68-3ac585006266\") " pod="openshift-image-registry/image-registry-74757bccd5-qmrkp"
Oct 30 18:33:57.153787 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.153304    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/3f3fe764-2652-46c9-8e54-7882c0e6bcf1-host-cni-netd\") pod \"ovnkube-node-xfcdl\" (UID: \"3f3fe764-2652-46c9-8e54-7882c0e6bcf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xfcdl"
Oct 30 18:33:57.153787 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.153325    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f2a80d89-61d1-4694-8734-004ce5cfb22b-etc-tuned\") pod \"tuned-z9jmw\" (UID: \"f2a80d89-61d1-4694-8734-004ce5cfb22b\") " pod="openshift-cluster-node-tuning-operator/tuned-z9jmw"
Oct 30 18:33:57.153787 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.153328    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351-installation-pull-secrets\") pod \"image-registry-64bf5b6b76-dpvvs\" (UID: \"9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351\") " pod="openshift-image-registry/image-registry-64bf5b6b76-dpvvs"
Oct 30 18:33:57.153787 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.153367    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a0823e8f-bbdd-477b-95ac-dad05d23cefa-metrics-certs\") pod \"network-metrics-daemon-v5phn\" (UID: \"a0823e8f-bbdd-477b-95ac-dad05d23cefa\") " pod="openshift-multus/network-metrics-daemon-v5phn"
Oct 30 18:33:57.153787 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.153382    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8789ec5b-226e-4551-bf68-3ac585006266-trusted-ca\") pod \"image-registry-74757bccd5-qmrkp\" (UID: \"8789ec5b-226e-4551-bf68-3ac585006266\") " pod="openshift-image-registry/image-registry-74757bccd5-qmrkp"
Oct 30 18:33:57.153787 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.153447    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f2a80d89-61d1-4694-8734-004ce5cfb22b-etc-kubernetes\") pod \"tuned-z9jmw\" (UID: \"f2a80d89-61d1-4694-8734-004ce5cfb22b\") " pod="openshift-cluster-node-tuning-operator/tuned-z9jmw"
Oct 30 18:33:57.153787 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.153471    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3f3fe764-2652-46c9-8e54-7882c0e6bcf1-var-lib-openvswitch\") pod \"ovnkube-node-xfcdl\" (UID: \"3f3fe764-2652-46c9-8e54-7882c0e6bcf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xfcdl"
Oct 30 18:33:57.153787 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.153497    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/aa3f3b62-a1ab-4113-bef3-ed1b290e47c1-system-cni-dir\") pod \"multus-9ssmv\" (UID: \"aa3f3b62-a1ab-4113-bef3-ed1b290e47c1\") " pod="openshift-multus/multus-9ssmv"
Oct 30 18:33:57.153787 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.153496    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351-installation-pull-secrets\") pod \"image-registry-64bf5b6b76-dpvvs\" (UID: \"9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351\") " pod="openshift-image-registry/image-registry-64bf5b6b76-dpvvs"
Oct 30 18:33:57.153787 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.153534    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/aa3f3b62-a1ab-4113-bef3-ed1b290e47c1-multus-cni-dir\") pod \"multus-9ssmv\" (UID: \"aa3f3b62-a1ab-4113-bef3-ed1b290e47c1\") " pod="openshift-multus/multus-9ssmv"
Oct 30 18:33:57.153787 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.153558    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ad78ad05-b0af-4adf-a6bc-0f993b8a6288-kubelet-dir\") pod \"aws-ebs-csi-driver-node-7j9jw\" (UID: \"ad78ad05-b0af-4adf-a6bc-0f993b8a6288\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7j9jw"
Oct 30 18:33:57.153787 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.153581    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d81cef55-16ee-4677-ae12-86de3c9eb216-tmp-dir\") pod \"node-resolver-76f9s\" (UID: \"d81cef55-16ee-4677-ae12-86de3c9eb216\") " pod="openshift-dns/node-resolver-76f9s"
Oct 30 18:33:57.153787 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.153631    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/09c13759-50d4-40e6-b854-4691822b8c8a-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-5bqdc\" (UID: \"09c13759-50d4-40e6-b854-4691822b8c8a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-5bqdc"
Oct 30 18:33:57.153787 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.153653    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d81cef55-16ee-4677-ae12-86de3c9eb216-tmp-dir\") pod \"node-resolver-76f9s\" (UID: \"d81cef55-16ee-4677-ae12-86de3c9eb216\") " pod="openshift-dns/node-resolver-76f9s"
Oct 30 18:33:57.153787 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.153659    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3f3fe764-2652-46c9-8e54-7882c0e6bcf1-host-cni-bin\") pod \"ovnkube-node-xfcdl\" (UID: \"3f3fe764-2652-46c9-8e54-7882c0e6bcf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xfcdl"
Oct 30 18:33:57.153787 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.153690    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wrmcg\" (UniqueName: \"kubernetes.io/projected/57ca8600-79e7-4f59-9d11-b1c9ec3d358a-kube-api-access-wrmcg\") pod \"insights-runtime-extractor-jbsw6\" (UID: \"57ca8600-79e7-4f59-9d11-b1c9ec3d358a\") " pod="openshift-insights/insights-runtime-extractor-jbsw6"
Oct 30 18:33:57.153787 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.153784    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/09c13759-50d4-40e6-b854-4691822b8c8a-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-5bqdc\" (UID: \"09c13759-50d4-40e6-b854-4691822b8c8a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-5bqdc"
Oct 30 18:33:57.153787 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.153792    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/aa3f3b62-a1ab-4113-bef3-ed1b290e47c1-multus-conf-dir\") pod \"multus-9ssmv\" (UID: \"aa3f3b62-a1ab-4113-bef3-ed1b290e47c1\") " pod="openshift-multus/multus-9ssmv"
Oct 30 18:33:57.154492 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.153820    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m4j2q\" (UniqueName: \"kubernetes.io/projected/382e8b31-a7fa-43f3-9f15-4c61627b0c04-kube-api-access-m4j2q\") pod \"console-operator-7cd56b8dbc-7x4rb\" (UID: \"382e8b31-a7fa-43f3-9f15-4c61627b0c04\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-7x4rb"
Oct 30 18:33:57.154492 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.153871    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/2e49faaa-a718-44c8-9237-5a41169faee7-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-rzrjz\" (UID: \"2e49faaa-a718-44c8-9237-5a41169faee7\") " pod="openshift-multus/multus-additional-cni-plugins-rzrjz"
Oct 30 18:33:57.154492 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.153891    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/da28351c-1c8b-40be-85a0-3aad427e3a17-service-ca-bundle\") pod \"router-default-7dfcf9fd6c-vh769\" (UID: \"da28351c-1c8b-40be-85a0-3aad427e3a17\") " pod="openshift-ingress/router-default-7dfcf9fd6c-vh769"
Oct 30 18:33:57.154492 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.153907    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/36a04851-81db-40cd-97ef-8e2e00a08604-config\") pod \"service-ca-operator-544d749c8c-kr5qq\" (UID: \"36a04851-81db-40cd-97ef-8e2e00a08604\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-kr5qq"
Oct 30 18:33:57.154492 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.153925    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/8ecf2b62-8053-4bba-a852-c6f6119d3288-konnectivity-ca\") pod \"konnectivity-agent-lmdpp\" (UID: \"8ecf2b62-8053-4bba-a852-c6f6119d3288\") " pod="kube-system/konnectivity-agent-lmdpp"
Oct 30 18:33:57.154492 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.153947    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f2a80d89-61d1-4694-8734-004ce5cfb22b-var-lib-kubelet\") pod \"tuned-z9jmw\" (UID: \"f2a80d89-61d1-4694-8734-004ce5cfb22b\") " pod="openshift-cluster-node-tuning-operator/tuned-z9jmw"
Oct 30 18:33:57.154492 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.154047    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7c09088b-3a07-426d-8804-f0f5bac4d27c-serving-cert\") pod \"insights-operator-7c4986d449-xvgqc\" (UID: \"7c09088b-3a07-426d-8804-f0f5bac4d27c\") " pod="openshift-insights/insights-operator-7c4986d449-xvgqc"
Oct 30 18:33:57.154492 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.154087    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/36a04851-81db-40cd-97ef-8e2e00a08604-config\") pod \"service-ca-operator-544d749c8c-kr5qq\" (UID: \"36a04851-81db-40cd-97ef-8e2e00a08604\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-kr5qq"
Oct 30 18:33:57.154492 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.154088    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3f3fe764-2652-46c9-8e54-7882c0e6bcf1-host-run-netns\") pod \"ovnkube-node-xfcdl\" (UID: \"3f3fe764-2652-46c9-8e54-7882c0e6bcf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xfcdl"
Oct 30 18:33:57.154492 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.154129    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3f3fe764-2652-46c9-8e54-7882c0e6bcf1-run-openvswitch\") pod \"ovnkube-node-xfcdl\" (UID: \"3f3fe764-2652-46c9-8e54-7882c0e6bcf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xfcdl"
Oct 30 18:33:57.154492 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.154163    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t56q5\" (UniqueName: \"kubernetes.io/projected/a0823e8f-bbdd-477b-95ac-dad05d23cefa-kube-api-access-t56q5\") pod \"network-metrics-daemon-v5phn\" (UID: \"a0823e8f-bbdd-477b-95ac-dad05d23cefa\") " pod="openshift-multus/network-metrics-daemon-v5phn"
Oct 30 18:33:57.154492 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.154203    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/46a80637-6ccf-4a56-9904-f5e2fe5784ad-serviceca\") pod \"node-ca-g78lj\" (UID: \"46a80637-6ccf-4a56-9904-f5e2fe5784ad\") " pod="openshift-image-registry/node-ca-g78lj"
Oct 30 18:33:57.154492 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.154229    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4rztt\" (UniqueName: \"kubernetes.io/projected/68195e19-b1ff-4148-9fae-3545f2997ff5-kube-api-access-4rztt\") pod \"cluster-monitoring-operator-884576c57-gmswj\" (UID: \"68195e19-b1ff-4148-9fae-3545f2997ff5\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-gmswj"
Oct 30 18:33:57.154492 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.154246    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/7c09088b-3a07-426d-8804-f0f5bac4d27c-snapshots\") pod \"insights-operator-7c4986d449-xvgqc\" (UID: \"7c09088b-3a07-426d-8804-f0f5bac4d27c\") " pod="openshift-insights/insights-operator-7c4986d449-xvgqc"
Oct 30 18:33:57.154492 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.154263    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wzxwr\" (UniqueName: \"kubernetes.io/projected/3f3fe764-2652-46c9-8e54-7882c0e6bcf1-kube-api-access-wzxwr\") pod \"ovnkube-node-xfcdl\" (UID: \"3f3fe764-2652-46c9-8e54-7882c0e6bcf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xfcdl"
Oct 30 18:33:57.154492 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.154294    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/57ca8600-79e7-4f59-9d11-b1c9ec3d358a-crio-socket\") pod \"insights-runtime-extractor-jbsw6\" (UID: \"57ca8600-79e7-4f59-9d11-b1c9ec3d358a\") " pod="openshift-insights/insights-runtime-extractor-jbsw6"
Oct 30 18:33:57.154492 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.154315    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/aa3f3b62-a1ab-4113-bef3-ed1b290e47c1-cnibin\") pod \"multus-9ssmv\" (UID: \"aa3f3b62-a1ab-4113-bef3-ed1b290e47c1\") " pod="openshift-multus/multus-9ssmv"
Oct 30 18:33:57.154492 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.154339    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/aa3f3b62-a1ab-4113-bef3-ed1b290e47c1-host-var-lib-cni-multus\") pod \"multus-9ssmv\" (UID: \"aa3f3b62-a1ab-4113-bef3-ed1b290e47c1\") " pod="openshift-multus/multus-9ssmv"
Oct 30 18:33:57.154492 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.154369    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/7c09088b-3a07-426d-8804-f0f5bac4d27c-snapshots\") pod \"insights-operator-7c4986d449-xvgqc\" (UID: \"7c09088b-3a07-426d-8804-f0f5bac4d27c\") " pod="openshift-insights/insights-operator-7c4986d449-xvgqc"
Oct 30 18:33:57.154492 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.154385    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f2a80d89-61d1-4694-8734-004ce5cfb22b-host\") pod \"tuned-z9jmw\" (UID: \"f2a80d89-61d1-4694-8734-004ce5cfb22b\") " pod="openshift-cluster-node-tuning-operator/tuned-z9jmw"
Oct 30 18:33:57.154492 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.154422    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351-ca-trust-extracted\") pod \"image-registry-64bf5b6b76-dpvvs\" (UID: \"9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351\") " pod="openshift-image-registry/image-registry-64bf5b6b76-dpvvs"
Oct 30 18:33:57.154492 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.154466    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/46a80637-6ccf-4a56-9904-f5e2fe5784ad-serviceca\") pod \"node-ca-g78lj\" (UID: \"46a80637-6ccf-4a56-9904-f5e2fe5784ad\") " pod="openshift-image-registry/node-ca-g78lj"
Oct 30 18:33:57.154492 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.154476    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f2a80d89-61d1-4694-8734-004ce5cfb22b-run\") pod \"tuned-z9jmw\" (UID: \"f2a80d89-61d1-4694-8734-004ce5cfb22b\") " pod="openshift-cluster-node-tuning-operator/tuned-z9jmw"
Oct 30 18:33:57.154492 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.154488    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351-ca-trust-extracted\") pod \"image-registry-64bf5b6b76-dpvvs\" (UID: \"9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351\") " pod="openshift-image-registry/image-registry-64bf5b6b76-dpvvs"
Oct 30 18:33:57.155140 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.154524    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f2a80d89-61d1-4694-8734-004ce5cfb22b-lib-modules\") pod \"tuned-z9jmw\" (UID: \"f2a80d89-61d1-4694-8734-004ce5cfb22b\") " pod="openshift-cluster-node-tuning-operator/tuned-z9jmw"
Oct 30 18:33:57.155140 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.154550    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8wvhh\" (UniqueName: \"kubernetes.io/projected/f2a80d89-61d1-4694-8734-004ce5cfb22b-kube-api-access-8wvhh\") pod \"tuned-z9jmw\" (UID: \"f2a80d89-61d1-4694-8734-004ce5cfb22b\") " pod="openshift-cluster-node-tuning-operator/tuned-z9jmw"
Oct 30 18:33:57.155140 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.154566    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e6b2a584-ae78-4e80-b595-9b1a5897d556-cert\") pod \"ingress-canary-g9fhp\" (UID: \"e6b2a584-ae78-4e80-b595-9b1a5897d556\") " pod="openshift-ingress-canary/ingress-canary-g9fhp"
Oct 30 18:33:57.155140 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.154612    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/da28351c-1c8b-40be-85a0-3aad427e3a17-metrics-certs\") pod \"router-default-7dfcf9fd6c-vh769\" (UID: \"da28351c-1c8b-40be-85a0-3aad427e3a17\") " pod="openshift-ingress/router-default-7dfcf9fd6c-vh769"
Oct 30 18:33:57.155140 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.154662    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/aa3f3b62-a1ab-4113-bef3-ed1b290e47c1-cni-binary-copy\") pod \"multus-9ssmv\" (UID: \"aa3f3b62-a1ab-4113-bef3-ed1b290e47c1\") " pod="openshift-multus/multus-9ssmv"
Oct 30 18:33:57.155140 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.154681    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2e49faaa-a718-44c8-9237-5a41169faee7-system-cni-dir\") pod \"multus-additional-cni-plugins-rzrjz\" (UID: \"2e49faaa-a718-44c8-9237-5a41169faee7\") " pod="openshift-multus/multus-additional-cni-plugins-rzrjz"
Oct 30 18:33:57.155140 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.154697    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7c09088b-3a07-426d-8804-f0f5bac4d27c-tmp\") pod \"insights-operator-7c4986d449-xvgqc\" (UID: \"7c09088b-3a07-426d-8804-f0f5bac4d27c\") " pod="openshift-insights/insights-operator-7c4986d449-xvgqc"
Oct 30 18:33:57.155140 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.154728    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/46d9ac39-ee45-4e43-bda7-3931c3e599cf-kubelet-config\") pod \"global-pull-secret-syncer-g2svp\" (UID: \"46d9ac39-ee45-4e43-bda7-3931c3e599cf\") " pod="kube-system/global-pull-secret-syncer-g2svp"
Oct 30 18:33:57.155140 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.154750    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/46d9ac39-ee45-4e43-bda7-3931c3e599cf-dbus\") pod \"global-pull-secret-syncer-g2svp\" (UID: \"46d9ac39-ee45-4e43-bda7-3931c3e599cf\") " pod="kube-system/global-pull-secret-syncer-g2svp"
Oct 30 18:33:57.155140 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.154767    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7c09088b-3a07-426d-8804-f0f5bac4d27c-tmp\") pod \"insights-operator-7c4986d449-xvgqc\" (UID: \"7c09088b-3a07-426d-8804-f0f5bac4d27c\") " pod="openshift-insights/insights-operator-7c4986d449-xvgqc"
Oct 30 18:33:57.155140 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.154781    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/57ca8600-79e7-4f59-9d11-b1c9ec3d358a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jbsw6\" (UID: \"57ca8600-79e7-4f59-9d11-b1c9ec3d358a\") " pod="openshift-insights/insights-runtime-extractor-jbsw6"
Oct 30 18:33:57.155140 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.154798    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/aa3f3b62-a1ab-4113-bef3-ed1b290e47c1-host-var-lib-kubelet\") pod \"multus-9ssmv\" (UID: \"aa3f3b62-a1ab-4113-bef3-ed1b290e47c1\") " pod="openshift-multus/multus-9ssmv"
Oct 30 18:33:57.155140 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.154837    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/382e8b31-a7fa-43f3-9f15-4c61627b0c04-trusted-ca\") pod \"console-operator-7cd56b8dbc-7x4rb\" (UID: \"382e8b31-a7fa-43f3-9f15-4c61627b0c04\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-7x4rb"
Oct 30 18:33:57.155140 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.154861    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ad78ad05-b0af-4adf-a6bc-0f993b8a6288-etc-selinux\") pod \"aws-ebs-csi-driver-node-7j9jw\" (UID: \"ad78ad05-b0af-4adf-a6bc-0f993b8a6288\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7j9jw"
Oct 30 18:33:57.155140 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.154906    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/3f3fe764-2652-46c9-8e54-7882c0e6bcf1-host-kubelet\") pod \"ovnkube-node-xfcdl\" (UID: \"3f3fe764-2652-46c9-8e54-7882c0e6bcf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xfcdl"
Oct 30 18:33:57.155140 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.154936    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/3f3fe764-2652-46c9-8e54-7882c0e6bcf1-ovnkube-script-lib\") pod \"ovnkube-node-xfcdl\" (UID: \"3f3fe764-2652-46c9-8e54-7882c0e6bcf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xfcdl"
Oct 30 18:33:57.155140 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.154964    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/97b19aa3-42be-4def-8d7a-8e3516852e79-metrics-tls\") pod \"dns-default-r2nx2\" (UID: \"97b19aa3-42be-4def-8d7a-8e3516852e79\") " pod="openshift-dns/dns-default-r2nx2"
Oct 30 18:33:57.155140 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.154982    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ad78ad05-b0af-4adf-a6bc-0f993b8a6288-device-dir\") pod \"aws-ebs-csi-driver-node-7j9jw\" (UID: \"ad78ad05-b0af-4adf-a6bc-0f993b8a6288\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7j9jw"
Oct 30 18:33:57.155140 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.155032    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f2a80d89-61d1-4694-8734-004ce5cfb22b-etc-systemd\") pod \"tuned-z9jmw\" (UID: \"f2a80d89-61d1-4694-8734-004ce5cfb22b\") " pod="openshift-cluster-node-tuning-operator/tuned-z9jmw"
Oct 30 18:33:57.155140 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.155057    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f2a80d89-61d1-4694-8734-004ce5cfb22b-tmp\") pod \"tuned-z9jmw\" (UID: \"f2a80d89-61d1-4694-8734-004ce5cfb22b\") " pod="openshift-cluster-node-tuning-operator/tuned-z9jmw"
Oct 30 18:33:57.155140 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.155079    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/382e8b31-a7fa-43f3-9f15-4c61627b0c04-trusted-ca\") pod \"console-operator-7cd56b8dbc-7x4rb\" (UID: \"382e8b31-a7fa-43f3-9f15-4c61627b0c04\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-7x4rb"
Oct 30 18:33:57.155140 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.155101    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-24cp9\" (UniqueName: \"kubernetes.io/projected/e6b2a584-ae78-4e80-b595-9b1a5897d556-kube-api-access-24cp9\") pod \"ingress-canary-g9fhp\" (UID: \"e6b2a584-ae78-4e80-b595-9b1a5897d556\") " pod="openshift-ingress-canary/ingress-canary-g9fhp"
Oct 30 18:33:57.155140 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.155123    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f2a80d89-61d1-4694-8734-004ce5cfb22b-tmp\") pod \"tuned-z9jmw\" (UID: \"f2a80d89-61d1-4694-8734-004ce5cfb22b\") " pod="openshift-cluster-node-tuning-operator/tuned-z9jmw"
Oct 30 18:33:57.155140 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.155143    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8789ec5b-226e-4551-bf68-3ac585006266-registry-tls\") pod \"image-registry-74757bccd5-qmrkp\" (UID: \"8789ec5b-226e-4551-bf68-3ac585006266\") " pod="openshift-image-registry/image-registry-74757bccd5-qmrkp"
Oct 30 18:33:57.155953 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.155230    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fzrkt\" (UniqueName: \"kubernetes.io/projected/15d485f9-5a4b-4f4b-9e59-a6eb062b4c5c-kube-api-access-fzrkt\") pod \"cluster-samples-operator-6ddb6c8b94-pm58m\" (UID: \"15d485f9-5a4b-4f4b-9e59-a6eb062b4c5c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-pm58m"
Oct 30 18:33:57.155953 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.155254    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-btt6c\" (UniqueName: \"kubernetes.io/projected/dedb8b54-d0ce-434c-9461-db90a189c121-kube-api-access-btt6c\") pod \"kas-connection-checker-685895c7c9-4b95q\" (UID: \"dedb8b54-d0ce-434c-9461-db90a189c121\") " pod="kube-system/kas-connection-checker-685895c7c9-4b95q"
Oct 30 18:33:57.155953 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.155323    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/aa3f3b62-a1ab-4113-bef3-ed1b290e47c1-multus-daemon-config\") pod \"multus-9ssmv\" (UID: \"aa3f3b62-a1ab-4113-bef3-ed1b290e47c1\") " pod="openshift-multus/multus-9ssmv"
Oct 30 18:33:57.155953 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.155410    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/8ecf2b62-8053-4bba-a852-c6f6119d3288-agent-certs\") pod \"konnectivity-agent-lmdpp\" (UID: \"8ecf2b62-8053-4bba-a852-c6f6119d3288\") " pod="kube-system/konnectivity-agent-lmdpp"
Oct 30 18:33:57.155953 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.155450    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6llht\" (UniqueName: \"kubernetes.io/projected/2e49faaa-a718-44c8-9237-5a41169faee7-kube-api-access-6llht\") pod \"multus-additional-cni-plugins-rzrjz\" (UID: \"2e49faaa-a718-44c8-9237-5a41169faee7\") " pod="openshift-multus/multus-additional-cni-plugins-rzrjz"
Oct 30 18:33:57.155953 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.155490    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/71fce796-0032-481b-80b3-66fbb731452c-nginx-conf\") pod \"networking-console-plugin-86c478bb88-r8rgt\" (UID: \"71fce796-0032-481b-80b3-66fbb731452c\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-r8rgt"
Oct 30 18:33:57.155953 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.155589    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/3f3fe764-2652-46c9-8e54-7882c0e6bcf1-ovnkube-config\") pod \"ovnkube-node-xfcdl\" (UID: \"3f3fe764-2652-46c9-8e54-7882c0e6bcf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xfcdl"
Oct 30 18:33:57.155953 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.155624    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/aa3f3b62-a1ab-4113-bef3-ed1b290e47c1-hostroot\") pod \"multus-9ssmv\" (UID: \"aa3f3b62-a1ab-4113-bef3-ed1b290e47c1\") " pod="openshift-multus/multus-9ssmv"
Oct 30 18:33:57.155953 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.155683    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/aa3f3b62-a1ab-4113-bef3-ed1b290e47c1-host-run-multus-certs\") pod \"multus-9ssmv\" (UID: \"aa3f3b62-a1ab-4113-bef3-ed1b290e47c1\") " pod="openshift-multus/multus-9ssmv"
Oct 30 18:33:57.173586 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.173562    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Oct 30 18:33:57.173586 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.173595    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Oct 30 18:33:57.173810 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.173797    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Oct 30 18:33:57.173867 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.173818    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Oct 30 18:33:57.173867 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.173839    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Oct 30 18:33:57.173867 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.173852    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Oct 30 18:33:57.174016 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.173873    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-nh9hp\""
Oct 30 18:33:57.174016 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.173882    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-cr7s6\""
Oct 30 18:33:57.174016 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.173922    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-lwjtg\""
Oct 30 18:33:57.174016 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.173936    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-77vj9\""
Oct 30 18:33:57.174196 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.174067    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Oct 30 18:33:57.174196 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.174067    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-qb49q\""
Oct 30 18:33:57.174330 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.174260    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-cgz7p\""
Oct 30 18:33:57.174382 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.174367    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7c09088b-3a07-426d-8804-f0f5bac4d27c-serving-cert\") pod \"insights-operator-7c4986d449-xvgqc\" (UID: \"7c09088b-3a07-426d-8804-f0f5bac4d27c\") " pod="openshift-insights/insights-operator-7c4986d449-xvgqc"
Oct 30 18:33:57.174578 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.174560    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-dockercfg-mlq55\""
Oct 30 18:33:57.174678 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.174575    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-b7gnv\""
Oct 30 18:33:57.174678 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.174570    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-jkxcr\""
Oct 30 18:33:57.174678 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.174558    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Oct 30 18:33:57.174678 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.174589    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-8wmd6\""
Oct 30 18:33:57.174678 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.174592    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-g6mvb\""
Oct 30 18:33:57.174678 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.174594    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-lmrrc\""
Oct 30 18:33:57.174678 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.174594    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-sqz88\""
Oct 30 18:33:57.174955 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.174593    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-pw2xp\""
Oct 30 18:33:57.174955 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.174623    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-wkpbb\""
Oct 30 18:33:57.174955 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.174776    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-phb6k\""
Oct 30 18:33:57.174955 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.174781    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e6b2a584-ae78-4e80-b595-9b1a5897d556-cert\") pod \"ingress-canary-g9fhp\" (UID: \"e6b2a584-ae78-4e80-b595-9b1a5897d556\") " pod="openshift-ingress-canary/ingress-canary-g9fhp"
Oct 30 18:33:57.174955 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.174841    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-5qqcs\""
Oct 30 18:33:57.175170 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.175088    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Oct 30 18:33:57.175170 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.175118    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-k76xp\""
Oct 30 18:33:57.175170 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.175130    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Oct 30 18:33:57.175274 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.175257    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-727z7\""
Oct 30 18:33:57.175385 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.175362    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-4bpx7\""
Oct 30 18:33:57.176539 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.176523    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/8ecf2b62-8053-4bba-a852-c6f6119d3288-agent-certs\") pod \"konnectivity-agent-lmdpp\" (UID: \"8ecf2b62-8053-4bba-a852-c6f6119d3288\") " pod="kube-system/konnectivity-agent-lmdpp"
Oct 30 18:33:57.180329 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.180294    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Oct 30 18:33:57.180329 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.180312    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Oct 30 18:33:57.180329 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.180334    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Oct 30 18:33:57.180531 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.180350    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Oct 30 18:33:57.180531 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.180365    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Oct 30 18:33:57.180531 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.180384    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Oct 30 18:33:57.180531 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.180395    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Oct 30 18:33:57.180531 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.180402    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Oct 30 18:33:57.180531 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.180406    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Oct 30 18:33:57.180531 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.180424    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Oct 30 18:33:57.180531 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.180429    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Oct 30 18:33:57.180531 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.180417    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Oct 30 18:33:57.180531 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.180395    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Oct 30 18:33:57.180531 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.180452    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Oct 30 18:33:57.180531 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.180464    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Oct 30 18:33:57.180531 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.180468    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Oct 30 18:33:57.180531 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.180537    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Oct 30 18:33:57.181083 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.180557    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Oct 30 18:33:57.181083 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.180579    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Oct 30 18:33:57.181083 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.180728    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Oct 30 18:33:57.181083 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.181041    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7c09088b-3a07-426d-8804-f0f5bac4d27c-service-ca-bundle\") pod \"insights-operator-7c4986d449-xvgqc\" (UID: \"7c09088b-3a07-426d-8804-f0f5bac4d27c\") " pod="openshift-insights/insights-operator-7c4986d449-xvgqc"
Oct 30 18:33:57.181265 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.181252    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Oct 30 18:33:57.181670 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.181651    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7c09088b-3a07-426d-8804-f0f5bac4d27c-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-xvgqc\" (UID: \"7c09088b-3a07-426d-8804-f0f5bac4d27c\") " pod="openshift-insights/insights-operator-7c4986d449-xvgqc"
Oct 30 18:33:57.183264 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.183249    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/3f3fe764-2652-46c9-8e54-7882c0e6bcf1-ovn-node-metrics-cert\") pod \"ovnkube-node-xfcdl\" (UID: \"3f3fe764-2652-46c9-8e54-7882c0e6bcf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xfcdl"
Oct 30 18:33:57.183338 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.183318    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/3f3fe764-2652-46c9-8e54-7882c0e6bcf1-env-overrides\") pod \"ovnkube-node-xfcdl\" (UID: \"3f3fe764-2652-46c9-8e54-7882c0e6bcf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xfcdl"
Oct 30 18:33:57.183371 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.183352    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/46d9ac39-ee45-4e43-bda7-3931c3e599cf-original-pull-secret\") pod \"global-pull-secret-syncer-g2svp\" (UID: \"46d9ac39-ee45-4e43-bda7-3931c3e599cf\") " pod="kube-system/global-pull-secret-syncer-g2svp"
Oct 30 18:33:57.184467 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.184452    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/2e49faaa-a718-44c8-9237-5a41169faee7-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-rzrjz\" (UID: \"2e49faaa-a718-44c8-9237-5a41169faee7\") " pod="openshift-multus/multus-additional-cni-plugins-rzrjz"
Oct 30 18:33:57.184679 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.184665    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Oct 30 18:33:57.184904 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.184892    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Oct 30 18:33:57.185169 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.185154    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Oct 30 18:33:57.185169 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.185164    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Oct 30 18:33:57.185245 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.185169    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Oct 30 18:33:57.185523 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.185458    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Oct 30 18:33:57.185523 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.185482    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Oct 30 18:33:57.185523 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.185490    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Oct 30 18:33:57.185523 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.185507    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Oct 30 18:33:57.185704 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.185532    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Oct 30 18:33:57.185704 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.185650    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/aa3f3b62-a1ab-4113-bef3-ed1b290e47c1-multus-daemon-config\") pod \"multus-9ssmv\" (UID: \"aa3f3b62-a1ab-4113-bef3-ed1b290e47c1\") " pod="openshift-multus/multus-9ssmv"
Oct 30 18:33:57.185804 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.185713    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Oct 30 18:33:57.185804 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.185736    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/71fce796-0032-481b-80b3-66fbb731452c-nginx-conf\") pod \"networking-console-plugin-86c478bb88-r8rgt\" (UID: \"71fce796-0032-481b-80b3-66fbb731452c\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-r8rgt"
Oct 30 18:33:57.185804 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.185780    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Oct 30 18:33:57.185960 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.185806    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/3f3fe764-2652-46c9-8e54-7882c0e6bcf1-ovnkube-config\") pod \"ovnkube-node-xfcdl\" (UID: \"3f3fe764-2652-46c9-8e54-7882c0e6bcf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xfcdl"
Oct 30 18:33:57.185960 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.185904    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Oct 30 18:33:57.192257 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.192240    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/97b19aa3-42be-4def-8d7a-8e3516852e79-config-volume\") pod \"dns-default-r2nx2\" (UID: \"97b19aa3-42be-4def-8d7a-8e3516852e79\") " pod="openshift-dns/dns-default-r2nx2"
Oct 30 18:33:57.192257 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.192246    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/2e49faaa-a718-44c8-9237-5a41169faee7-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-rzrjz\" (UID: \"2e49faaa-a718-44c8-9237-5a41169faee7\") " pod="openshift-multus/multus-additional-cni-plugins-rzrjz"
Oct 30 18:33:57.192376 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.192297    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/63090f48-9143-4118-9906-d759c0aff52c-iptables-alerter-script\") pod \"iptables-alerter-rcjmv\" (UID: \"63090f48-9143-4118-9906-d759c0aff52c\") " pod="openshift-network-operator/iptables-alerter-rcjmv"
Oct 30 18:33:57.193400 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.193385    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2e49faaa-a718-44c8-9237-5a41169faee7-cni-binary-copy\") pod \"multus-additional-cni-plugins-rzrjz\" (UID: \"2e49faaa-a718-44c8-9237-5a41169faee7\") " pod="openshift-multus/multus-additional-cni-plugins-rzrjz"
Oct 30 18:33:57.194561 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.194546    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/8ecf2b62-8053-4bba-a852-c6f6119d3288-konnectivity-ca\") pod \"konnectivity-agent-lmdpp\" (UID: \"8ecf2b62-8053-4bba-a852-c6f6119d3288\") " pod="kube-system/konnectivity-agent-lmdpp"
Oct 30 18:33:57.195629 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.195614    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/3f3fe764-2652-46c9-8e54-7882c0e6bcf1-ovnkube-script-lib\") pod \"ovnkube-node-xfcdl\" (UID: \"3f3fe764-2652-46c9-8e54-7882c0e6bcf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xfcdl"
Oct 30 18:33:57.195679 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.195666    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/aa3f3b62-a1ab-4113-bef3-ed1b290e47c1-cni-binary-copy\") pod \"multus-9ssmv\" (UID: \"aa3f3b62-a1ab-4113-bef3-ed1b290e47c1\") " pod="openshift-multus/multus-9ssmv"
Oct 30 18:33:57.211715 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.211689    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c45qr\" (UniqueName: \"kubernetes.io/projected/613020c0-f3e4-4650-896f-748a40f0ab8b-kube-api-access-c45qr\") pod \"network-check-target-8xskh\" (UID: \"613020c0-f3e4-4650-896f-748a40f0ab8b\") " pod="openshift-network-diagnostics/network-check-target-8xskh"
Oct 30 18:33:57.212357 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.212339    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cvxhl\" (UniqueName: \"kubernetes.io/projected/7c09088b-3a07-426d-8804-f0f5bac4d27c-kube-api-access-cvxhl\") pod \"insights-operator-7c4986d449-xvgqc\" (UID: \"7c09088b-3a07-426d-8804-f0f5bac4d27c\") " pod="openshift-insights/insights-operator-7c4986d449-xvgqc"
Oct 30 18:33:57.213787 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.213771    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rw8g9\" (UniqueName: \"kubernetes.io/projected/36a04851-81db-40cd-97ef-8e2e00a08604-kube-api-access-rw8g9\") pod \"service-ca-operator-544d749c8c-kr5qq\" (UID: \"36a04851-81db-40cd-97ef-8e2e00a08604\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-kr5qq"
Oct 30 18:33:57.217098 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.217078    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8789ec5b-226e-4551-bf68-3ac585006266-bound-sa-token\") pod \"image-registry-74757bccd5-qmrkp\" (UID: \"8789ec5b-226e-4551-bf68-3ac585006266\") " pod="openshift-image-registry/image-registry-74757bccd5-qmrkp"
Oct 30 18:33:57.217186 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.217141    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5f2ls\" (UniqueName: \"kubernetes.io/projected/ad78ad05-b0af-4adf-a6bc-0f993b8a6288-kube-api-access-5f2ls\") pod \"aws-ebs-csi-driver-node-7j9jw\" (UID: \"ad78ad05-b0af-4adf-a6bc-0f993b8a6288\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7j9jw"
Oct 30 18:33:57.217352 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.217334    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nk8r7\" (UniqueName: \"kubernetes.io/projected/df9e5074-3bb1-4817-8855-b045cee61004-kube-api-access-nk8r7\") pod \"kas-connection-checker-685895c7c9-f9ldt\" (UID: \"df9e5074-3bb1-4817-8855-b045cee61004\") " pod="kube-system/kas-connection-checker-685895c7c9-f9ldt"
Oct 30 18:33:57.217826 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.217811    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vf948\" (UniqueName: \"kubernetes.io/projected/8789ec5b-226e-4551-bf68-3ac585006266-kube-api-access-vf948\") pod \"image-registry-74757bccd5-qmrkp\" (UID: \"8789ec5b-226e-4551-bf68-3ac585006266\") " pod="openshift-image-registry/image-registry-74757bccd5-qmrkp"
Oct 30 18:33:57.218547 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.218528    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rvrxn\" (UniqueName: \"kubernetes.io/projected/aa3f3b62-a1ab-4113-bef3-ed1b290e47c1-kube-api-access-rvrxn\") pod \"multus-9ssmv\" (UID: \"aa3f3b62-a1ab-4113-bef3-ed1b290e47c1\") " pod="openshift-multus/multus-9ssmv"
Oct 30 18:33:57.219503 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.219488    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9j2qw\" (UniqueName: \"kubernetes.io/projected/81de5dce-c40f-4bd5-878c-5d75a0487066-kube-api-access-9j2qw\") pod \"network-check-source-6f78d59f99-4969l\" (UID: \"81de5dce-c40f-4bd5-878c-5d75a0487066\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-4969l"
Oct 30 18:33:57.219584 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.219570    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tc2fp\" (UniqueName: \"kubernetes.io/projected/97b19aa3-42be-4def-8d7a-8e3516852e79-kube-api-access-tc2fp\") pod \"dns-default-r2nx2\" (UID: \"97b19aa3-42be-4def-8d7a-8e3516852e79\") " pod="openshift-dns/dns-default-r2nx2"
Oct 30 18:33:57.221163 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.221144    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kk8qr\" (UniqueName: \"kubernetes.io/projected/f4ac17da-6b5b-4441-9b4e-ebe696e06507-kube-api-access-kk8qr\") pod \"volume-data-source-validator-58c9f5f46d-jwwq2\" (UID: \"f4ac17da-6b5b-4441-9b4e-ebe696e06507\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-jwwq2"
Oct 30 18:33:57.222384 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.222368    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wrmcg\" (UniqueName: \"kubernetes.io/projected/57ca8600-79e7-4f59-9d11-b1c9ec3d358a-kube-api-access-wrmcg\") pod \"insights-runtime-extractor-jbsw6\" (UID: \"57ca8600-79e7-4f59-9d11-b1c9ec3d358a\") " pod="openshift-insights/insights-runtime-extractor-jbsw6"
Oct 30 18:33:57.222728 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.222709    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jt475\" (UniqueName: \"kubernetes.io/projected/a3cf021c-2c20-4687-9b3a-cfa65ce1ee8b-kube-api-access-jt475\") pod \"migrator-66df5d496b-ht9zn\" (UID: \"a3cf021c-2c20-4687-9b3a-cfa65ce1ee8b\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-ht9zn"
Oct 30 18:33:57.222784 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.222747    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gzhnk\" (UniqueName: \"kubernetes.io/projected/d81cef55-16ee-4677-ae12-86de3c9eb216-kube-api-access-gzhnk\") pod \"node-resolver-76f9s\" (UID: \"d81cef55-16ee-4677-ae12-86de3c9eb216\") " pod="openshift-dns/node-resolver-76f9s"
Oct 30 18:33:57.222859 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.222844    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4rztt\" (UniqueName: \"kubernetes.io/projected/68195e19-b1ff-4148-9fae-3545f2997ff5-kube-api-access-4rztt\") pod \"cluster-monitoring-operator-884576c57-gmswj\" (UID: \"68195e19-b1ff-4148-9fae-3545f2997ff5\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-gmswj"
Oct 30 18:33:57.223464 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.223448    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x2978\" (UniqueName: \"kubernetes.io/projected/46a80637-6ccf-4a56-9904-f5e2fe5784ad-kube-api-access-x2978\") pod \"node-ca-g78lj\" (UID: \"46a80637-6ccf-4a56-9904-f5e2fe5784ad\") " pod="openshift-image-registry/node-ca-g78lj"
Oct 30 18:33:57.223843 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.223829    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t56q5\" (UniqueName: \"kubernetes.io/projected/a0823e8f-bbdd-477b-95ac-dad05d23cefa-kube-api-access-t56q5\") pod \"network-metrics-daemon-v5phn\" (UID: \"a0823e8f-bbdd-477b-95ac-dad05d23cefa\") " pod="openshift-multus/network-metrics-daemon-v5phn"
Oct 30 18:33:57.223894 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.223878    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cftdb\" (UniqueName: \"kubernetes.io/projected/da28351c-1c8b-40be-85a0-3aad427e3a17-kube-api-access-cftdb\") pod \"router-default-7dfcf9fd6c-vh769\" (UID: \"da28351c-1c8b-40be-85a0-3aad427e3a17\") " pod="openshift-ingress/router-default-7dfcf9fd6c-vh769"
Oct 30 18:33:57.224848 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.224833    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m4j2q\" (UniqueName: \"kubernetes.io/projected/382e8b31-a7fa-43f3-9f15-4c61627b0c04-kube-api-access-m4j2q\") pod \"console-operator-7cd56b8dbc-7x4rb\" (UID: \"382e8b31-a7fa-43f3-9f15-4c61627b0c04\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-7x4rb"
Oct 30 18:33:57.228093 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.228077    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351-bound-sa-token\") pod \"image-registry-64bf5b6b76-dpvvs\" (UID: \"9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351\") " pod="openshift-image-registry/image-registry-64bf5b6b76-dpvvs"
Oct 30 18:33:57.246765 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.246744    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6llht\" (UniqueName: \"kubernetes.io/projected/2e49faaa-a718-44c8-9237-5a41169faee7-kube-api-access-6llht\") pod \"multus-additional-cni-plugins-rzrjz\" (UID: \"2e49faaa-a718-44c8-9237-5a41169faee7\") " pod="openshift-multus/multus-additional-cni-plugins-rzrjz"
Oct 30 18:33:57.247480 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.247465    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bgxbg\" (UniqueName: \"kubernetes.io/projected/63090f48-9143-4118-9906-d759c0aff52c-kube-api-access-bgxbg\") pod \"iptables-alerter-rcjmv\" (UID: \"63090f48-9143-4118-9906-d759c0aff52c\") " pod="openshift-network-operator/iptables-alerter-rcjmv"
Oct 30 18:33:57.247779 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.247763    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wzxwr\" (UniqueName: \"kubernetes.io/projected/3f3fe764-2652-46c9-8e54-7882c0e6bcf1-kube-api-access-wzxwr\") pod \"ovnkube-node-xfcdl\" (UID: \"3f3fe764-2652-46c9-8e54-7882c0e6bcf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xfcdl"
Oct 30 18:33:57.249063 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.249044    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vxw47\" (UniqueName: \"kubernetes.io/projected/9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351-kube-api-access-vxw47\") pod \"image-registry-64bf5b6b76-dpvvs\" (UID: \"9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351\") " pod="openshift-image-registry/image-registry-64bf5b6b76-dpvvs"
Oct 30 18:33:57.249545 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.249527    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-24cp9\" (UniqueName: \"kubernetes.io/projected/e6b2a584-ae78-4e80-b595-9b1a5897d556-kube-api-access-24cp9\") pod \"ingress-canary-g9fhp\" (UID: \"e6b2a584-ae78-4e80-b595-9b1a5897d556\") " pod="openshift-ingress-canary/ingress-canary-g9fhp"
Oct 30 18:33:57.249726 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.249708    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k5crl\" (UniqueName: \"kubernetes.io/projected/09c13759-50d4-40e6-b854-4691822b8c8a-kube-api-access-k5crl\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-5bqdc\" (UID: \"09c13759-50d4-40e6-b854-4691822b8c8a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-5bqdc"
Oct 30 18:33:57.249970 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.249955    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-btt6c\" (UniqueName: \"kubernetes.io/projected/dedb8b54-d0ce-434c-9461-db90a189c121-kube-api-access-btt6c\") pod \"kas-connection-checker-685895c7c9-4b95q\" (UID: \"dedb8b54-d0ce-434c-9461-db90a189c121\") " pod="kube-system/kas-connection-checker-685895c7c9-4b95q"
Oct 30 18:33:57.250155 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.250137    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fzrkt\" (UniqueName: \"kubernetes.io/projected/15d485f9-5a4b-4f4b-9e59-a6eb062b4c5c-kube-api-access-fzrkt\") pod \"cluster-samples-operator-6ddb6c8b94-pm58m\" (UID: \"15d485f9-5a4b-4f4b-9e59-a6eb062b4c5c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-pm58m"
Oct 30 18:33:57.250508 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.250492    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x4hfg\" (UniqueName: \"kubernetes.io/projected/e77c4479-d6e5-48a8-ac82-102a9f1a2c4a-kube-api-access-x4hfg\") pod \"kas-connection-checker-685895c7c9-2grbp\" (UID: \"e77c4479-d6e5-48a8-ac82-102a9f1a2c4a\") " pod="kube-system/kas-connection-checker-685895c7c9-2grbp"
Oct 30 18:33:57.254333 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.254319    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8wvhh\" (UniqueName: \"kubernetes.io/projected/f2a80d89-61d1-4694-8734-004ce5cfb22b-kube-api-access-8wvhh\") pod \"tuned-z9jmw\" (UID: \"f2a80d89-61d1-4694-8734-004ce5cfb22b\") " pod="openshift-cluster-node-tuning-operator/tuned-z9jmw"
Oct 30 18:33:57.256209 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.256195    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ad78ad05-b0af-4adf-a6bc-0f993b8a6288-socket-dir\") pod \"aws-ebs-csi-driver-node-7j9jw\" (UID: \"ad78ad05-b0af-4adf-a6bc-0f993b8a6288\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7j9jw"
Oct 30 18:33:57.256247 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.256217    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/aa3f3b62-a1ab-4113-bef3-ed1b290e47c1-host-var-lib-cni-bin\") pod \"multus-9ssmv\" (UID: \"aa3f3b62-a1ab-4113-bef3-ed1b290e47c1\") " pod="openshift-multus/multus-9ssmv"
Oct 30 18:33:57.256247 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.256234    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ad78ad05-b0af-4adf-a6bc-0f993b8a6288-registration-dir\") pod \"aws-ebs-csi-driver-node-7j9jw\" (UID: \"ad78ad05-b0af-4adf-a6bc-0f993b8a6288\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7j9jw"
Oct 30 18:33:57.256351 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.256249    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351-registry-tls\") pod \"image-registry-64bf5b6b76-dpvvs\" (UID: \"9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351\") " pod="openshift-image-registry/image-registry-64bf5b6b76-dpvvs"
Oct 30 18:33:57.256351 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.256301    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/aa3f3b62-a1ab-4113-bef3-ed1b290e47c1-host-var-lib-cni-bin\") pod \"multus-9ssmv\" (UID: \"aa3f3b62-a1ab-4113-bef3-ed1b290e47c1\") " pod="openshift-multus/multus-9ssmv"
Oct 30 18:33:57.256351 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.256323    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ad78ad05-b0af-4adf-a6bc-0f993b8a6288-socket-dir\") pod \"aws-ebs-csi-driver-node-7j9jw\" (UID: \"ad78ad05-b0af-4adf-a6bc-0f993b8a6288\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7j9jw"
Oct 30 18:33:57.256351 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:33:57.256332    4888 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 30 18:33:57.256351 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:33:57.256342    4888 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-64bf5b6b76-dpvvs: secret "image-registry-tls" not found
Oct 30 18:33:57.256558 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.256347    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ad78ad05-b0af-4adf-a6bc-0f993b8a6288-registration-dir\") pod \"aws-ebs-csi-driver-node-7j9jw\" (UID: \"ad78ad05-b0af-4adf-a6bc-0f993b8a6288\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7j9jw"
Oct 30 18:33:57.256558 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.256362    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/aa3f3b62-a1ab-4113-bef3-ed1b290e47c1-multus-socket-dir-parent\") pod \"multus-9ssmv\" (UID: \"aa3f3b62-a1ab-4113-bef3-ed1b290e47c1\") " pod="openshift-multus/multus-9ssmv"
Oct 30 18:33:57.256558 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.256327    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/aa3f3b62-a1ab-4113-bef3-ed1b290e47c1-multus-socket-dir-parent\") pod \"multus-9ssmv\" (UID: \"aa3f3b62-a1ab-4113-bef3-ed1b290e47c1\") " pod="openshift-multus/multus-9ssmv"
Oct 30 18:33:57.256558 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:33:57.256412    4888 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351-registry-tls podName:9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351 nodeName:}" failed. No retries permitted until 2026-10-30 18:33:57.756398352 +0000 UTC m=+2.003804378 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351-registry-tls") pod "image-registry-64bf5b6b76-dpvvs" (UID: "9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351") : secret "image-registry-tls" not found
Oct 30 18:33:57.256558 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.256442    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/aa3f3b62-a1ab-4113-bef3-ed1b290e47c1-etc-kubernetes\") pod \"multus-9ssmv\" (UID: \"aa3f3b62-a1ab-4113-bef3-ed1b290e47c1\") " pod="openshift-multus/multus-9ssmv"
Oct 30 18:33:57.256558 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.256460    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/aa3f3b62-a1ab-4113-bef3-ed1b290e47c1-host-run-netns\") pod \"multus-9ssmv\" (UID: \"aa3f3b62-a1ab-4113-bef3-ed1b290e47c1\") " pod="openshift-multus/multus-9ssmv"
Oct 30 18:33:57.256558 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.256477    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3f3fe764-2652-46c9-8e54-7882c0e6bcf1-host-run-ovn-kubernetes\") pod \"ovnkube-node-xfcdl\" (UID: \"3f3fe764-2652-46c9-8e54-7882c0e6bcf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xfcdl"
Oct 30 18:33:57.256558 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.256479    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/aa3f3b62-a1ab-4113-bef3-ed1b290e47c1-etc-kubernetes\") pod \"multus-9ssmv\" (UID: \"aa3f3b62-a1ab-4113-bef3-ed1b290e47c1\") " pod="openshift-multus/multus-9ssmv"
Oct 30 18:33:57.256558 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.256496    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3f3fe764-2652-46c9-8e54-7882c0e6bcf1-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-xfcdl\" (UID: \"3f3fe764-2652-46c9-8e54-7882c0e6bcf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xfcdl"
Oct 30 18:33:57.256558 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.256538    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/aa3f3b62-a1ab-4113-bef3-ed1b290e47c1-host-run-netns\") pod \"multus-9ssmv\" (UID: \"aa3f3b62-a1ab-4113-bef3-ed1b290e47c1\") " pod="openshift-multus/multus-9ssmv"
Oct 30 18:33:57.256558 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.256556    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/aa3f3b62-a1ab-4113-bef3-ed1b290e47c1-host-run-k8s-cni-cncf-io\") pod \"multus-9ssmv\" (UID: \"aa3f3b62-a1ab-4113-bef3-ed1b290e47c1\") " pod="openshift-multus/multus-9ssmv"
Oct 30 18:33:57.256558 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.256526    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3f3fe764-2652-46c9-8e54-7882c0e6bcf1-host-run-ovn-kubernetes\") pod \"ovnkube-node-xfcdl\" (UID: \"3f3fe764-2652-46c9-8e54-7882c0e6bcf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xfcdl"
Oct 30 18:33:57.256974 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.256559    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3f3fe764-2652-46c9-8e54-7882c0e6bcf1-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-xfcdl\" (UID: \"3f3fe764-2652-46c9-8e54-7882c0e6bcf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xfcdl"
Oct 30 18:33:57.256974 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.256580    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ad78ad05-b0af-4adf-a6bc-0f993b8a6288-sys-fs\") pod \"aws-ebs-csi-driver-node-7j9jw\" (UID: \"ad78ad05-b0af-4adf-a6bc-0f993b8a6288\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7j9jw"
Oct 30 18:33:57.256974 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.256599    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/63090f48-9143-4118-9906-d759c0aff52c-host-slash\") pod \"iptables-alerter-rcjmv\" (UID: \"63090f48-9143-4118-9906-d759c0aff52c\") " pod="openshift-network-operator/iptables-alerter-rcjmv"
Oct 30 18:33:57.256974 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.256612    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f2a80d89-61d1-4694-8734-004ce5cfb22b-etc-sysctl-d\") pod \"tuned-z9jmw\" (UID: \"f2a80d89-61d1-4694-8734-004ce5cfb22b\") " pod="openshift-cluster-node-tuning-operator/tuned-z9jmw"
Oct 30 18:33:57.256974 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.256627    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f2a80d89-61d1-4694-8734-004ce5cfb22b-etc-sysctl-conf\") pod \"tuned-z9jmw\" (UID: \"f2a80d89-61d1-4694-8734-004ce5cfb22b\") " pod="openshift-cluster-node-tuning-operator/tuned-z9jmw"
Oct 30 18:33:57.256974 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.256646    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/3f3fe764-2652-46c9-8e54-7882c0e6bcf1-systemd-units\") pod \"ovnkube-node-xfcdl\" (UID: \"3f3fe764-2652-46c9-8e54-7882c0e6bcf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xfcdl"
Oct 30 18:33:57.256974 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.256652    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/aa3f3b62-a1ab-4113-bef3-ed1b290e47c1-host-run-k8s-cni-cncf-io\") pod \"multus-9ssmv\" (UID: \"aa3f3b62-a1ab-4113-bef3-ed1b290e47c1\") " pod="openshift-multus/multus-9ssmv"
Oct 30 18:33:57.256974 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.256663    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/63090f48-9143-4118-9906-d759c0aff52c-host-slash\") pod \"iptables-alerter-rcjmv\" (UID: \"63090f48-9143-4118-9906-d759c0aff52c\") " pod="openshift-network-operator/iptables-alerter-rcjmv"
Oct 30 18:33:57.256974 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.256667    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3f3fe764-2652-46c9-8e54-7882c0e6bcf1-host-slash\") pod \"ovnkube-node-xfcdl\" (UID: \"3f3fe764-2652-46c9-8e54-7882c0e6bcf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xfcdl"
Oct 30 18:33:57.256974 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.256687    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ad78ad05-b0af-4adf-a6bc-0f993b8a6288-sys-fs\") pod \"aws-ebs-csi-driver-node-7j9jw\" (UID: \"ad78ad05-b0af-4adf-a6bc-0f993b8a6288\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7j9jw"
Oct 30 18:33:57.256974 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.256698    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3f3fe764-2652-46c9-8e54-7882c0e6bcf1-host-slash\") pod \"ovnkube-node-xfcdl\" (UID: \"3f3fe764-2652-46c9-8e54-7882c0e6bcf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xfcdl"
Oct 30 18:33:57.256974 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.256705    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/3f3fe764-2652-46c9-8e54-7882c0e6bcf1-systemd-units\") pod \"ovnkube-node-xfcdl\" (UID: \"3f3fe764-2652-46c9-8e54-7882c0e6bcf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xfcdl"
Oct 30 18:33:57.256974 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.256698    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/3f3fe764-2652-46c9-8e54-7882c0e6bcf1-run-systemd\") pod \"ovnkube-node-xfcdl\" (UID: \"3f3fe764-2652-46c9-8e54-7882c0e6bcf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xfcdl"
Oct 30 18:33:57.256974 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.256728    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/3f3fe764-2652-46c9-8e54-7882c0e6bcf1-run-systemd\") pod \"ovnkube-node-xfcdl\" (UID: \"3f3fe764-2652-46c9-8e54-7882c0e6bcf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xfcdl"
Oct 30 18:33:57.256974 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.256743    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/3f3fe764-2652-46c9-8e54-7882c0e6bcf1-node-log\") pod \"ovnkube-node-xfcdl\" (UID: \"3f3fe764-2652-46c9-8e54-7882c0e6bcf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xfcdl"
Oct 30 18:33:57.256974 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.256745    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f2a80d89-61d1-4694-8734-004ce5cfb22b-etc-sysctl-d\") pod \"tuned-z9jmw\" (UID: \"f2a80d89-61d1-4694-8734-004ce5cfb22b\") " pod="openshift-cluster-node-tuning-operator/tuned-z9jmw"
Oct 30 18:33:57.256974 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.256772    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2e49faaa-a718-44c8-9237-5a41169faee7-tuning-conf-dir\") pod \"multus-additional-cni-plugins-rzrjz\" (UID: \"2e49faaa-a718-44c8-9237-5a41169faee7\") " pod="openshift-multus/multus-additional-cni-plugins-rzrjz"
Oct 30 18:33:57.256974 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.256783    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f2a80d89-61d1-4694-8734-004ce5cfb22b-etc-sysctl-conf\") pod \"tuned-z9jmw\" (UID: \"f2a80d89-61d1-4694-8734-004ce5cfb22b\") " pod="openshift-cluster-node-tuning-operator/tuned-z9jmw"
Oct 30 18:33:57.256974 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.256799    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/3f3fe764-2652-46c9-8e54-7882c0e6bcf1-log-socket\") pod \"ovnkube-node-xfcdl\" (UID: \"3f3fe764-2652-46c9-8e54-7882c0e6bcf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xfcdl"
Oct 30 18:33:57.256974 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.256808    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2e49faaa-a718-44c8-9237-5a41169faee7-tuning-conf-dir\") pod \"multus-additional-cni-plugins-rzrjz\" (UID: \"2e49faaa-a718-44c8-9237-5a41169faee7\") " pod="openshift-multus/multus-additional-cni-plugins-rzrjz"
Oct 30 18:33:57.256974 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.256812    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/3f3fe764-2652-46c9-8e54-7882c0e6bcf1-node-log\") pod \"ovnkube-node-xfcdl\" (UID: \"3f3fe764-2652-46c9-8e54-7882c0e6bcf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xfcdl"
Oct 30 18:33:57.256974 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.256822    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/46a80637-6ccf-4a56-9904-f5e2fe5784ad-host\") pod \"node-ca-g78lj\" (UID: \"46a80637-6ccf-4a56-9904-f5e2fe5784ad\") " pod="openshift-image-registry/node-ca-g78lj"
Oct 30 18:33:57.256974 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.256838    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/3f3fe764-2652-46c9-8e54-7882c0e6bcf1-run-ovn\") pod \"ovnkube-node-xfcdl\" (UID: \"3f3fe764-2652-46c9-8e54-7882c0e6bcf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xfcdl"
Oct 30 18:33:57.256974 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.256850    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/3f3fe764-2652-46c9-8e54-7882c0e6bcf1-log-socket\") pod \"ovnkube-node-xfcdl\" (UID: \"3f3fe764-2652-46c9-8e54-7882c0e6bcf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xfcdl"
Oct 30 18:33:57.256974 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.256853    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/3f3fe764-2652-46c9-8e54-7882c0e6bcf1-host-cni-netd\") pod \"ovnkube-node-xfcdl\" (UID: \"3f3fe764-2652-46c9-8e54-7882c0e6bcf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xfcdl"
Oct 30 18:33:57.256974 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.256882    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a0823e8f-bbdd-477b-95ac-dad05d23cefa-metrics-certs\") pod \"network-metrics-daemon-v5phn\" (UID: \"a0823e8f-bbdd-477b-95ac-dad05d23cefa\") " pod="openshift-multus/network-metrics-daemon-v5phn"
Oct 30 18:33:57.256974 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.256894    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/3f3fe764-2652-46c9-8e54-7882c0e6bcf1-run-ovn\") pod \"ovnkube-node-xfcdl\" (UID: \"3f3fe764-2652-46c9-8e54-7882c0e6bcf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xfcdl"
Oct 30 18:33:57.256974 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.256909    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f2a80d89-61d1-4694-8734-004ce5cfb22b-etc-kubernetes\") pod \"tuned-z9jmw\" (UID: \"f2a80d89-61d1-4694-8734-004ce5cfb22b\") " pod="openshift-cluster-node-tuning-operator/tuned-z9jmw"
Oct 30 18:33:57.256974 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.256910    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/3f3fe764-2652-46c9-8e54-7882c0e6bcf1-host-cni-netd\") pod \"ovnkube-node-xfcdl\" (UID: \"3f3fe764-2652-46c9-8e54-7882c0e6bcf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xfcdl"
Oct 30 18:33:57.256974 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.256884    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/46a80637-6ccf-4a56-9904-f5e2fe5784ad-host\") pod \"node-ca-g78lj\" (UID: \"46a80637-6ccf-4a56-9904-f5e2fe5784ad\") " pod="openshift-image-registry/node-ca-g78lj"
Oct 30 18:33:57.256974 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.256925    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3f3fe764-2652-46c9-8e54-7882c0e6bcf1-var-lib-openvswitch\") pod \"ovnkube-node-xfcdl\" (UID: \"3f3fe764-2652-46c9-8e54-7882c0e6bcf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xfcdl"
Oct 30 18:33:57.256974 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.256962    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/aa3f3b62-a1ab-4113-bef3-ed1b290e47c1-system-cni-dir\") pod \"multus-9ssmv\" (UID: \"aa3f3b62-a1ab-4113-bef3-ed1b290e47c1\") " pod="openshift-multus/multus-9ssmv"
Oct 30 18:33:57.256974 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.256966    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3f3fe764-2652-46c9-8e54-7882c0e6bcf1-var-lib-openvswitch\") pod \"ovnkube-node-xfcdl\" (UID: \"3f3fe764-2652-46c9-8e54-7882c0e6bcf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xfcdl"
Oct 30 18:33:57.256974 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:33:57.256967    4888 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Oct 30 18:33:57.256974 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.256988    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/aa3f3b62-a1ab-4113-bef3-ed1b290e47c1-multus-cni-dir\") pod \"multus-9ssmv\" (UID: \"aa3f3b62-a1ab-4113-bef3-ed1b290e47c1\") " pod="openshift-multus/multus-9ssmv"
Oct 30 18:33:57.258307 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.257009    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ad78ad05-b0af-4adf-a6bc-0f993b8a6288-kubelet-dir\") pod \"aws-ebs-csi-driver-node-7j9jw\" (UID: \"ad78ad05-b0af-4adf-a6bc-0f993b8a6288\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7j9jw"
Oct 30 18:33:57.258307 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:33:57.257021    4888 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a0823e8f-bbdd-477b-95ac-dad05d23cefa-metrics-certs podName:a0823e8f-bbdd-477b-95ac-dad05d23cefa nodeName:}" failed. No retries permitted until 2026-10-30 18:33:57.757010877 +0000 UTC m=+2.004416903 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a0823e8f-bbdd-477b-95ac-dad05d23cefa-metrics-certs") pod "network-metrics-daemon-v5phn" (UID: "a0823e8f-bbdd-477b-95ac-dad05d23cefa") : secret "metrics-daemon-secret" not found
Oct 30 18:33:57.258307 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.257039    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ad78ad05-b0af-4adf-a6bc-0f993b8a6288-kubelet-dir\") pod \"aws-ebs-csi-driver-node-7j9jw\" (UID: \"ad78ad05-b0af-4adf-a6bc-0f993b8a6288\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7j9jw"
Oct 30 18:33:57.258307 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.257040    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3f3fe764-2652-46c9-8e54-7882c0e6bcf1-host-cni-bin\") pod \"ovnkube-node-xfcdl\" (UID: \"3f3fe764-2652-46c9-8e54-7882c0e6bcf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xfcdl"
Oct 30 18:33:57.258307 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.257064    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3f3fe764-2652-46c9-8e54-7882c0e6bcf1-host-cni-bin\") pod \"ovnkube-node-xfcdl\" (UID: \"3f3fe764-2652-46c9-8e54-7882c0e6bcf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xfcdl"
Oct 30 18:33:57.258307 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.256974    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f2a80d89-61d1-4694-8734-004ce5cfb22b-etc-kubernetes\") pod \"tuned-z9jmw\" (UID: \"f2a80d89-61d1-4694-8734-004ce5cfb22b\") " pod="openshift-cluster-node-tuning-operator/tuned-z9jmw"
Oct 30 18:33:57.258307 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.257066    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/aa3f3b62-a1ab-4113-bef3-ed1b290e47c1-multus-conf-dir\") pod \"multus-9ssmv\" (UID: \"aa3f3b62-a1ab-4113-bef3-ed1b290e47c1\") " pod="openshift-multus/multus-9ssmv"
Oct 30 18:33:57.258307 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.257088    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/aa3f3b62-a1ab-4113-bef3-ed1b290e47c1-multus-cni-dir\") pod \"multus-9ssmv\" (UID: \"aa3f3b62-a1ab-4113-bef3-ed1b290e47c1\") " pod="openshift-multus/multus-9ssmv"
Oct 30 18:33:57.258307 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.257095    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/aa3f3b62-a1ab-4113-bef3-ed1b290e47c1-multus-conf-dir\") pod \"multus-9ssmv\" (UID: \"aa3f3b62-a1ab-4113-bef3-ed1b290e47c1\") " pod="openshift-multus/multus-9ssmv"
Oct 30 18:33:57.258307 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.257101    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/da28351c-1c8b-40be-85a0-3aad427e3a17-service-ca-bundle\") pod \"router-default-7dfcf9fd6c-vh769\" (UID: \"da28351c-1c8b-40be-85a0-3aad427e3a17\") " pod="openshift-ingress/router-default-7dfcf9fd6c-vh769"
Oct 30 18:33:57.258307 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.257130    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f2a80d89-61d1-4694-8734-004ce5cfb22b-var-lib-kubelet\") pod \"tuned-z9jmw\" (UID: \"f2a80d89-61d1-4694-8734-004ce5cfb22b\") " pod="openshift-cluster-node-tuning-operator/tuned-z9jmw"
Oct 30 18:33:57.258307 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.257157    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3f3fe764-2652-46c9-8e54-7882c0e6bcf1-host-run-netns\") pod \"ovnkube-node-xfcdl\" (UID: \"3f3fe764-2652-46c9-8e54-7882c0e6bcf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xfcdl"
Oct 30 18:33:57.258307 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.257179    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3f3fe764-2652-46c9-8e54-7882c0e6bcf1-host-run-netns\") pod \"ovnkube-node-xfcdl\" (UID: \"3f3fe764-2652-46c9-8e54-7882c0e6bcf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xfcdl"
Oct 30 18:33:57.258307 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.257066    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/aa3f3b62-a1ab-4113-bef3-ed1b290e47c1-system-cni-dir\") pod \"multus-9ssmv\" (UID: \"aa3f3b62-a1ab-4113-bef3-ed1b290e47c1\") " pod="openshift-multus/multus-9ssmv"
Oct 30 18:33:57.258307 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:33:57.257192    4888 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/da28351c-1c8b-40be-85a0-3aad427e3a17-service-ca-bundle podName:da28351c-1c8b-40be-85a0-3aad427e3a17 nodeName:}" failed. No retries permitted until 2026-10-30 18:33:57.757176946 +0000 UTC m=+2.004582964 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/da28351c-1c8b-40be-85a0-3aad427e3a17-service-ca-bundle") pod "router-default-7dfcf9fd6c-vh769" (UID: "da28351c-1c8b-40be-85a0-3aad427e3a17") : configmap references non-existent config key: service-ca.crt
Oct 30 18:33:57.258307 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.257181    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3f3fe764-2652-46c9-8e54-7882c0e6bcf1-run-openvswitch\") pod \"ovnkube-node-xfcdl\" (UID: \"3f3fe764-2652-46c9-8e54-7882c0e6bcf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xfcdl"
Oct 30 18:33:57.258307 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.257210    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3f3fe764-2652-46c9-8e54-7882c0e6bcf1-run-openvswitch\") pod \"ovnkube-node-xfcdl\" (UID: \"3f3fe764-2652-46c9-8e54-7882c0e6bcf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xfcdl"
Oct 30 18:33:57.258307 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.257230    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/57ca8600-79e7-4f59-9d11-b1c9ec3d358a-crio-socket\") pod \"insights-runtime-extractor-jbsw6\" (UID: \"57ca8600-79e7-4f59-9d11-b1c9ec3d358a\") " pod="openshift-insights/insights-runtime-extractor-jbsw6"
Oct 30 18:33:57.258307 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.257248    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/aa3f3b62-a1ab-4113-bef3-ed1b290e47c1-cnibin\") pod \"multus-9ssmv\" (UID: \"aa3f3b62-a1ab-4113-bef3-ed1b290e47c1\") " pod="openshift-multus/multus-9ssmv"
Oct 30 18:33:57.258307 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.257160    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f2a80d89-61d1-4694-8734-004ce5cfb22b-var-lib-kubelet\") pod \"tuned-z9jmw\" (UID: \"f2a80d89-61d1-4694-8734-004ce5cfb22b\") " pod="openshift-cluster-node-tuning-operator/tuned-z9jmw"
Oct 30 18:33:57.258307 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.257263    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/aa3f3b62-a1ab-4113-bef3-ed1b290e47c1-host-var-lib-cni-multus\") pod \"multus-9ssmv\" (UID: \"aa3f3b62-a1ab-4113-bef3-ed1b290e47c1\") " pod="openshift-multus/multus-9ssmv"
Oct 30 18:33:57.258307 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.257299    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/57ca8600-79e7-4f59-9d11-b1c9ec3d358a-crio-socket\") pod \"insights-runtime-extractor-jbsw6\" (UID: \"57ca8600-79e7-4f59-9d11-b1c9ec3d358a\") " pod="openshift-insights/insights-runtime-extractor-jbsw6"
Oct 30 18:33:57.258307 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.257310    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/aa3f3b62-a1ab-4113-bef3-ed1b290e47c1-cnibin\") pod \"multus-9ssmv\" (UID: \"aa3f3b62-a1ab-4113-bef3-ed1b290e47c1\") " pod="openshift-multus/multus-9ssmv"
Oct 30 18:33:57.258307 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.257301    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f2a80d89-61d1-4694-8734-004ce5cfb22b-host\") pod \"tuned-z9jmw\" (UID: \"f2a80d89-61d1-4694-8734-004ce5cfb22b\") " pod="openshift-cluster-node-tuning-operator/tuned-z9jmw"
Oct 30 18:33:57.258307 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.257327    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/aa3f3b62-a1ab-4113-bef3-ed1b290e47c1-host-var-lib-cni-multus\") pod \"multus-9ssmv\" (UID: \"aa3f3b62-a1ab-4113-bef3-ed1b290e47c1\") " pod="openshift-multus/multus-9ssmv"
Oct 30 18:33:57.258307 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.257337    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f2a80d89-61d1-4694-8734-004ce5cfb22b-run\") pod \"tuned-z9jmw\" (UID: \"f2a80d89-61d1-4694-8734-004ce5cfb22b\") " pod="openshift-cluster-node-tuning-operator/tuned-z9jmw"
Oct 30 18:33:57.258307 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.257330    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f2a80d89-61d1-4694-8734-004ce5cfb22b-host\") pod \"tuned-z9jmw\" (UID: \"f2a80d89-61d1-4694-8734-004ce5cfb22b\") " pod="openshift-cluster-node-tuning-operator/tuned-z9jmw"
Oct 30 18:33:57.258307 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.257354    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f2a80d89-61d1-4694-8734-004ce5cfb22b-lib-modules\") pod \"tuned-z9jmw\" (UID: \"f2a80d89-61d1-4694-8734-004ce5cfb22b\") " pod="openshift-cluster-node-tuning-operator/tuned-z9jmw"
Oct 30 18:33:57.258307 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.257383    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f2a80d89-61d1-4694-8734-004ce5cfb22b-run\") pod \"tuned-z9jmw\" (UID: \"f2a80d89-61d1-4694-8734-004ce5cfb22b\") " pod="openshift-cluster-node-tuning-operator/tuned-z9jmw"
Oct 30 18:33:57.258307 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.257443    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/da28351c-1c8b-40be-85a0-3aad427e3a17-metrics-certs\") pod \"router-default-7dfcf9fd6c-vh769\" (UID: \"da28351c-1c8b-40be-85a0-3aad427e3a17\") " pod="openshift-ingress/router-default-7dfcf9fd6c-vh769"
Oct 30 18:33:57.258307 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.257468    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f2a80d89-61d1-4694-8734-004ce5cfb22b-lib-modules\") pod \"tuned-z9jmw\" (UID: \"f2a80d89-61d1-4694-8734-004ce5cfb22b\") " pod="openshift-cluster-node-tuning-operator/tuned-z9jmw"
Oct 30 18:33:57.258307 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.257502    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2e49faaa-a718-44c8-9237-5a41169faee7-system-cni-dir\") pod \"multus-additional-cni-plugins-rzrjz\" (UID: \"2e49faaa-a718-44c8-9237-5a41169faee7\") " pod="openshift-multus/multus-additional-cni-plugins-rzrjz"
Oct 30 18:33:57.258307 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.257530    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/46d9ac39-ee45-4e43-bda7-3931c3e599cf-kubelet-config\") pod \"global-pull-secret-syncer-g2svp\" (UID: \"46d9ac39-ee45-4e43-bda7-3931c3e599cf\") " pod="kube-system/global-pull-secret-syncer-g2svp"
Oct 30 18:33:57.258307 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.257544    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/46d9ac39-ee45-4e43-bda7-3931c3e599cf-dbus\") pod \"global-pull-secret-syncer-g2svp\" (UID: \"46d9ac39-ee45-4e43-bda7-3931c3e599cf\") " pod="kube-system/global-pull-secret-syncer-g2svp"
Oct 30 18:33:57.258307 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:33:57.257530    4888 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Oct 30 18:33:57.258307 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.257566    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2e49faaa-a718-44c8-9237-5a41169faee7-system-cni-dir\") pod \"multus-additional-cni-plugins-rzrjz\" (UID: \"2e49faaa-a718-44c8-9237-5a41169faee7\") " pod="openshift-multus/multus-additional-cni-plugins-rzrjz"
Oct 30 18:33:57.258307 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.257574    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/57ca8600-79e7-4f59-9d11-b1c9ec3d358a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jbsw6\" (UID: \"57ca8600-79e7-4f59-9d11-b1c9ec3d358a\") " pod="openshift-insights/insights-runtime-extractor-jbsw6"
Oct 30 18:33:57.258307 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:33:57.257608    4888 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/da28351c-1c8b-40be-85a0-3aad427e3a17-metrics-certs podName:da28351c-1c8b-40be-85a0-3aad427e3a17 nodeName:}" failed. No retries permitted until 2026-10-30 18:33:57.757595088 +0000 UTC m=+2.005001094 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/da28351c-1c8b-40be-85a0-3aad427e3a17-metrics-certs") pod "router-default-7dfcf9fd6c-vh769" (UID: "da28351c-1c8b-40be-85a0-3aad427e3a17") : secret "router-metrics-certs-default" not found
Oct 30 18:33:57.258307 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.257613    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/46d9ac39-ee45-4e43-bda7-3931c3e599cf-kubelet-config\") pod \"global-pull-secret-syncer-g2svp\" (UID: \"46d9ac39-ee45-4e43-bda7-3931c3e599cf\") " pod="kube-system/global-pull-secret-syncer-g2svp"
Oct 30 18:33:57.258307 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.257620    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/46d9ac39-ee45-4e43-bda7-3931c3e599cf-dbus\") pod \"global-pull-secret-syncer-g2svp\" (UID: \"46d9ac39-ee45-4e43-bda7-3931c3e599cf\") " pod="kube-system/global-pull-secret-syncer-g2svp"
Oct 30 18:33:57.258307 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.257626    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/aa3f3b62-a1ab-4113-bef3-ed1b290e47c1-host-var-lib-kubelet\") pod \"multus-9ssmv\" (UID: \"aa3f3b62-a1ab-4113-bef3-ed1b290e47c1\") " pod="openshift-multus/multus-9ssmv"
Oct 30 18:33:57.258307 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.257650    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/aa3f3b62-a1ab-4113-bef3-ed1b290e47c1-host-var-lib-kubelet\") pod \"multus-9ssmv\" (UID: \"aa3f3b62-a1ab-4113-bef3-ed1b290e47c1\") " pod="openshift-multus/multus-9ssmv"
Oct 30 18:33:57.258307 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.257651    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ad78ad05-b0af-4adf-a6bc-0f993b8a6288-etc-selinux\") pod \"aws-ebs-csi-driver-node-7j9jw\" (UID: \"ad78ad05-b0af-4adf-a6bc-0f993b8a6288\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7j9jw"
Oct 30 18:33:57.258307 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.257674    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/3f3fe764-2652-46c9-8e54-7882c0e6bcf1-host-kubelet\") pod \"ovnkube-node-xfcdl\" (UID: \"3f3fe764-2652-46c9-8e54-7882c0e6bcf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xfcdl"
Oct 30 18:33:57.258307 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.257679    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ad78ad05-b0af-4adf-a6bc-0f993b8a6288-etc-selinux\") pod \"aws-ebs-csi-driver-node-7j9jw\" (UID: \"ad78ad05-b0af-4adf-a6bc-0f993b8a6288\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7j9jw"
Oct 30 18:33:57.258307 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.257696    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/97b19aa3-42be-4def-8d7a-8e3516852e79-metrics-tls\") pod \"dns-default-r2nx2\" (UID: \"97b19aa3-42be-4def-8d7a-8e3516852e79\") " pod="openshift-dns/dns-default-r2nx2"
Oct 30 18:33:57.258307 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.257710    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ad78ad05-b0af-4adf-a6bc-0f993b8a6288-device-dir\") pod \"aws-ebs-csi-driver-node-7j9jw\" (UID: \"ad78ad05-b0af-4adf-a6bc-0f993b8a6288\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7j9jw"
Oct 30 18:33:57.258307 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.257725    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f2a80d89-61d1-4694-8734-004ce5cfb22b-etc-systemd\") pod \"tuned-z9jmw\" (UID: \"f2a80d89-61d1-4694-8734-004ce5cfb22b\") " pod="openshift-cluster-node-tuning-operator/tuned-z9jmw"
Oct 30 18:33:57.258307 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.257739    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8789ec5b-226e-4551-bf68-3ac585006266-registry-tls\") pod \"image-registry-74757bccd5-qmrkp\" (UID: \"8789ec5b-226e-4551-bf68-3ac585006266\") " pod="openshift-image-registry/image-registry-74757bccd5-qmrkp"
Oct 30 18:33:57.258307 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.257759    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/aa3f3b62-a1ab-4113-bef3-ed1b290e47c1-hostroot\") pod \"multus-9ssmv\" (UID: \"aa3f3b62-a1ab-4113-bef3-ed1b290e47c1\") " pod="openshift-multus/multus-9ssmv"
Oct 30 18:33:57.258307 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.257778    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ad78ad05-b0af-4adf-a6bc-0f993b8a6288-device-dir\") pod \"aws-ebs-csi-driver-node-7j9jw\" (UID: \"ad78ad05-b0af-4adf-a6bc-0f993b8a6288\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7j9jw"
Oct 30 18:33:57.258307 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.257785    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f2a80d89-61d1-4694-8734-004ce5cfb22b-etc-systemd\") pod \"tuned-z9jmw\" (UID: \"f2a80d89-61d1-4694-8734-004ce5cfb22b\") " pod="openshift-cluster-node-tuning-operator/tuned-z9jmw"
Oct 30 18:33:57.258307 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.257782    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/aa3f3b62-a1ab-4113-bef3-ed1b290e47c1-host-run-multus-certs\") pod \"multus-9ssmv\" (UID: \"aa3f3b62-a1ab-4113-bef3-ed1b290e47c1\") " pod="openshift-multus/multus-9ssmv"
Oct 30 18:33:57.258307 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.257786    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/3f3fe764-2652-46c9-8e54-7882c0e6bcf1-host-kubelet\") pod \"ovnkube-node-xfcdl\" (UID: \"3f3fe764-2652-46c9-8e54-7882c0e6bcf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xfcdl"
Oct 30 18:33:57.258307 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:33:57.257781    4888 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 30 18:33:57.258307 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.257824    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/aa3f3b62-a1ab-4113-bef3-ed1b290e47c1-host-run-multus-certs\") pod \"multus-9ssmv\" (UID: \"aa3f3b62-a1ab-4113-bef3-ed1b290e47c1\") " pod="openshift-multus/multus-9ssmv"
Oct 30 18:33:57.258307 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:33:57.257832    4888 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 30 18:33:57.258307 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:33:57.257855    4888 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-74757bccd5-qmrkp: secret "image-registry-tls" not found
Oct 30 18:33:57.258307 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.257863    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/aa3f3b62-a1ab-4113-bef3-ed1b290e47c1-hostroot\") pod \"multus-9ssmv\" (UID: \"aa3f3b62-a1ab-4113-bef3-ed1b290e47c1\") " pod="openshift-multus/multus-9ssmv"
Oct 30 18:33:57.258307 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.257880    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/15d485f9-5a4b-4f4b-9e59-a6eb062b4c5c-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-pm58m\" (UID: \"15d485f9-5a4b-4f4b-9e59-a6eb062b4c5c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-pm58m"
Oct 30 18:33:57.258307 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:33:57.257894    4888 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8789ec5b-226e-4551-bf68-3ac585006266-registry-tls podName:8789ec5b-226e-4551-bf68-3ac585006266 nodeName:}" failed. No retries permitted until 2026-10-30 18:33:57.757885711 +0000 UTC m=+2.005291730 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8789ec5b-226e-4551-bf68-3ac585006266-registry-tls") pod "image-registry-74757bccd5-qmrkp" (UID: "8789ec5b-226e-4551-bf68-3ac585006266") : secret "image-registry-tls" not found
Oct 30 18:33:57.258307 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:33:57.257921    4888 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/97b19aa3-42be-4def-8d7a-8e3516852e79-metrics-tls podName:97b19aa3-42be-4def-8d7a-8e3516852e79 nodeName:}" failed. No retries permitted until 2026-10-30 18:33:57.757915376 +0000 UTC m=+2.005321378 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/97b19aa3-42be-4def-8d7a-8e3516852e79-metrics-tls") pod "dns-default-r2nx2" (UID: "97b19aa3-42be-4def-8d7a-8e3516852e79") : secret "dns-default-metrics-tls" not found
Oct 30 18:33:57.258307 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:33:57.257932    4888 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Oct 30 18:33:57.258307 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.257940    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3f3fe764-2652-46c9-8e54-7882c0e6bcf1-etc-openvswitch\") pod \"ovnkube-node-xfcdl\" (UID: \"3f3fe764-2652-46c9-8e54-7882c0e6bcf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xfcdl"
Oct 30 18:33:57.258307 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.257956    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2e49faaa-a718-44c8-9237-5a41169faee7-cnibin\") pod \"multus-additional-cni-plugins-rzrjz\" (UID: \"2e49faaa-a718-44c8-9237-5a41169faee7\") " pod="openshift-multus/multus-additional-cni-plugins-rzrjz"
Oct 30 18:33:57.258307 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:33:57.257962    4888 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/15d485f9-5a4b-4f4b-9e59-a6eb062b4c5c-samples-operator-tls podName:15d485f9-5a4b-4f4b-9e59-a6eb062b4c5c nodeName:}" failed. No retries permitted until 2026-10-30 18:33:57.757954167 +0000 UTC m=+2.005360170 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/15d485f9-5a4b-4f4b-9e59-a6eb062b4c5c-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-pm58m" (UID: "15d485f9-5a4b-4f4b-9e59-a6eb062b4c5c") : secret "samples-operator-tls" not found
Oct 30 18:33:57.258307 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.257978    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f2a80d89-61d1-4694-8734-004ce5cfb22b-etc-sysconfig\") pod \"tuned-z9jmw\" (UID: \"f2a80d89-61d1-4694-8734-004ce5cfb22b\") " pod="openshift-cluster-node-tuning-operator/tuned-z9jmw"
Oct 30 18:33:57.258307 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.257978    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3f3fe764-2652-46c9-8e54-7882c0e6bcf1-etc-openvswitch\") pod \"ovnkube-node-xfcdl\" (UID: \"3f3fe764-2652-46c9-8e54-7882c0e6bcf1\") " pod="openshift-ovn-kubernetes/ovnkube-node-xfcdl"
Oct 30 18:33:57.258307 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.257994    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f2a80d89-61d1-4694-8734-004ce5cfb22b-sys\") pod \"tuned-z9jmw\" (UID: \"f2a80d89-61d1-4694-8734-004ce5cfb22b\") " pod="openshift-cluster-node-tuning-operator/tuned-z9jmw"
Oct 30 18:33:57.258307 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.257995    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2e49faaa-a718-44c8-9237-5a41169faee7-cnibin\") pod \"multus-additional-cni-plugins-rzrjz\" (UID: \"2e49faaa-a718-44c8-9237-5a41169faee7\") " pod="openshift-multus/multus-additional-cni-plugins-rzrjz"
Oct 30 18:33:57.258307 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.258017    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/71fce796-0032-481b-80b3-66fbb731452c-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-r8rgt\" (UID: \"71fce796-0032-481b-80b3-66fbb731452c\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-r8rgt"
Oct 30 18:33:57.258307 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.258031    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f2a80d89-61d1-4694-8734-004ce5cfb22b-etc-sysconfig\") pod \"tuned-z9jmw\" (UID: \"f2a80d89-61d1-4694-8734-004ce5cfb22b\") " pod="openshift-cluster-node-tuning-operator/tuned-z9jmw"
Oct 30 18:33:57.258307 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.258037    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/57ca8600-79e7-4f59-9d11-b1c9ec3d358a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jbsw6\" (UID: \"57ca8600-79e7-4f59-9d11-b1c9ec3d358a\") " pod="openshift-insights/insights-runtime-extractor-jbsw6"
Oct 30 18:33:57.258307 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.258049    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/57ca8600-79e7-4f59-9d11-b1c9ec3d358a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jbsw6\" (UID: \"57ca8600-79e7-4f59-9d11-b1c9ec3d358a\") " pod="openshift-insights/insights-runtime-extractor-jbsw6"
Oct 30 18:33:57.258307 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:33:57.258068    4888 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Oct 30 18:33:57.258307 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.258069    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f2a80d89-61d1-4694-8734-004ce5cfb22b-etc-modprobe-d\") pod \"tuned-z9jmw\" (UID: \"f2a80d89-61d1-4694-8734-004ce5cfb22b\") " pod="openshift-cluster-node-tuning-operator/tuned-z9jmw"
Oct 30 18:33:57.258307 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:33:57.258082    4888 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 30 18:33:57.258307 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.258103    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/68195e19-b1ff-4148-9fae-3545f2997ff5-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-gmswj\" (UID: \"68195e19-b1ff-4148-9fae-3545f2997ff5\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-gmswj"
Oct 30 18:33:57.258307 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:33:57.258108    4888 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/57ca8600-79e7-4f59-9d11-b1c9ec3d358a-insights-runtime-extractor-tls podName:57ca8600-79e7-4f59-9d11-b1c9ec3d358a nodeName:}" failed. No retries permitted until 2026-10-30 18:33:57.758101991 +0000 UTC m=+2.005507993 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/57ca8600-79e7-4f59-9d11-b1c9ec3d358a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jbsw6" (UID: "57ca8600-79e7-4f59-9d11-b1c9ec3d358a") : secret "insights-runtime-extractor-tls" not found
Oct 30 18:33:57.258307 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.258053    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f2a80d89-61d1-4694-8734-004ce5cfb22b-sys\") pod \"tuned-z9jmw\" (UID: \"f2a80d89-61d1-4694-8734-004ce5cfb22b\") " pod="openshift-cluster-node-tuning-operator/tuned-z9jmw"
Oct 30 18:33:57.258307 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:33:57.258149    4888 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Oct 30 18:33:57.258307 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:33:57.258162    4888 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/71fce796-0032-481b-80b3-66fbb731452c-networking-console-plugin-cert podName:71fce796-0032-481b-80b3-66fbb731452c nodeName:}" failed. No retries permitted until 2026-10-30 18:33:57.758144114 +0000 UTC m=+2.005550123 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/71fce796-0032-481b-80b3-66fbb731452c-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-r8rgt" (UID: "71fce796-0032-481b-80b3-66fbb731452c") : secret "networking-console-plugin-cert" not found
Oct 30 18:33:57.258307 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.258187    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d81cef55-16ee-4677-ae12-86de3c9eb216-hosts-file\") pod \"node-resolver-76f9s\" (UID: \"d81cef55-16ee-4677-ae12-86de3c9eb216\") " pod="openshift-dns/node-resolver-76f9s"
Oct 30 18:33:57.258307 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.258178    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f2a80d89-61d1-4694-8734-004ce5cfb22b-etc-modprobe-d\") pod \"tuned-z9jmw\" (UID: \"f2a80d89-61d1-4694-8734-004ce5cfb22b\") " pod="openshift-cluster-node-tuning-operator/tuned-z9jmw"
Oct 30 18:33:57.258307 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:33:57.258263    4888 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68195e19-b1ff-4148-9fae-3545f2997ff5-cluster-monitoring-operator-tls podName:68195e19-b1ff-4148-9fae-3545f2997ff5 nodeName:}" failed. No retries permitted until 2026-10-30 18:33:57.758246053 +0000 UTC m=+2.005652057 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/68195e19-b1ff-4148-9fae-3545f2997ff5-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-gmswj" (UID: "68195e19-b1ff-4148-9fae-3545f2997ff5") : secret "cluster-monitoring-operator-tls" not found
Oct 30 18:33:57.260581 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.258308    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d81cef55-16ee-4677-ae12-86de3c9eb216-hosts-file\") pod \"node-resolver-76f9s\" (UID: \"d81cef55-16ee-4677-ae12-86de3c9eb216\") " pod="openshift-dns/node-resolver-76f9s"
Oct 30 18:33:57.291446 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.291430    4888 scope.go:122] "RemoveContainer" containerID="b7e8a752d2699b7e5514e278ff1b7c9fd161b532812a42ac3605f0cdaad0c160"
Oct 30 18:33:57.304777 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.304760    4888 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 18:33:57.762833 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.762759    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a0823e8f-bbdd-477b-95ac-dad05d23cefa-metrics-certs\") pod \"network-metrics-daemon-v5phn\" (UID: \"a0823e8f-bbdd-477b-95ac-dad05d23cefa\") " pod="openshift-multus/network-metrics-daemon-v5phn"
Oct 30 18:33:57.762833 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.762799    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/da28351c-1c8b-40be-85a0-3aad427e3a17-service-ca-bundle\") pod \"router-default-7dfcf9fd6c-vh769\" (UID: \"da28351c-1c8b-40be-85a0-3aad427e3a17\") " pod="openshift-ingress/router-default-7dfcf9fd6c-vh769"
Oct 30 18:33:57.762833 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.762827    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/da28351c-1c8b-40be-85a0-3aad427e3a17-metrics-certs\") pod \"router-default-7dfcf9fd6c-vh769\" (UID: \"da28351c-1c8b-40be-85a0-3aad427e3a17\") " pod="openshift-ingress/router-default-7dfcf9fd6c-vh769"
Oct 30 18:33:57.763067 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.762848    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/97b19aa3-42be-4def-8d7a-8e3516852e79-metrics-tls\") pod \"dns-default-r2nx2\" (UID: \"97b19aa3-42be-4def-8d7a-8e3516852e79\") " pod="openshift-dns/dns-default-r2nx2"
Oct 30 18:33:57.763067 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.762884    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8789ec5b-226e-4551-bf68-3ac585006266-registry-tls\") pod \"image-registry-74757bccd5-qmrkp\" (UID: \"8789ec5b-226e-4551-bf68-3ac585006266\") " pod="openshift-image-registry/image-registry-74757bccd5-qmrkp"
Oct 30 18:33:57.763067 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.762905    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/15d485f9-5a4b-4f4b-9e59-a6eb062b4c5c-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-pm58m\" (UID: \"15d485f9-5a4b-4f4b-9e59-a6eb062b4c5c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-pm58m"
Oct 30 18:33:57.763067 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:33:57.762921    4888 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Oct 30 18:33:57.763067 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:33:57.762936    4888 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/da28351c-1c8b-40be-85a0-3aad427e3a17-service-ca-bundle podName:da28351c-1c8b-40be-85a0-3aad427e3a17 nodeName:}" failed. No retries permitted until 2026-10-30 18:33:58.762915085 +0000 UTC m=+3.010321104 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/da28351c-1c8b-40be-85a0-3aad427e3a17-service-ca-bundle") pod "router-default-7dfcf9fd6c-vh769" (UID: "da28351c-1c8b-40be-85a0-3aad427e3a17") : configmap references non-existent config key: service-ca.crt
Oct 30 18:33:57.763067 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:33:57.762983    4888 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 30 18:33:57.763067 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:33:57.762989    4888 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a0823e8f-bbdd-477b-95ac-dad05d23cefa-metrics-certs podName:a0823e8f-bbdd-477b-95ac-dad05d23cefa nodeName:}" failed. No retries permitted until 2026-10-30 18:33:58.762969238 +0000 UTC m=+3.010375262 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a0823e8f-bbdd-477b-95ac-dad05d23cefa-metrics-certs") pod "network-metrics-daemon-v5phn" (UID: "a0823e8f-bbdd-477b-95ac-dad05d23cefa") : secret "metrics-daemon-secret" not found
Oct 30 18:33:57.763067 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:33:57.762996    4888 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Oct 30 18:33:57.763067 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:33:57.763001    4888 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 30 18:33:57.763067 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:33:57.763014    4888 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-74757bccd5-qmrkp: secret "image-registry-tls" not found
Oct 30 18:33:57.763067 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:33:57.763013    4888 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Oct 30 18:33:57.763067 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.762979    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/71fce796-0032-481b-80b3-66fbb731452c-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-r8rgt\" (UID: \"71fce796-0032-481b-80b3-66fbb731452c\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-r8rgt"
Oct 30 18:33:57.763067 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:33:57.763038    4888 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Oct 30 18:33:57.763067 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:33:57.763022    4888 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/97b19aa3-42be-4def-8d7a-8e3516852e79-metrics-tls podName:97b19aa3-42be-4def-8d7a-8e3516852e79 nodeName:}" failed. No retries permitted until 2026-10-30 18:33:58.763010416 +0000 UTC m=+3.010416435 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/97b19aa3-42be-4def-8d7a-8e3516852e79-metrics-tls") pod "dns-default-r2nx2" (UID: "97b19aa3-42be-4def-8d7a-8e3516852e79") : secret "dns-default-metrics-tls" not found
Oct 30 18:33:57.763067 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:33:57.763078    4888 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/15d485f9-5a4b-4f4b-9e59-a6eb062b4c5c-samples-operator-tls podName:15d485f9-5a4b-4f4b-9e59-a6eb062b4c5c nodeName:}" failed. No retries permitted until 2026-10-30 18:33:58.76306487 +0000 UTC m=+3.010470876 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/15d485f9-5a4b-4f4b-9e59-a6eb062b4c5c-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-pm58m" (UID: "15d485f9-5a4b-4f4b-9e59-a6eb062b4c5c") : secret "samples-operator-tls" not found
Oct 30 18:33:57.763611 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:33:57.763105    4888 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8789ec5b-226e-4551-bf68-3ac585006266-registry-tls podName:8789ec5b-226e-4551-bf68-3ac585006266 nodeName:}" failed. No retries permitted until 2026-10-30 18:33:58.763095764 +0000 UTC m=+3.010501782 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8789ec5b-226e-4551-bf68-3ac585006266-registry-tls") pod "image-registry-74757bccd5-qmrkp" (UID: "8789ec5b-226e-4551-bf68-3ac585006266") : secret "image-registry-tls" not found
Oct 30 18:33:57.763611 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:33:57.763123    4888 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/da28351c-1c8b-40be-85a0-3aad427e3a17-metrics-certs podName:da28351c-1c8b-40be-85a0-3aad427e3a17 nodeName:}" failed. No retries permitted until 2026-10-30 18:33:58.763118013 +0000 UTC m=+3.010524017 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/da28351c-1c8b-40be-85a0-3aad427e3a17-metrics-certs") pod "router-default-7dfcf9fd6c-vh769" (UID: "da28351c-1c8b-40be-85a0-3aad427e3a17") : secret "router-metrics-certs-default" not found
Oct 30 18:33:57.763611 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:33:57.763136    4888 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/71fce796-0032-481b-80b3-66fbb731452c-networking-console-plugin-cert podName:71fce796-0032-481b-80b3-66fbb731452c nodeName:}" failed. No retries permitted until 2026-10-30 18:33:58.76313132 +0000 UTC m=+3.010537323 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/71fce796-0032-481b-80b3-66fbb731452c-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-r8rgt" (UID: "71fce796-0032-481b-80b3-66fbb731452c") : secret "networking-console-plugin-cert" not found
Oct 30 18:33:57.763611 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.763152    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/57ca8600-79e7-4f59-9d11-b1c9ec3d358a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jbsw6\" (UID: \"57ca8600-79e7-4f59-9d11-b1c9ec3d358a\") " pod="openshift-insights/insights-runtime-extractor-jbsw6"
Oct 30 18:33:57.763611 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.763177    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/68195e19-b1ff-4148-9fae-3545f2997ff5-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-gmswj\" (UID: \"68195e19-b1ff-4148-9fae-3545f2997ff5\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-gmswj"
Oct 30 18:33:57.763611 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.763212    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351-registry-tls\") pod \"image-registry-64bf5b6b76-dpvvs\" (UID: \"9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351\") " pod="openshift-image-registry/image-registry-64bf5b6b76-dpvvs"
Oct 30 18:33:57.763611 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:33:57.763273    4888 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Oct 30 18:33:57.763611 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:33:57.763319    4888 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 30 18:33:57.763611 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:33:57.763326    4888 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 30 18:33:57.763611 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:33:57.763340    4888 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-64bf5b6b76-dpvvs: secret "image-registry-tls" not found
Oct 30 18:33:57.763611 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:33:57.763327    4888 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68195e19-b1ff-4148-9fae-3545f2997ff5-cluster-monitoring-operator-tls podName:68195e19-b1ff-4148-9fae-3545f2997ff5 nodeName:}" failed. No retries permitted until 2026-10-30 18:33:58.763318924 +0000 UTC m=+3.010724926 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/68195e19-b1ff-4148-9fae-3545f2997ff5-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-gmswj" (UID: "68195e19-b1ff-4148-9fae-3545f2997ff5") : secret "cluster-monitoring-operator-tls" not found
Oct 30 18:33:57.763611 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:33:57.763378    4888 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351-registry-tls podName:9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351 nodeName:}" failed. No retries permitted until 2026-10-30 18:33:58.763370239 +0000 UTC m=+3.010776242 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351-registry-tls") pod "image-registry-64bf5b6b76-dpvvs" (UID: "9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351") : secret "image-registry-tls" not found
Oct 30 18:33:57.763611 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:33:57.763388    4888 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/57ca8600-79e7-4f59-9d11-b1c9ec3d358a-insights-runtime-extractor-tls podName:57ca8600-79e7-4f59-9d11-b1c9ec3d358a nodeName:}" failed. No retries permitted until 2026-10-30 18:33:58.763383049 +0000 UTC m=+3.010789051 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/57ca8600-79e7-4f59-9d11-b1c9ec3d358a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jbsw6" (UID: "57ca8600-79e7-4f59-9d11-b1c9ec3d358a") : secret "insights-runtime-extractor-tls" not found
Oct 30 18:33:57.941977 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.941950    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-7x4rb_382e8b31-a7fa-43f3-9f15-4c61627b0c04/console-operator/2.log"
Oct 30 18:33:57.942444 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.942412    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-7x4rb_382e8b31-a7fa-43f3-9f15-4c61627b0c04/console-operator/1.log"
Oct 30 18:33:57.942485 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.942443    4888 generic.go:356] "Generic (PLEG): container finished" podID="382e8b31-a7fa-43f3-9f15-4c61627b0c04" containerID="6e4c592dcf60d4f6dc81cdb78927a382458a17ccfb80bf9fb1850fda6902eb5c" exitCode=255
Oct 30 18:33:57.942546 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.942524    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-7x4rb" event={"ID":"382e8b31-a7fa-43f3-9f15-4c61627b0c04","Type":"ContainerDied","Data":"6e4c592dcf60d4f6dc81cdb78927a382458a17ccfb80bf9fb1850fda6902eb5c"}
Oct 30 18:33:57.942606 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.942591    4888 scope.go:122] "RemoveContainer" containerID="b7e8a752d2699b7e5514e278ff1b7c9fd161b532812a42ac3605f0cdaad0c160"
Oct 30 18:33:57.942990 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:57.942969    4888 scope.go:122] "RemoveContainer" containerID="6e4c592dcf60d4f6dc81cdb78927a382458a17ccfb80bf9fb1850fda6902eb5c"
Oct 30 18:33:57.943137 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:33:57.943116    4888 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-7x4rb_openshift-console-operator(382e8b31-a7fa-43f3-9f15-4c61627b0c04)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-7x4rb" podUID="382e8b31-a7fa-43f3-9f15-4c61627b0c04"
Oct 30 18:33:58.773577 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:58.773539    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/da28351c-1c8b-40be-85a0-3aad427e3a17-service-ca-bundle\") pod \"router-default-7dfcf9fd6c-vh769\" (UID: \"da28351c-1c8b-40be-85a0-3aad427e3a17\") " pod="openshift-ingress/router-default-7dfcf9fd6c-vh769"
Oct 30 18:33:58.773809 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:58.773610    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/da28351c-1c8b-40be-85a0-3aad427e3a17-metrics-certs\") pod \"router-default-7dfcf9fd6c-vh769\" (UID: \"da28351c-1c8b-40be-85a0-3aad427e3a17\") " pod="openshift-ingress/router-default-7dfcf9fd6c-vh769"
Oct 30 18:33:58.773809 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:58.773647    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/97b19aa3-42be-4def-8d7a-8e3516852e79-metrics-tls\") pod \"dns-default-r2nx2\" (UID: \"97b19aa3-42be-4def-8d7a-8e3516852e79\") " pod="openshift-dns/dns-default-r2nx2"
Oct 30 18:33:58.773809 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:58.773677    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8789ec5b-226e-4551-bf68-3ac585006266-registry-tls\") pod \"image-registry-74757bccd5-qmrkp\" (UID: \"8789ec5b-226e-4551-bf68-3ac585006266\") " pod="openshift-image-registry/image-registry-74757bccd5-qmrkp"
Oct 30 18:33:58.773809 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:33:58.773696    4888 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/da28351c-1c8b-40be-85a0-3aad427e3a17-service-ca-bundle podName:da28351c-1c8b-40be-85a0-3aad427e3a17 nodeName:}" failed. No retries permitted until 2026-10-30 18:34:00.773670878 +0000 UTC m=+5.021076881 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/da28351c-1c8b-40be-85a0-3aad427e3a17-service-ca-bundle") pod "router-default-7dfcf9fd6c-vh769" (UID: "da28351c-1c8b-40be-85a0-3aad427e3a17") : configmap references non-existent config key: service-ca.crt
Oct 30 18:33:58.773809 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:33:58.773778    4888 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Oct 30 18:33:58.773809 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:33:58.773782    4888 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 30 18:33:58.773809 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:58.773803    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/15d485f9-5a4b-4f4b-9e59-a6eb062b4c5c-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-pm58m\" (UID: \"15d485f9-5a4b-4f4b-9e59-a6eb062b4c5c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-pm58m"
Oct 30 18:33:58.774129 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:33:58.773784    4888 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 30 18:33:58.774129 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:33:58.773848    4888 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-74757bccd5-qmrkp: secret "image-registry-tls" not found
Oct 30 18:33:58.774129 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:33:58.773835    4888 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/da28351c-1c8b-40be-85a0-3aad427e3a17-metrics-certs podName:da28351c-1c8b-40be-85a0-3aad427e3a17 nodeName:}" failed. No retries permitted until 2026-10-30 18:34:00.773819464 +0000 UTC m=+5.021225471 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/da28351c-1c8b-40be-85a0-3aad427e3a17-metrics-certs") pod "router-default-7dfcf9fd6c-vh769" (UID: "da28351c-1c8b-40be-85a0-3aad427e3a17") : secret "router-metrics-certs-default" not found
Oct 30 18:33:58.774129 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:33:58.773874    4888 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Oct 30 18:33:58.774129 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:33:58.773902    4888 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/97b19aa3-42be-4def-8d7a-8e3516852e79-metrics-tls podName:97b19aa3-42be-4def-8d7a-8e3516852e79 nodeName:}" failed. No retries permitted until 2026-10-30 18:34:00.773891672 +0000 UTC m=+5.021297675 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/97b19aa3-42be-4def-8d7a-8e3516852e79-metrics-tls") pod "dns-default-r2nx2" (UID: "97b19aa3-42be-4def-8d7a-8e3516852e79") : secret "dns-default-metrics-tls" not found
Oct 30 18:33:58.774129 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:33:58.773917    4888 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8789ec5b-226e-4551-bf68-3ac585006266-registry-tls podName:8789ec5b-226e-4551-bf68-3ac585006266 nodeName:}" failed. No retries permitted until 2026-10-30 18:34:00.773908418 +0000 UTC m=+5.021314420 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8789ec5b-226e-4551-bf68-3ac585006266-registry-tls") pod "image-registry-74757bccd5-qmrkp" (UID: "8789ec5b-226e-4551-bf68-3ac585006266") : secret "image-registry-tls" not found
Oct 30 18:33:58.774129 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:58.773937    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/71fce796-0032-481b-80b3-66fbb731452c-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-r8rgt\" (UID: \"71fce796-0032-481b-80b3-66fbb731452c\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-r8rgt"
Oct 30 18:33:58.774129 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:58.773972    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/57ca8600-79e7-4f59-9d11-b1c9ec3d358a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jbsw6\" (UID: \"57ca8600-79e7-4f59-9d11-b1c9ec3d358a\") " pod="openshift-insights/insights-runtime-extractor-jbsw6"
Oct 30 18:33:58.774129 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:33:58.773992    4888 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Oct 30 18:33:58.774129 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:58.774005    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/68195e19-b1ff-4148-9fae-3545f2997ff5-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-gmswj\" (UID: \"68195e19-b1ff-4148-9fae-3545f2997ff5\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-gmswj"
Oct 30 18:33:58.774129 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:33:58.774031    4888 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/71fce796-0032-481b-80b3-66fbb731452c-networking-console-plugin-cert podName:71fce796-0032-481b-80b3-66fbb731452c nodeName:}" failed. No retries permitted until 2026-10-30 18:34:00.774019458 +0000 UTC m=+5.021425478 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/71fce796-0032-481b-80b3-66fbb731452c-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-r8rgt" (UID: "71fce796-0032-481b-80b3-66fbb731452c") : secret "networking-console-plugin-cert" not found
Oct 30 18:33:58.774129 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:33:58.774066    4888 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Oct 30 18:33:58.774129 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:58.774070    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351-registry-tls\") pod \"image-registry-64bf5b6b76-dpvvs\" (UID: \"9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351\") " pod="openshift-image-registry/image-registry-64bf5b6b76-dpvvs"
Oct 30 18:33:58.774129 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:33:58.774098    4888 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68195e19-b1ff-4148-9fae-3545f2997ff5-cluster-monitoring-operator-tls podName:68195e19-b1ff-4148-9fae-3545f2997ff5 nodeName:}" failed. No retries permitted until 2026-10-30 18:34:00.774088159 +0000 UTC m=+5.021494162 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/68195e19-b1ff-4148-9fae-3545f2997ff5-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-gmswj" (UID: "68195e19-b1ff-4148-9fae-3545f2997ff5") : secret "cluster-monitoring-operator-tls" not found
Oct 30 18:33:58.774129 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:33:58.774137    4888 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 30 18:33:58.774707 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:33:58.774148    4888 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-64bf5b6b76-dpvvs: secret "image-registry-tls" not found
Oct 30 18:33:58.774707 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:58.774153    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a0823e8f-bbdd-477b-95ac-dad05d23cefa-metrics-certs\") pod \"network-metrics-daemon-v5phn\" (UID: \"a0823e8f-bbdd-477b-95ac-dad05d23cefa\") " pod="openshift-multus/network-metrics-daemon-v5phn"
Oct 30 18:33:58.774707 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:33:58.774177    4888 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351-registry-tls podName:9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351 nodeName:}" failed. No retries permitted until 2026-10-30 18:34:00.774166717 +0000 UTC m=+5.021572735 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351-registry-tls") pod "image-registry-64bf5b6b76-dpvvs" (UID: "9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351") : secret "image-registry-tls" not found
Oct 30 18:33:58.774707 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:33:58.774208    4888 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Oct 30 18:33:58.774707 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:33:58.774230    4888 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/15d485f9-5a4b-4f4b-9e59-a6eb062b4c5c-samples-operator-tls podName:15d485f9-5a4b-4f4b-9e59-a6eb062b4c5c nodeName:}" failed. No retries permitted until 2026-10-30 18:34:00.774220959 +0000 UTC m=+5.021627004 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/15d485f9-5a4b-4f4b-9e59-a6eb062b4c5c-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-pm58m" (UID: "15d485f9-5a4b-4f4b-9e59-a6eb062b4c5c") : secret "samples-operator-tls" not found
Oct 30 18:33:58.774707 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:33:58.774255    4888 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a0823e8f-bbdd-477b-95ac-dad05d23cefa-metrics-certs podName:a0823e8f-bbdd-477b-95ac-dad05d23cefa nodeName:}" failed. No retries permitted until 2026-10-30 18:34:00.774237536 +0000 UTC m=+5.021643539 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a0823e8f-bbdd-477b-95ac-dad05d23cefa-metrics-certs") pod "network-metrics-daemon-v5phn" (UID: "a0823e8f-bbdd-477b-95ac-dad05d23cefa") : secret "metrics-daemon-secret" not found
Oct 30 18:33:58.774707 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:33:58.774276    4888 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 30 18:33:58.774707 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:33:58.774329    4888 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/57ca8600-79e7-4f59-9d11-b1c9ec3d358a-insights-runtime-extractor-tls podName:57ca8600-79e7-4f59-9d11-b1c9ec3d358a nodeName:}" failed. No retries permitted until 2026-10-30 18:34:00.77432161 +0000 UTC m=+5.021727613 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/57ca8600-79e7-4f59-9d11-b1c9ec3d358a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jbsw6" (UID: "57ca8600-79e7-4f59-9d11-b1c9ec3d358a") : secret "insights-runtime-extractor-tls" not found
Oct 30 18:33:58.949871 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:58.949845    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-7x4rb_382e8b31-a7fa-43f3-9f15-4c61627b0c04/console-operator/2.log"
Oct 30 18:33:59.343180 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:59.343153    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-76f9s_d81cef55-16ee-4677-ae12-86de3c9eb216/dns-node-resolver/0.log"
Oct 30 18:33:59.548052 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:59.547962    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7cd56b8dbc-7x4rb"
Oct 30 18:33:59.548358 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:59.548344    4888 scope.go:122] "RemoveContainer" containerID="6e4c592dcf60d4f6dc81cdb78927a382458a17ccfb80bf9fb1850fda6902eb5c"
Oct 30 18:33:59.548563 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:33:59.548541    4888 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-7x4rb_openshift-console-operator(382e8b31-a7fa-43f3-9f15-4c61627b0c04)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-7x4rb" podUID="382e8b31-a7fa-43f3-9f15-4c61627b0c04"
Oct 30 18:33:59.955140 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:33:59.955111    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-ht9zn" event={"ID":"a3cf021c-2c20-4687-9b3a-cfa65ce1ee8b","Type":"ContainerStarted","Data":"2097273f91820e2b329c1bb4d001433d4c0159c8123ee6ff4dfa3f436ca1e918"}
Oct 30 18:34:00.758114 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:00.758085    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-g78lj_46a80637-6ccf-4a56-9904-f5e2fe5784ad/node-ca/0.log"
Oct 30 18:34:00.793565 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:00.793537    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/15d485f9-5a4b-4f4b-9e59-a6eb062b4c5c-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-pm58m\" (UID: \"15d485f9-5a4b-4f4b-9e59-a6eb062b4c5c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-pm58m"
Oct 30 18:34:00.793702 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:00.793574    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/71fce796-0032-481b-80b3-66fbb731452c-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-r8rgt\" (UID: \"71fce796-0032-481b-80b3-66fbb731452c\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-r8rgt"
Oct 30 18:34:00.793702 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:00.793592    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/57ca8600-79e7-4f59-9d11-b1c9ec3d358a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jbsw6\" (UID: \"57ca8600-79e7-4f59-9d11-b1c9ec3d358a\") " pod="openshift-insights/insights-runtime-extractor-jbsw6"
Oct 30 18:34:00.793702 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:00.793611    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/68195e19-b1ff-4148-9fae-3545f2997ff5-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-gmswj\" (UID: \"68195e19-b1ff-4148-9fae-3545f2997ff5\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-gmswj"
Oct 30 18:34:00.793702 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:00.793650    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351-registry-tls\") pod \"image-registry-64bf5b6b76-dpvvs\" (UID: \"9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351\") " pod="openshift-image-registry/image-registry-64bf5b6b76-dpvvs"
Oct 30 18:34:00.793702 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:34:00.793685    4888 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Oct 30 18:34:00.793702 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:34:00.793694    4888 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Oct 30 18:34:00.793933 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:00.793710    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a0823e8f-bbdd-477b-95ac-dad05d23cefa-metrics-certs\") pod \"network-metrics-daemon-v5phn\" (UID: \"a0823e8f-bbdd-477b-95ac-dad05d23cefa\") " pod="openshift-multus/network-metrics-daemon-v5phn"
Oct 30 18:34:00.793933 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:34:00.793696    4888 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 30 18:34:00.793933 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:34:00.793745    4888 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/15d485f9-5a4b-4f4b-9e59-a6eb062b4c5c-samples-operator-tls podName:15d485f9-5a4b-4f4b-9e59-a6eb062b4c5c nodeName:}" failed. No retries permitted until 2026-10-30 18:34:04.793730617 +0000 UTC m=+9.041136624 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/15d485f9-5a4b-4f4b-9e59-a6eb062b4c5c-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-pm58m" (UID: "15d485f9-5a4b-4f4b-9e59-a6eb062b4c5c") : secret "samples-operator-tls" not found
Oct 30 18:34:00.793933 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:00.793765    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/da28351c-1c8b-40be-85a0-3aad427e3a17-service-ca-bundle\") pod \"router-default-7dfcf9fd6c-vh769\" (UID: \"da28351c-1c8b-40be-85a0-3aad427e3a17\") " pod="openshift-ingress/router-default-7dfcf9fd6c-vh769"
Oct 30 18:34:00.793933 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:34:00.793773    4888 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/57ca8600-79e7-4f59-9d11-b1c9ec3d358a-insights-runtime-extractor-tls podName:57ca8600-79e7-4f59-9d11-b1c9ec3d358a nodeName:}" failed. No retries permitted until 2026-10-30 18:34:04.793757314 +0000 UTC m=+9.041163339 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/57ca8600-79e7-4f59-9d11-b1c9ec3d358a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jbsw6" (UID: "57ca8600-79e7-4f59-9d11-b1c9ec3d358a") : secret "insights-runtime-extractor-tls" not found
Oct 30 18:34:00.793933 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:34:00.793795    4888 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/71fce796-0032-481b-80b3-66fbb731452c-networking-console-plugin-cert podName:71fce796-0032-481b-80b3-66fbb731452c nodeName:}" failed. No retries permitted until 2026-10-30 18:34:04.793785192 +0000 UTC m=+9.041191196 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/71fce796-0032-481b-80b3-66fbb731452c-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-r8rgt" (UID: "71fce796-0032-481b-80b3-66fbb731452c") : secret "networking-console-plugin-cert" not found
Oct 30 18:34:00.793933 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:00.793843    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/da28351c-1c8b-40be-85a0-3aad427e3a17-metrics-certs\") pod \"router-default-7dfcf9fd6c-vh769\" (UID: \"da28351c-1c8b-40be-85a0-3aad427e3a17\") " pod="openshift-ingress/router-default-7dfcf9fd6c-vh769"
Oct 30 18:34:00.793933 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:34:00.793852    4888 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/da28351c-1c8b-40be-85a0-3aad427e3a17-service-ca-bundle podName:da28351c-1c8b-40be-85a0-3aad427e3a17 nodeName:}" failed. No retries permitted until 2026-10-30 18:34:04.793841771 +0000 UTC m=+9.041247777 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/da28351c-1c8b-40be-85a0-3aad427e3a17-service-ca-bundle") pod "router-default-7dfcf9fd6c-vh769" (UID: "da28351c-1c8b-40be-85a0-3aad427e3a17") : configmap references non-existent config key: service-ca.crt
Oct 30 18:34:00.793933 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:00.793889    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/97b19aa3-42be-4def-8d7a-8e3516852e79-metrics-tls\") pod \"dns-default-r2nx2\" (UID: \"97b19aa3-42be-4def-8d7a-8e3516852e79\") " pod="openshift-dns/dns-default-r2nx2"
Oct 30 18:34:00.793933 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:34:00.793895    4888 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Oct 30 18:34:00.793933 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:00.793919    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8789ec5b-226e-4551-bf68-3ac585006266-registry-tls\") pod \"image-registry-74757bccd5-qmrkp\" (UID: \"8789ec5b-226e-4551-bf68-3ac585006266\") " pod="openshift-image-registry/image-registry-74757bccd5-qmrkp"
Oct 30 18:34:00.793933 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:34:00.793927    4888 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/da28351c-1c8b-40be-85a0-3aad427e3a17-metrics-certs podName:da28351c-1c8b-40be-85a0-3aad427e3a17 nodeName:}" failed. No retries permitted until 2026-10-30 18:34:04.793917118 +0000 UTC m=+9.041323121 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/da28351c-1c8b-40be-85a0-3aad427e3a17-metrics-certs") pod "router-default-7dfcf9fd6c-vh769" (UID: "da28351c-1c8b-40be-85a0-3aad427e3a17") : secret "router-metrics-certs-default" not found
Oct 30 18:34:00.794553 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:00.794011    4888 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Oct 30 18:34:00.794553 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:34:00.794016    4888 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 30 18:34:00.794553 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:34:00.794086    4888 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/97b19aa3-42be-4def-8d7a-8e3516852e79-metrics-tls podName:97b19aa3-42be-4def-8d7a-8e3516852e79 nodeName:}" failed. No retries permitted until 2026-10-30 18:34:04.794067959 +0000 UTC m=+9.041473991 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/97b19aa3-42be-4def-8d7a-8e3516852e79-metrics-tls") pod "dns-default-r2nx2" (UID: "97b19aa3-42be-4def-8d7a-8e3516852e79") : secret "dns-default-metrics-tls" not found
Oct 30 18:34:00.796431 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:00.796410    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8789ec5b-226e-4551-bf68-3ac585006266-registry-tls\") pod \"image-registry-74757bccd5-qmrkp\" (UID: \"8789ec5b-226e-4551-bf68-3ac585006266\") " pod="openshift-image-registry/image-registry-74757bccd5-qmrkp"
Oct 30 18:34:00.796532 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:00.796429    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a0823e8f-bbdd-477b-95ac-dad05d23cefa-metrics-certs\") pod \"network-metrics-daemon-v5phn\" (UID: \"a0823e8f-bbdd-477b-95ac-dad05d23cefa\") " pod="openshift-multus/network-metrics-daemon-v5phn"
Oct 30 18:34:00.796532 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:00.796508    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/68195e19-b1ff-4148-9fae-3545f2997ff5-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-gmswj\" (UID: \"68195e19-b1ff-4148-9fae-3545f2997ff5\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-gmswj"
Oct 30 18:34:00.796603 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:00.796564    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351-registry-tls\") pod \"image-registry-64bf5b6b76-dpvvs\" (UID: \"9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351\") " pod="openshift-image-registry/image-registry-64bf5b6b76-dpvvs"
Oct 30 18:34:00.891407 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:00.891372    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-gmswj"
Oct 30 18:34:00.892481 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:00.892460    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-74757bccd5-qmrkp"
Oct 30 18:34:00.898870 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:00.898852    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v5phn"
Oct 30 18:34:00.901791 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:00.901766    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-64bf5b6b76-dpvvs"
Oct 30 18:34:01.145128 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:01.145092    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-g9fhp_e6b2a584-ae78-4e80-b595-9b1a5897d556/serve-healthcheck-canary/0.log"
Oct 30 18:34:01.340275 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:01.340190    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-884576c57-gmswj"]
Oct 30 18:34:01.342688 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:34:01.342660    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68195e19_b1ff_4148_9fae_3545f2997ff5.slice/crio-6730ed4564c791cf325776b44455bbcb432a812f54e8413ca0ae23e5224c9d08 WatchSource:0}: Error finding container 6730ed4564c791cf325776b44455bbcb432a812f54e8413ca0ae23e5224c9d08: Status 404 returned error can't find the container with id 6730ed4564c791cf325776b44455bbcb432a812f54e8413ca0ae23e5224c9d08
Oct 30 18:34:01.346156 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:01.346140    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-74757bccd5-qmrkp"]
Oct 30 18:34:01.387892 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:34:01.387867    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8789ec5b_226e_4551_bf68_3ac585006266.slice/crio-af9cd9640d45b150520e18d32103869659c5392878f7e76255779a09719a65eb WatchSource:0}: Error finding container af9cd9640d45b150520e18d32103869659c5392878f7e76255779a09719a65eb: Status 404 returned error can't find the container with id af9cd9640d45b150520e18d32103869659c5392878f7e76255779a09719a65eb
Oct 30 18:34:01.403390 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:01.403370    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-64bf5b6b76-dpvvs"]
Oct 30 18:34:01.421242 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:34:01.421216    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9bdb5a6c_6c32_4e70_b4cf_6e4d610a8351.slice/crio-f0669606e17e7131c510c046039ad6044074fb9fe711bb78061b3e4f82816e2d WatchSource:0}: Error finding container f0669606e17e7131c510c046039ad6044074fb9fe711bb78061b3e4f82816e2d: Status 404 returned error can't find the container with id f0669606e17e7131c510c046039ad6044074fb9fe711bb78061b3e4f82816e2d
Oct 30 18:34:01.456472 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:01.456454    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-v5phn"]
Oct 30 18:34:01.459799 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:34:01.459774    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0823e8f_bbdd_477b_95ac_dad05d23cefa.slice/crio-53bbe67d8c3f83dcb2a819269f7cecad96ecbbcfd605038f3dbcbe86ef2b4074 WatchSource:0}: Error finding container 53bbe67d8c3f83dcb2a819269f7cecad96ecbbcfd605038f3dbcbe86ef2b4074: Status 404 returned error can't find the container with id 53bbe67d8c3f83dcb2a819269f7cecad96ecbbcfd605038f3dbcbe86ef2b4074
Oct 30 18:34:01.544317 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:01.544276    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-66df5d496b-ht9zn_a3cf021c-2c20-4687-9b3a-cfa65ce1ee8b/migrator/0.log"
Oct 30 18:34:01.944542 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:01.944507    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7d4bc45dd6-5bqdc_09c13759-50d4-40e6-b854-4691822b8c8a/kube-storage-version-migrator-operator/0.log"
Oct 30 18:34:01.962577 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:01.962546    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-v5phn" event={"ID":"a0823e8f-bbdd-477b-95ac-dad05d23cefa","Type":"ContainerStarted","Data":"53bbe67d8c3f83dcb2a819269f7cecad96ecbbcfd605038f3dbcbe86ef2b4074"}
Oct 30 18:34:01.964013 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:01.963980    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-74757bccd5-qmrkp" event={"ID":"8789ec5b-226e-4551-bf68-3ac585006266","Type":"ContainerStarted","Data":"e0cf016a42be2cc96c517195f83aded8b7b76dbfa849ddcbdd7dfcb378295c5c"}
Oct 30 18:34:01.964121 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:01.964023    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-74757bccd5-qmrkp" event={"ID":"8789ec5b-226e-4551-bf68-3ac585006266","Type":"ContainerStarted","Data":"af9cd9640d45b150520e18d32103869659c5392878f7e76255779a09719a65eb"}
Oct 30 18:34:01.965141 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:01.965115    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-gmswj" event={"ID":"68195e19-b1ff-4148-9fae-3545f2997ff5","Type":"ContainerStarted","Data":"6730ed4564c791cf325776b44455bbcb432a812f54e8413ca0ae23e5224c9d08"}
Oct 30 18:34:01.966615 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:01.966596    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-64bf5b6b76-dpvvs" event={"ID":"9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351","Type":"ContainerStarted","Data":"9cc96a881f01d5d37e977a094ed4d424114e78a12531bee8e8dd9d5ea62ef811"}
Oct 30 18:34:01.966680 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:01.966629    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-64bf5b6b76-dpvvs" event={"ID":"9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351","Type":"ContainerStarted","Data":"f0669606e17e7131c510c046039ad6044074fb9fe711bb78061b3e4f82816e2d"}
Oct 30 18:34:01.999921 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:01.999900    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-xfcdl"
Oct 30 18:34:02.046359 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:02.046334    4888 kubelet.go:2727] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7cd56b8dbc-7x4rb"
Oct 30 18:34:02.046674 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:02.046661    4888 scope.go:122] "RemoveContainer" containerID="6e4c592dcf60d4f6dc81cdb78927a382458a17ccfb80bf9fb1850fda6902eb5c"
Oct 30 18:34:02.046828 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:34:02.046812    4888 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-7x4rb_openshift-console-operator(382e8b31-a7fa-43f3-9f15-4c61627b0c04)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-7x4rb" podUID="382e8b31-a7fa-43f3-9f15-4c61627b0c04"
Oct 30 18:34:02.969885 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:02.969862    4888 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 30 18:34:02.970239 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:02.969863    4888 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 30 18:34:03.502088 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:03.502058    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-xfcdl"
Oct 30 18:34:03.502339 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:03.502323    4888 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 30 18:34:03.577188 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:03.577156    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-74757bccd5-qmrkp"
Oct 30 18:34:03.975121 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:03.975076    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-v5phn" event={"ID":"a0823e8f-bbdd-477b-95ac-dad05d23cefa","Type":"ContainerStarted","Data":"3fb3608b5006965d27b06f9366d5315519d914216b454229347d2f16fcce2a92"}
Oct 30 18:34:03.975601 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:03.975138    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-v5phn" event={"ID":"a0823e8f-bbdd-477b-95ac-dad05d23cefa","Type":"ContainerStarted","Data":"ea169598c19bd71c57e10e23dfeec25abc975a9732a75317bb22335f88cea98c"}
Oct 30 18:34:03.976786 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:03.976765    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-gmswj" event={"ID":"68195e19-b1ff-4148-9fae-3545f2997ff5","Type":"ContainerStarted","Data":"1efbbb64090cea9bfed1d3075aead125cca2f9efa48277fd1ec6c93b361bebfe"}
Oct 30 18:34:04.833902 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:04.833870    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/97b19aa3-42be-4def-8d7a-8e3516852e79-metrics-tls\") pod \"dns-default-r2nx2\" (UID: \"97b19aa3-42be-4def-8d7a-8e3516852e79\") " pod="openshift-dns/dns-default-r2nx2"
Oct 30 18:34:04.833902 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:04.833912    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/15d485f9-5a4b-4f4b-9e59-a6eb062b4c5c-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-pm58m\" (UID: \"15d485f9-5a4b-4f4b-9e59-a6eb062b4c5c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-pm58m"
Oct 30 18:34:04.834171 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:04.833935    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/71fce796-0032-481b-80b3-66fbb731452c-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-r8rgt\" (UID: \"71fce796-0032-481b-80b3-66fbb731452c\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-r8rgt"
Oct 30 18:34:04.834171 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:04.833954    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/57ca8600-79e7-4f59-9d11-b1c9ec3d358a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jbsw6\" (UID: \"57ca8600-79e7-4f59-9d11-b1c9ec3d358a\") " pod="openshift-insights/insights-runtime-extractor-jbsw6"
Oct 30 18:34:04.834171 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:04.834002    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/da28351c-1c8b-40be-85a0-3aad427e3a17-service-ca-bundle\") pod \"router-default-7dfcf9fd6c-vh769\" (UID: \"da28351c-1c8b-40be-85a0-3aad427e3a17\") " pod="openshift-ingress/router-default-7dfcf9fd6c-vh769"
Oct 30 18:34:04.834171 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:04.834028    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/da28351c-1c8b-40be-85a0-3aad427e3a17-metrics-certs\") pod \"router-default-7dfcf9fd6c-vh769\" (UID: \"da28351c-1c8b-40be-85a0-3aad427e3a17\") " pod="openshift-ingress/router-default-7dfcf9fd6c-vh769"
Oct 30 18:34:04.834171 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:34:04.834044    4888 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Oct 30 18:34:04.834171 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:34:04.834146    4888 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/da28351c-1c8b-40be-85a0-3aad427e3a17-service-ca-bundle podName:da28351c-1c8b-40be-85a0-3aad427e3a17 nodeName:}" failed. No retries permitted until 2026-10-30 18:34:12.834125864 +0000 UTC m=+17.081531868 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/da28351c-1c8b-40be-85a0-3aad427e3a17-service-ca-bundle") pod "router-default-7dfcf9fd6c-vh769" (UID: "da28351c-1c8b-40be-85a0-3aad427e3a17") : configmap references non-existent config key: service-ca.crt
Oct 30 18:34:04.834171 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:34:04.834146    4888 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 30 18:34:04.834521 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:34:04.834228    4888 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/15d485f9-5a4b-4f4b-9e59-a6eb062b4c5c-samples-operator-tls podName:15d485f9-5a4b-4f4b-9e59-a6eb062b4c5c nodeName:}" failed. No retries permitted until 2026-10-30 18:34:12.834209036 +0000 UTC m=+17.081615056 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/15d485f9-5a4b-4f4b-9e59-a6eb062b4c5c-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-pm58m" (UID: "15d485f9-5a4b-4f4b-9e59-a6eb062b4c5c") : secret "samples-operator-tls" not found
Oct 30 18:34:04.834521 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:34:04.834248    4888 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/57ca8600-79e7-4f59-9d11-b1c9ec3d358a-insights-runtime-extractor-tls podName:57ca8600-79e7-4f59-9d11-b1c9ec3d358a nodeName:}" failed. No retries permitted until 2026-10-30 18:34:12.834239229 +0000 UTC m=+17.081645236 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/57ca8600-79e7-4f59-9d11-b1c9ec3d358a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jbsw6" (UID: "57ca8600-79e7-4f59-9d11-b1c9ec3d358a") : secret "insights-runtime-extractor-tls" not found
Oct 30 18:34:04.836656 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:04.836636    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/da28351c-1c8b-40be-85a0-3aad427e3a17-metrics-certs\") pod \"router-default-7dfcf9fd6c-vh769\" (UID: \"da28351c-1c8b-40be-85a0-3aad427e3a17\") " pod="openshift-ingress/router-default-7dfcf9fd6c-vh769"
Oct 30 18:34:04.836717 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:04.836652    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/97b19aa3-42be-4def-8d7a-8e3516852e79-metrics-tls\") pod \"dns-default-r2nx2\" (UID: \"97b19aa3-42be-4def-8d7a-8e3516852e79\") " pod="openshift-dns/dns-default-r2nx2"
Oct 30 18:34:04.836717 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:04.836652    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/71fce796-0032-481b-80b3-66fbb731452c-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-r8rgt\" (UID: \"71fce796-0032-481b-80b3-66fbb731452c\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-r8rgt"
Oct 30 18:34:05.096034 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:05.095950    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-86c478bb88-r8rgt"
Oct 30 18:34:05.103086 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:05.103057    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-r2nx2"
Oct 30 18:34:05.183722 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:05.183696    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-8xskh"
Oct 30 18:34:05.470984 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:05.470914    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-86c478bb88-r8rgt"]
Oct 30 18:34:05.473791 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:34:05.473768    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71fce796_0032_481b_80b3_66fbb731452c.slice/crio-a6f9c097fff62e3233b2d77936ce7e3290db6cf5c8ac0174d467edbfbca4211a WatchSource:0}: Error finding container a6f9c097fff62e3233b2d77936ce7e3290db6cf5c8ac0174d467edbfbca4211a: Status 404 returned error can't find the container with id a6f9c097fff62e3233b2d77936ce7e3290db6cf5c8ac0174d467edbfbca4211a
Oct 30 18:34:05.536853 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:05.536825    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-r2nx2"]
Oct 30 18:34:05.539590 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:34:05.539565    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97b19aa3_42be_4def_8d7a_8e3516852e79.slice/crio-ae775cfceb91de45b38901f6a14ed665459c318228793362e1f9077ffdf81c37 WatchSource:0}: Error finding container ae775cfceb91de45b38901f6a14ed665459c318228793362e1f9077ffdf81c37: Status 404 returned error can't find the container with id ae775cfceb91de45b38901f6a14ed665459c318228793362e1f9077ffdf81c37
Oct 30 18:34:05.983888 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:05.983852    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-r2nx2" event={"ID":"97b19aa3-42be-4def-8d7a-8e3516852e79","Type":"ContainerStarted","Data":"ae775cfceb91de45b38901f6a14ed665459c318228793362e1f9077ffdf81c37"}
Oct 30 18:34:05.987984 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:05.985621    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-86c478bb88-r8rgt" event={"ID":"71fce796-0032-481b-80b3-66fbb731452c","Type":"ContainerStarted","Data":"a6f9c097fff62e3233b2d77936ce7e3290db6cf5c8ac0174d467edbfbca4211a"}
Oct 30 18:34:07.994598 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:07.994559    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-86c478bb88-r8rgt" event={"ID":"71fce796-0032-481b-80b3-66fbb731452c","Type":"ContainerStarted","Data":"7bb49c1af7e2328eb6614d9f20768f2b9dbf2a9804e7c93141a7b14eb74ef9d0"}
Oct 30 18:34:07.996467 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:07.996442    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-r2nx2" event={"ID":"97b19aa3-42be-4def-8d7a-8e3516852e79","Type":"ContainerStarted","Data":"261f105e79859db3b03219267042ccb19e2de86546bf948897a7aaad501d6213"}
Oct 30 18:34:07.996574 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:07.996486    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-r2nx2" event={"ID":"97b19aa3-42be-4def-8d7a-8e3516852e79","Type":"ContainerStarted","Data":"6ade34690bcab53dae200ffb311026bb94d0d21d22de22a675f0b8666ef7c5ad"}
Oct 30 18:34:07.996612 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:07.996589    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-r2nx2"
Oct 30 18:34:08.568414 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:08.568378    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-64bf5b6b76-dpvvs"
Oct 30 18:34:10.320099 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:10.320070    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-lmdpp"
Oct 30 18:34:10.883763 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:10.883727    4888 scope.go:122] "RemoveContainer" containerID="6e4c592dcf60d4f6dc81cdb78927a382458a17ccfb80bf9fb1850fda6902eb5c"
Oct 30 18:34:11.009259 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:11.009234    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-7x4rb_382e8b31-a7fa-43f3-9f15-4c61627b0c04/console-operator/2.log"
Oct 30 18:34:11.009424 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:11.009354    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-7x4rb" event={"ID":"382e8b31-a7fa-43f3-9f15-4c61627b0c04","Type":"ContainerStarted","Data":"6c3e58d6135d89e8ac06f83accc41ca94971c64420b8c3cf1a8d6af59a4da96f"}
Oct 30 18:34:11.009643 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:11.009622    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7cd56b8dbc-7x4rb"
Oct 30 18:34:11.010566 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:11.010539    4888 patch_prober.go:29] interesting pod/console-operator-7cd56b8dbc-7x4rb container/console-operator namespace/openshift-console-operator: Readiness probe status=failure output="Get \"https://10.132.0.11:8443/readyz\": dial tcp 10.132.0.11:8443: connect: connection refused" start-of-body=
Oct 30 18:34:11.010677 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:11.010586    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-console-operator/console-operator-7cd56b8dbc-7x4rb" podUID="382e8b31-a7fa-43f3-9f15-4c61627b0c04" containerName="console-operator" probeResult="failure" output="Get \"https://10.132.0.11:8443/readyz\": dial tcp 10.132.0.11:8443: connect: connection refused"
Oct 30 18:34:12.017127 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:12.017104    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-7cd56b8dbc-7x4rb"
Oct 30 18:34:12.901026 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:12.900993    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/da28351c-1c8b-40be-85a0-3aad427e3a17-service-ca-bundle\") pod \"router-default-7dfcf9fd6c-vh769\" (UID: \"da28351c-1c8b-40be-85a0-3aad427e3a17\") " pod="openshift-ingress/router-default-7dfcf9fd6c-vh769"
Oct 30 18:34:12.901168 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:12.901039    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/15d485f9-5a4b-4f4b-9e59-a6eb062b4c5c-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-pm58m\" (UID: \"15d485f9-5a4b-4f4b-9e59-a6eb062b4c5c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-pm58m"
Oct 30 18:34:12.901168 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:12.901074    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/57ca8600-79e7-4f59-9d11-b1c9ec3d358a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jbsw6\" (UID: \"57ca8600-79e7-4f59-9d11-b1c9ec3d358a\") " pod="openshift-insights/insights-runtime-extractor-jbsw6"
Oct 30 18:34:12.901721 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:12.901686    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/da28351c-1c8b-40be-85a0-3aad427e3a17-service-ca-bundle\") pod \"router-default-7dfcf9fd6c-vh769\" (UID: \"da28351c-1c8b-40be-85a0-3aad427e3a17\") " pod="openshift-ingress/router-default-7dfcf9fd6c-vh769"
Oct 30 18:34:12.903647 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:12.903625    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/15d485f9-5a4b-4f4b-9e59-a6eb062b4c5c-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-pm58m\" (UID: \"15d485f9-5a4b-4f4b-9e59-a6eb062b4c5c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-pm58m"
Oct 30 18:34:12.903768 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:12.903652    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/57ca8600-79e7-4f59-9d11-b1c9ec3d358a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jbsw6\" (UID: \"57ca8600-79e7-4f59-9d11-b1c9ec3d358a\") " pod="openshift-insights/insights-runtime-extractor-jbsw6"
Oct 30 18:34:13.190548 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:13.190483    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-7dfcf9fd6c-vh769"
Oct 30 18:34:13.193689 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:13.193667    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-pm58m"
Oct 30 18:34:13.203375 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:13.203352    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-jbsw6"
Oct 30 18:34:13.671496 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:13.671471    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-pm58m"]
Oct 30 18:34:13.711075 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:34:13.711049    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda28351c_1c8b_40be_85a0_3aad427e3a17.slice/crio-6474b0fcf840485ef837b117aa34da3859afc311dbd65a7fd439bba45070a419 WatchSource:0}: Error finding container 6474b0fcf840485ef837b117aa34da3859afc311dbd65a7fd439bba45070a419: Status 404 returned error can't find the container with id 6474b0fcf840485ef837b117aa34da3859afc311dbd65a7fd439bba45070a419
Oct 30 18:34:13.723301 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:13.723266    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-7dfcf9fd6c-vh769"]
Oct 30 18:34:13.724137 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:34:13.724118    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57ca8600_79e7_4f59_9d11_b1c9ec3d358a.slice/crio-8a577c507dbeb8fddbe224fe5e15e0af34316aa715235ef581a9f5e0c7ec6047 WatchSource:0}: Error finding container 8a577c507dbeb8fddbe224fe5e15e0af34316aa715235ef581a9f5e0c7ec6047: Status 404 returned error can't find the container with id 8a577c507dbeb8fddbe224fe5e15e0af34316aa715235ef581a9f5e0c7ec6047
Oct 30 18:34:13.724225 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:13.724170    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-jbsw6"]
Oct 30 18:34:14.022193 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:14.022100    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-7dfcf9fd6c-vh769" event={"ID":"da28351c-1c8b-40be-85a0-3aad427e3a17","Type":"ContainerStarted","Data":"49a7f8b46cbe572e22738479580e96df372a1a1c5b46cde7c4f4fc08bfb8cf41"}
Oct 30 18:34:14.022193 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:14.022179    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-7dfcf9fd6c-vh769" event={"ID":"da28351c-1c8b-40be-85a0-3aad427e3a17","Type":"ContainerStarted","Data":"6474b0fcf840485ef837b117aa34da3859afc311dbd65a7fd439bba45070a419"}
Oct 30 18:34:14.023748 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:14.023722    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jbsw6" event={"ID":"57ca8600-79e7-4f59-9d11-b1c9ec3d358a","Type":"ContainerStarted","Data":"b2c02f6cab8fcef765739bd40ad48d76934068b44aa4620ba866aca978f45679"}
Oct 30 18:34:14.023866 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:14.023773    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jbsw6" event={"ID":"57ca8600-79e7-4f59-9d11-b1c9ec3d358a","Type":"ContainerStarted","Data":"8a577c507dbeb8fddbe224fe5e15e0af34316aa715235ef581a9f5e0c7ec6047"}
Oct 30 18:34:14.024754 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:14.024735    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-pm58m" event={"ID":"15d485f9-5a4b-4f4b-9e59-a6eb062b4c5c","Type":"ContainerStarted","Data":"85d1ddeb4c42491288481053c6eb453e5eac9c6afaf5cd5bde122ec2652299b9"}
Oct 30 18:34:14.190969 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:14.190943    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-7dfcf9fd6c-vh769"
Oct 30 18:34:14.193594 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:14.193571    4888 patch_prober.go:29] interesting pod/router-default-7dfcf9fd6c-vh769 container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-proxy-http failed: reason withheld
Oct 30 18:34:14.193594 ip-10-0-134-46 kubenswrapper[4888]: [-]has-synced failed: reason withheld
Oct 30 18:34:14.193594 ip-10-0-134-46 kubenswrapper[4888]: [+]process-running ok
Oct 30 18:34:14.193594 ip-10-0-134-46 kubenswrapper[4888]: healthz check failed
Oct 30 18:34:14.193783 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:14.193610    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-7dfcf9fd6c-vh769" podUID="da28351c-1c8b-40be-85a0-3aad427e3a17" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Oct 30 18:34:15.192036 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:15.192009    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-7dfcf9fd6c-vh769"
Oct 30 18:34:15.634466 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:15.634425    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-xfcdl"
Oct 30 18:34:15.979633 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:15.979539    4888 patch_prober.go:29] interesting pod/image-registry-74757bccd5-qmrkp container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Oct 30 18:34:15.979633 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:15.979603    4888 prober.go:122] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-74757bccd5-qmrkp" podUID="8789ec5b-226e-4551-bf68-3ac585006266" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 18:34:16.036337 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:16.036307    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jbsw6" event={"ID":"57ca8600-79e7-4f59-9d11-b1c9ec3d358a","Type":"ContainerStarted","Data":"8cdad4445ef3d18f44701e88afb6f2062fd1ace4733f3e1ff87300e6f4c5678f"}
Oct 30 18:34:16.036640 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:16.036615    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-7dfcf9fd6c-vh769"
Oct 30 18:34:16.038517 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:16.038499    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-7dfcf9fd6c-vh769"
Oct 30 18:34:16.664032 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:16.663512    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-74757bccd5-qmrkp"]
Oct 30 18:34:16.671335 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:16.671201    4888 patch_prober.go:29] interesting pod/image-registry-74757bccd5-qmrkp container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Oct 30 18:34:16.671335 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:16.671250    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-74757bccd5-qmrkp" podUID="8789ec5b-226e-4551-bf68-3ac585006266" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 18:34:17.042861 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:17.042832    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-pm58m" event={"ID":"15d485f9-5a4b-4f4b-9e59-a6eb062b4c5c","Type":"ContainerStarted","Data":"9e8632e52a5eb1f89c5ccf9e6c5ef85f0c71a6f03bfa08635108246ce470d3ef"}
Oct 30 18:34:17.042968 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:17.042891    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-pm58m" event={"ID":"15d485f9-5a4b-4f4b-9e59-a6eb062b4c5c","Type":"ContainerStarted","Data":"5603d14dcaff04e722e5ddb57ef2bc18e082a09c53d77cd48a742405398b754b"}
Oct 30 18:34:18.002926 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:18.002899    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-r2nx2"
Oct 30 18:34:18.051914 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:18.051073    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jbsw6" event={"ID":"57ca8600-79e7-4f59-9d11-b1c9ec3d358a","Type":"ContainerStarted","Data":"362afcaf0fac798630d6fc95772fdaa5ac98a90a145e83b887724830eb204a05"}
Oct 30 18:34:18.573014 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:18.572975    4888 patch_prober.go:29] interesting pod/image-registry-64bf5b6b76-dpvvs container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Oct 30 18:34:18.573207 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:18.573041    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-64bf5b6b76-dpvvs" podUID="9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 18:34:20.581719 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:20.581688    4888 patch_prober.go:29] interesting pod/image-registry-64bf5b6b76-dpvvs container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Oct 30 18:34:20.582131 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:20.581733    4888 prober.go:122] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-64bf5b6b76-dpvvs" podUID="9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 18:34:25.186549 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:25.186517    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-8xskh"
Oct 30 18:34:26.667772 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:26.667746    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-74757bccd5-qmrkp"
Oct 30 18:34:28.450339 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:28.450306    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-c759s"]
Oct 30 18:34:28.490874 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:28.490850    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-c759s"
Oct 30 18:34:28.573010 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:28.572986    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-64bf5b6b76-dpvvs"
Oct 30 18:34:28.583326 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:28.583304    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Oct 30 18:34:28.595043 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:28.595021    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-tc629\""
Oct 30 18:34:28.595170 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:28.595045    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Oct 30 18:34:28.595170 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:28.595098    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Oct 30 18:34:28.595170 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:28.595113    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Oct 30 18:34:28.623203 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:28.623169    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b1345e7d-caa8-4faf-ac76-e79a57a99e39-node-exporter-textfile\") pod \"node-exporter-c759s\" (UID: \"b1345e7d-caa8-4faf-ac76-e79a57a99e39\") " pod="openshift-monitoring/node-exporter-c759s"
Oct 30 18:34:28.623203 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:28.623199    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b1345e7d-caa8-4faf-ac76-e79a57a99e39-node-exporter-wtmp\") pod \"node-exporter-c759s\" (UID: \"b1345e7d-caa8-4faf-ac76-e79a57a99e39\") " pod="openshift-monitoring/node-exporter-c759s"
Oct 30 18:34:28.623406 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:28.623225    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b1345e7d-caa8-4faf-ac76-e79a57a99e39-metrics-client-ca\") pod \"node-exporter-c759s\" (UID: \"b1345e7d-caa8-4faf-ac76-e79a57a99e39\") " pod="openshift-monitoring/node-exporter-c759s"
Oct 30 18:34:28.623406 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:28.623245    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b1345e7d-caa8-4faf-ac76-e79a57a99e39-node-exporter-accelerators-collector-config\") pod \"node-exporter-c759s\" (UID: \"b1345e7d-caa8-4faf-ac76-e79a57a99e39\") " pod="openshift-monitoring/node-exporter-c759s"
Oct 30 18:34:28.623406 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:28.623395    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b1345e7d-caa8-4faf-ac76-e79a57a99e39-node-exporter-tls\") pod \"node-exporter-c759s\" (UID: \"b1345e7d-caa8-4faf-ac76-e79a57a99e39\") " pod="openshift-monitoring/node-exporter-c759s"
Oct 30 18:34:28.623548 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:28.623431    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b1345e7d-caa8-4faf-ac76-e79a57a99e39-root\") pod \"node-exporter-c759s\" (UID: \"b1345e7d-caa8-4faf-ac76-e79a57a99e39\") " pod="openshift-monitoring/node-exporter-c759s"
Oct 30 18:34:28.623548 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:28.623448    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b1345e7d-caa8-4faf-ac76-e79a57a99e39-sys\") pod \"node-exporter-c759s\" (UID: \"b1345e7d-caa8-4faf-ac76-e79a57a99e39\") " pod="openshift-monitoring/node-exporter-c759s"
Oct 30 18:34:28.623548 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:28.623492    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qpdw5\" (UniqueName: \"kubernetes.io/projected/b1345e7d-caa8-4faf-ac76-e79a57a99e39-kube-api-access-qpdw5\") pod \"node-exporter-c759s\" (UID: \"b1345e7d-caa8-4faf-ac76-e79a57a99e39\") " pod="openshift-monitoring/node-exporter-c759s"
Oct 30 18:34:28.623692 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:28.623618    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b1345e7d-caa8-4faf-ac76-e79a57a99e39-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-c759s\" (UID: \"b1345e7d-caa8-4faf-ac76-e79a57a99e39\") " pod="openshift-monitoring/node-exporter-c759s"
Oct 30 18:34:28.724772 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:28.724687    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b1345e7d-caa8-4faf-ac76-e79a57a99e39-node-exporter-tls\") pod \"node-exporter-c759s\" (UID: \"b1345e7d-caa8-4faf-ac76-e79a57a99e39\") " pod="openshift-monitoring/node-exporter-c759s"
Oct 30 18:34:28.724772 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:28.724740    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b1345e7d-caa8-4faf-ac76-e79a57a99e39-root\") pod \"node-exporter-c759s\" (UID: \"b1345e7d-caa8-4faf-ac76-e79a57a99e39\") " pod="openshift-monitoring/node-exporter-c759s"
Oct 30 18:34:28.725008 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:28.724796    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b1345e7d-caa8-4faf-ac76-e79a57a99e39-root\") pod \"node-exporter-c759s\" (UID: \"b1345e7d-caa8-4faf-ac76-e79a57a99e39\") " pod="openshift-monitoring/node-exporter-c759s"
Oct 30 18:34:28.725008 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:28.724859    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b1345e7d-caa8-4faf-ac76-e79a57a99e39-sys\") pod \"node-exporter-c759s\" (UID: \"b1345e7d-caa8-4faf-ac76-e79a57a99e39\") " pod="openshift-monitoring/node-exporter-c759s"
Oct 30 18:34:28.725008 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:28.724893    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qpdw5\" (UniqueName: \"kubernetes.io/projected/b1345e7d-caa8-4faf-ac76-e79a57a99e39-kube-api-access-qpdw5\") pod \"node-exporter-c759s\" (UID: \"b1345e7d-caa8-4faf-ac76-e79a57a99e39\") " pod="openshift-monitoring/node-exporter-c759s"
Oct 30 18:34:28.725008 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:28.724937    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b1345e7d-caa8-4faf-ac76-e79a57a99e39-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-c759s\" (UID: \"b1345e7d-caa8-4faf-ac76-e79a57a99e39\") " pod="openshift-monitoring/node-exporter-c759s"
Oct 30 18:34:28.725226 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:28.725014    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b1345e7d-caa8-4faf-ac76-e79a57a99e39-sys\") pod \"node-exporter-c759s\" (UID: \"b1345e7d-caa8-4faf-ac76-e79a57a99e39\") " pod="openshift-monitoring/node-exporter-c759s"
Oct 30 18:34:28.725226 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:28.725014    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b1345e7d-caa8-4faf-ac76-e79a57a99e39-node-exporter-textfile\") pod \"node-exporter-c759s\" (UID: \"b1345e7d-caa8-4faf-ac76-e79a57a99e39\") " pod="openshift-monitoring/node-exporter-c759s"
Oct 30 18:34:28.725226 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:28.725057    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b1345e7d-caa8-4faf-ac76-e79a57a99e39-node-exporter-wtmp\") pod \"node-exporter-c759s\" (UID: \"b1345e7d-caa8-4faf-ac76-e79a57a99e39\") " pod="openshift-monitoring/node-exporter-c759s"
Oct 30 18:34:28.725226 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:28.725126    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b1345e7d-caa8-4faf-ac76-e79a57a99e39-metrics-client-ca\") pod \"node-exporter-c759s\" (UID: \"b1345e7d-caa8-4faf-ac76-e79a57a99e39\") " pod="openshift-monitoring/node-exporter-c759s"
Oct 30 18:34:28.725226 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:28.725161    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b1345e7d-caa8-4faf-ac76-e79a57a99e39-node-exporter-accelerators-collector-config\") pod \"node-exporter-c759s\" (UID: \"b1345e7d-caa8-4faf-ac76-e79a57a99e39\") " pod="openshift-monitoring/node-exporter-c759s"
Oct 30 18:34:28.725540 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:28.725325    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b1345e7d-caa8-4faf-ac76-e79a57a99e39-node-exporter-textfile\") pod \"node-exporter-c759s\" (UID: \"b1345e7d-caa8-4faf-ac76-e79a57a99e39\") " pod="openshift-monitoring/node-exporter-c759s"
Oct 30 18:34:28.725540 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:28.725527    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b1345e7d-caa8-4faf-ac76-e79a57a99e39-node-exporter-wtmp\") pod \"node-exporter-c759s\" (UID: \"b1345e7d-caa8-4faf-ac76-e79a57a99e39\") " pod="openshift-monitoring/node-exporter-c759s"
Oct 30 18:34:28.725819 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:28.725795    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b1345e7d-caa8-4faf-ac76-e79a57a99e39-node-exporter-accelerators-collector-config\") pod \"node-exporter-c759s\" (UID: \"b1345e7d-caa8-4faf-ac76-e79a57a99e39\") " pod="openshift-monitoring/node-exporter-c759s"
Oct 30 18:34:28.725868 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:28.725818    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b1345e7d-caa8-4faf-ac76-e79a57a99e39-metrics-client-ca\") pod \"node-exporter-c759s\" (UID: \"b1345e7d-caa8-4faf-ac76-e79a57a99e39\") " pod="openshift-monitoring/node-exporter-c759s"
Oct 30 18:34:28.727341 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:28.727316    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b1345e7d-caa8-4faf-ac76-e79a57a99e39-node-exporter-tls\") pod \"node-exporter-c759s\" (UID: \"b1345e7d-caa8-4faf-ac76-e79a57a99e39\") " pod="openshift-monitoring/node-exporter-c759s"
Oct 30 18:34:28.727480 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:28.727460    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b1345e7d-caa8-4faf-ac76-e79a57a99e39-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-c759s\" (UID: \"b1345e7d-caa8-4faf-ac76-e79a57a99e39\") " pod="openshift-monitoring/node-exporter-c759s"
Oct 30 18:34:28.794691 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:28.794656    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qpdw5\" (UniqueName: \"kubernetes.io/projected/b1345e7d-caa8-4faf-ac76-e79a57a99e39-kube-api-access-qpdw5\") pod \"node-exporter-c759s\" (UID: \"b1345e7d-caa8-4faf-ac76-e79a57a99e39\") " pod="openshift-monitoring/node-exporter-c759s"
Oct 30 18:34:28.843481 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:28.843454    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-c759s"
Oct 30 18:34:28.874130 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:34:28.874102    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1345e7d_caa8_4faf_ac76_e79a57a99e39.slice/crio-7b59c61097e05efc5e6281b5e94af2c101140da51db5fd4233392b5a48ad4942 WatchSource:0}: Error finding container 7b59c61097e05efc5e6281b5e94af2c101140da51db5fd4233392b5a48ad4942: Status 404 returned error can't find the container with id 7b59c61097e05efc5e6281b5e94af2c101140da51db5fd4233392b5a48ad4942
Oct 30 18:34:29.093234 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:29.093201    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-c759s" event={"ID":"b1345e7d-caa8-4faf-ac76-e79a57a99e39","Type":"ContainerStarted","Data":"7b59c61097e05efc5e6281b5e94af2c101140da51db5fd4233392b5a48ad4942"}
Oct 30 18:34:31.103343 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:31.103310    4888 generic.go:356] "Generic (PLEG): container finished" podID="b1345e7d-caa8-4faf-ac76-e79a57a99e39" containerID="6937119fc60b48599aa63fcd865dfb60e4dd660db1fcca235e2254453c697a95" exitCode=0
Oct 30 18:34:31.103769 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:31.103361    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-c759s" event={"ID":"b1345e7d-caa8-4faf-ac76-e79a57a99e39","Type":"ContainerDied","Data":"6937119fc60b48599aa63fcd865dfb60e4dd660db1fcca235e2254453c697a95"}
Oct 30 18:34:32.109066 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:32.109035    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-c759s" event={"ID":"b1345e7d-caa8-4faf-ac76-e79a57a99e39","Type":"ContainerStarted","Data":"fac80492611764c9d498e56d21dd2244ecaa310c99671ae976d9f8c4dcdaf01b"}
Oct 30 18:34:32.109433 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:32.109082    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-c759s" event={"ID":"b1345e7d-caa8-4faf-ac76-e79a57a99e39","Type":"ContainerStarted","Data":"fa149fb4cf73b95df347227b7f2948c316df25b275cbbea89f74b5ec424b2b3d"}
Oct 30 18:34:32.281275 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:32.281213    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-c759s" podStartSLOduration=2.380439157 podStartE2EDuration="4.281200724s" totalImagesPullingTime="1.813102566s" totalInitContainerRuntime="87.659001ms" isStatefulPod=true podCreationTimestamp="2026-10-30 18:34:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:34:32.280627928 +0000 UTC m=+36.528033952" watchObservedRunningTime="2026-10-30 18:34:32.281200724 +0000 UTC m=+36.528606789"
Oct 30 18:34:35.616665 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:34:35.616577    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-jbsw6" containerName="extractor"
Oct 30 18:34:39.383500 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:39.383466    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-64bf5b6b76-dpvvs"]
Oct 30 18:34:39.759920 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:39.759836    4888 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmdpp_8ecf2b62-8053-4bba-a852-c6f6119d3288/konnectivity-agent/0.log" limit=1024
Oct 30 18:34:39.988400 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:39.988360    4888 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-lmdpp_8ecf2b62-8053-4bba-a852-c6f6119d3288/konnectivity-agent/0.log" limit=1024
Oct 30 18:34:41.698407 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:41.698367    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-image-registry/image-registry-74757bccd5-qmrkp" podUID="8789ec5b-226e-4551-bf68-3ac585006266" containerName="registry" containerID="cri-o://e0cf016a42be2cc96c517195f83aded8b7b76dbfa849ddcbdd7dfcb378295c5c" gracePeriod=30
Oct 30 18:34:41.937263 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:41.937242    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-74757bccd5-qmrkp"
Oct 30 18:34:42.034576 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:42.034499    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8789ec5b-226e-4551-bf68-3ac585006266-trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8789ec5b-226e-4551-bf68-3ac585006266-trusted-ca\") pod \"8789ec5b-226e-4551-bf68-3ac585006266\" (UID: \"8789ec5b-226e-4551-bf68-3ac585006266\") "
Oct 30 18:34:42.034576 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:42.034542    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8789ec5b-226e-4551-bf68-3ac585006266-bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8789ec5b-226e-4551-bf68-3ac585006266-bound-sa-token\") pod \"8789ec5b-226e-4551-bf68-3ac585006266\" (UID: \"8789ec5b-226e-4551-bf68-3ac585006266\") "
Oct 30 18:34:42.034576 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:42.034574    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8789ec5b-226e-4551-bf68-3ac585006266-ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8789ec5b-226e-4551-bf68-3ac585006266-ca-trust-extracted\") pod \"8789ec5b-226e-4551-bf68-3ac585006266\" (UID: \"8789ec5b-226e-4551-bf68-3ac585006266\") "
Oct 30 18:34:42.034801 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:42.034602    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8789ec5b-226e-4551-bf68-3ac585006266-installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8789ec5b-226e-4551-bf68-3ac585006266-installation-pull-secrets\") pod \"8789ec5b-226e-4551-bf68-3ac585006266\" (UID: \"8789ec5b-226e-4551-bf68-3ac585006266\") "
Oct 30 18:34:42.034801 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:42.034621    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8789ec5b-226e-4551-bf68-3ac585006266-kube-api-access-vf948\" (UniqueName: \"kubernetes.io/projected/8789ec5b-226e-4551-bf68-3ac585006266-kube-api-access-vf948\") pod \"8789ec5b-226e-4551-bf68-3ac585006266\" (UID: \"8789ec5b-226e-4551-bf68-3ac585006266\") "
Oct 30 18:34:42.034801 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:42.034651    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8789ec5b-226e-4551-bf68-3ac585006266-image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8789ec5b-226e-4551-bf68-3ac585006266-image-registry-private-configuration\") pod \"8789ec5b-226e-4551-bf68-3ac585006266\" (UID: \"8789ec5b-226e-4551-bf68-3ac585006266\") "
Oct 30 18:34:42.034940 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:42.034907    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8789ec5b-226e-4551-bf68-3ac585006266-registry-tls\" (UniqueName: \"kubernetes.io/projected/8789ec5b-226e-4551-bf68-3ac585006266-registry-tls\") pod \"8789ec5b-226e-4551-bf68-3ac585006266\" (UID: \"8789ec5b-226e-4551-bf68-3ac585006266\") "
Oct 30 18:34:42.035277 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:42.034994    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8789ec5b-226e-4551-bf68-3ac585006266-registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8789ec5b-226e-4551-bf68-3ac585006266-registry-certificates\") pod \"8789ec5b-226e-4551-bf68-3ac585006266\" (UID: \"8789ec5b-226e-4551-bf68-3ac585006266\") "
Oct 30 18:34:42.035277 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:42.034994    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8789ec5b-226e-4551-bf68-3ac585006266-trusted-ca" pod "8789ec5b-226e-4551-bf68-3ac585006266" (UID: "8789ec5b-226e-4551-bf68-3ac585006266"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:34:42.035465 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:42.035441    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8789ec5b-226e-4551-bf68-3ac585006266-registry-certificates" pod "8789ec5b-226e-4551-bf68-3ac585006266" (UID: "8789ec5b-226e-4551-bf68-3ac585006266"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:34:42.037740 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:42.037711    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8789ec5b-226e-4551-bf68-3ac585006266-registry-tls" pod "8789ec5b-226e-4551-bf68-3ac585006266" (UID: "8789ec5b-226e-4551-bf68-3ac585006266"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:34:42.037870 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:42.037847    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8789ec5b-226e-4551-bf68-3ac585006266-bound-sa-token" pod "8789ec5b-226e-4551-bf68-3ac585006266" (UID: "8789ec5b-226e-4551-bf68-3ac585006266"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:34:42.037941 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:42.037883    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8789ec5b-226e-4551-bf68-3ac585006266-image-registry-private-configuration" pod "8789ec5b-226e-4551-bf68-3ac585006266" (UID: "8789ec5b-226e-4551-bf68-3ac585006266"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:34:42.037941 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:42.037930    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8789ec5b-226e-4551-bf68-3ac585006266-installation-pull-secrets" pod "8789ec5b-226e-4551-bf68-3ac585006266" (UID: "8789ec5b-226e-4551-bf68-3ac585006266"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:34:42.038043 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:42.038001    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8789ec5b-226e-4551-bf68-3ac585006266-kube-api-access-vf948" pod "8789ec5b-226e-4551-bf68-3ac585006266" (UID: "8789ec5b-226e-4551-bf68-3ac585006266"). InnerVolumeSpecName "kube-api-access-vf948". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:34:42.041741 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:42.041709    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8789ec5b-226e-4551-bf68-3ac585006266-ca-trust-extracted" pod "8789ec5b-226e-4551-bf68-3ac585006266" (UID: "8789ec5b-226e-4551-bf68-3ac585006266"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:34:42.136091 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:42.136060    4888 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8789ec5b-226e-4551-bf68-3ac585006266-registry-certificates\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:34:42.136091 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:42.136086    4888 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8789ec5b-226e-4551-bf68-3ac585006266-trusted-ca\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:34:42.136278 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:42.136101    4888 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8789ec5b-226e-4551-bf68-3ac585006266-bound-sa-token\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:34:42.136278 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:42.136114    4888 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8789ec5b-226e-4551-bf68-3ac585006266-ca-trust-extracted\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:34:42.136278 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:42.136123    4888 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8789ec5b-226e-4551-bf68-3ac585006266-installation-pull-secrets\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:34:42.136278 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:42.136131    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vf948\" (UniqueName: \"kubernetes.io/projected/8789ec5b-226e-4551-bf68-3ac585006266-kube-api-access-vf948\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:34:42.136278 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:42.136141    4888 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8789ec5b-226e-4551-bf68-3ac585006266-image-registry-private-configuration\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:34:42.136278 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:42.136150    4888 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8789ec5b-226e-4551-bf68-3ac585006266-registry-tls\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:34:42.147559 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:42.147524    4888 generic.go:356] "Generic (PLEG): container finished" podID="8789ec5b-226e-4551-bf68-3ac585006266" containerID="e0cf016a42be2cc96c517195f83aded8b7b76dbfa849ddcbdd7dfcb378295c5c" exitCode=0
Oct 30 18:34:42.147705 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:42.147578    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-74757bccd5-qmrkp"
Oct 30 18:34:42.147705 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:42.147590    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-74757bccd5-qmrkp" event={"ID":"8789ec5b-226e-4551-bf68-3ac585006266","Type":"ContainerDied","Data":"e0cf016a42be2cc96c517195f83aded8b7b76dbfa849ddcbdd7dfcb378295c5c"}
Oct 30 18:34:42.147705 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:42.147640    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-74757bccd5-qmrkp" event={"ID":"8789ec5b-226e-4551-bf68-3ac585006266","Type":"ContainerDied","Data":"af9cd9640d45b150520e18d32103869659c5392878f7e76255779a09719a65eb"}
Oct 30 18:34:42.147705 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:42.147662    4888 scope.go:122] "RemoveContainer" containerID="e0cf016a42be2cc96c517195f83aded8b7b76dbfa849ddcbdd7dfcb378295c5c"
Oct 30 18:34:42.159710 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:42.156634    4888 scope.go:122] "RemoveContainer" containerID="e0cf016a42be2cc96c517195f83aded8b7b76dbfa849ddcbdd7dfcb378295c5c"
Oct 30 18:34:42.159710 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:34:42.156945    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e0cf016a42be2cc96c517195f83aded8b7b76dbfa849ddcbdd7dfcb378295c5c\": container with ID starting with e0cf016a42be2cc96c517195f83aded8b7b76dbfa849ddcbdd7dfcb378295c5c not found: ID does not exist" containerID="e0cf016a42be2cc96c517195f83aded8b7b76dbfa849ddcbdd7dfcb378295c5c"
Oct 30 18:34:42.159710 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:42.156984    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e0cf016a42be2cc96c517195f83aded8b7b76dbfa849ddcbdd7dfcb378295c5c"} err="failed to get container status \"e0cf016a42be2cc96c517195f83aded8b7b76dbfa849ddcbdd7dfcb378295c5c\": rpc error: code = NotFound desc = could not find container \"e0cf016a42be2cc96c517195f83aded8b7b76dbfa849ddcbdd7dfcb378295c5c\": container with ID starting with e0cf016a42be2cc96c517195f83aded8b7b76dbfa849ddcbdd7dfcb378295c5c not found: ID does not exist"
Oct 30 18:34:42.315059 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:42.314980    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-74757bccd5-qmrkp"]
Oct 30 18:34:42.319123 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:42.319095    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-74757bccd5-qmrkp"]
Oct 30 18:34:43.888693 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:34:43.888652    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8789ec5b-226e-4551-bf68-3ac585006266" path="/var/lib/kubelet/pods/8789ec5b-226e-4551-bf68-3ac585006266/volumes"
Oct 30 18:35:04.403116 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:35:04.403059    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-image-registry/image-registry-64bf5b6b76-dpvvs" podUID="9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351" containerName="registry" containerID="cri-o://9cc96a881f01d5d37e977a094ed4d424114e78a12531bee8e8dd9d5ea62ef811" gracePeriod=30
Oct 30 18:35:04.640846 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:35:04.640824    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-64bf5b6b76-dpvvs"
Oct 30 18:35:04.693886 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:35:04.693818    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351-image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351-image-registry-private-configuration\") pod \"9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351\" (UID: \"9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351\") "
Oct 30 18:35:04.693886 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:35:04.693856    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351-registry-tls\" (UniqueName: \"kubernetes.io/projected/9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351-registry-tls\") pod \"9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351\" (UID: \"9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351\") "
Oct 30 18:35:04.693886 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:35:04.693876    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351-trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351-trusted-ca\") pod \"9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351\" (UID: \"9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351\") "
Oct 30 18:35:04.694166 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:35:04.693909    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351-installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351-installation-pull-secrets\") pod \"9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351\" (UID: \"9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351\") "
Oct 30 18:35:04.694166 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:35:04.693940    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351-registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351-registry-certificates\") pod \"9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351\" (UID: \"9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351\") "
Oct 30 18:35:04.694166 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:35:04.694053    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351-kube-api-access-vxw47\" (UniqueName: \"kubernetes.io/projected/9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351-kube-api-access-vxw47\") pod \"9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351\" (UID: \"9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351\") "
Oct 30 18:35:04.694166 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:35:04.694117    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351-ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351-ca-trust-extracted\") pod \"9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351\" (UID: \"9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351\") "
Oct 30 18:35:04.694503 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:35:04.694180    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351-bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351-bound-sa-token\") pod \"9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351\" (UID: \"9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351\") "
Oct 30 18:35:04.694503 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:35:04.694373    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351-registry-certificates" pod "9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351" (UID: "9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:35:04.694612 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:35:04.694500    4888 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351-registry-certificates\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:35:04.694898 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:35:04.694857    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351-trusted-ca" pod "9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351" (UID: "9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:35:04.696697 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:35:04.696669    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351-kube-api-access-vxw47" pod "9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351" (UID: "9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351"). InnerVolumeSpecName "kube-api-access-vxw47". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:35:04.696840 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:35:04.696821    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351-registry-tls" pod "9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351" (UID: "9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:35:04.696993 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:35:04.696967    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351-installation-pull-secrets" pod "9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351" (UID: "9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:35:04.697063 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:35:04.697037    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351-bound-sa-token" pod "9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351" (UID: "9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:35:04.697120 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:35:04.697066    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351-image-registry-private-configuration" pod "9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351" (UID: "9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:35:04.701763 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:35:04.701735    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351-ca-trust-extracted" pod "9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351" (UID: "9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:35:04.794833 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:35:04.794799    4888 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351-ca-trust-extracted\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:35:04.794833 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:35:04.794828    4888 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351-bound-sa-token\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:35:04.794833 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:35:04.794839    4888 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351-image-registry-private-configuration\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:35:04.795045 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:35:04.794848    4888 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351-registry-tls\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:35:04.795045 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:35:04.794857    4888 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351-trusted-ca\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:35:04.795045 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:35:04.794865    4888 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351-installation-pull-secrets\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:35:04.795045 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:35:04.794876    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vxw47\" (UniqueName: \"kubernetes.io/projected/9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351-kube-api-access-vxw47\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:35:05.241658 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:35:05.241622    4888 generic.go:356] "Generic (PLEG): container finished" podID="9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351" containerID="9cc96a881f01d5d37e977a094ed4d424114e78a12531bee8e8dd9d5ea62ef811" exitCode=0
Oct 30 18:35:05.241828 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:35:05.241679    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-64bf5b6b76-dpvvs"
Oct 30 18:35:05.241828 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:35:05.241711    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-64bf5b6b76-dpvvs" event={"ID":"9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351","Type":"ContainerDied","Data":"9cc96a881f01d5d37e977a094ed4d424114e78a12531bee8e8dd9d5ea62ef811"}
Oct 30 18:35:05.241828 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:35:05.241779    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-64bf5b6b76-dpvvs" event={"ID":"9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351","Type":"ContainerDied","Data":"f0669606e17e7131c510c046039ad6044074fb9fe711bb78061b3e4f82816e2d"}
Oct 30 18:35:05.241828 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:35:05.241797    4888 scope.go:122] "RemoveContainer" containerID="9cc96a881f01d5d37e977a094ed4d424114e78a12531bee8e8dd9d5ea62ef811"
Oct 30 18:35:05.250599 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:35:05.250583    4888 scope.go:122] "RemoveContainer" containerID="9cc96a881f01d5d37e977a094ed4d424114e78a12531bee8e8dd9d5ea62ef811"
Oct 30 18:35:05.250842 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:35:05.250823    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9cc96a881f01d5d37e977a094ed4d424114e78a12531bee8e8dd9d5ea62ef811\": container with ID starting with 9cc96a881f01d5d37e977a094ed4d424114e78a12531bee8e8dd9d5ea62ef811 not found: ID does not exist" containerID="9cc96a881f01d5d37e977a094ed4d424114e78a12531bee8e8dd9d5ea62ef811"
Oct 30 18:35:05.250889 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:35:05.250849    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9cc96a881f01d5d37e977a094ed4d424114e78a12531bee8e8dd9d5ea62ef811"} err="failed to get container status \"9cc96a881f01d5d37e977a094ed4d424114e78a12531bee8e8dd9d5ea62ef811\": rpc error: code = NotFound desc = could not find container \"9cc96a881f01d5d37e977a094ed4d424114e78a12531bee8e8dd9d5ea62ef811\": container with ID starting with 9cc96a881f01d5d37e977a094ed4d424114e78a12531bee8e8dd9d5ea62ef811 not found: ID does not exist"
Oct 30 18:35:05.409740 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:35:05.409710    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-64bf5b6b76-dpvvs"]
Oct 30 18:35:05.460812 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:35:05.460784    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-64bf5b6b76-dpvvs"]
Oct 30 18:35:05.616465 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:35:05.616429    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-jbsw6" containerName="extractor"
Oct 30 18:35:05.887669 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:35:05.887599    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351" path="/var/lib/kubelet/pods/9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351/volumes"
Oct 30 18:35:06.250135 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:35:06.250057    4888 generic.go:356] "Generic (PLEG): container finished" podID="7c09088b-3a07-426d-8804-f0f5bac4d27c" containerID="276addd4803db94cae7b9a84b5533cf0ec00c4618e8b8d16f123052f2f82ceb4" exitCode=0
Oct 30 18:35:06.250135 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:35:06.250118    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-7c4986d449-xvgqc" event={"ID":"7c09088b-3a07-426d-8804-f0f5bac4d27c","Type":"ContainerDied","Data":"276addd4803db94cae7b9a84b5533cf0ec00c4618e8b8d16f123052f2f82ceb4"}
Oct 30 18:35:06.250443 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:35:06.250429    4888 scope.go:122] "RemoveContainer" containerID="276addd4803db94cae7b9a84b5533cf0ec00c4618e8b8d16f123052f2f82ceb4"
Oct 30 18:35:07.256018 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:35:07.255939    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-7c4986d449-xvgqc" event={"ID":"7c09088b-3a07-426d-8804-f0f5bac4d27c","Type":"ContainerStarted","Data":"a27c26859a6cfcdee31bdc5619c8b6817611eee5f8f77dfb8ecd897d2cdae1d7"}
Oct 30 18:35:07.257388 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:35:07.257367    4888 generic.go:356] "Generic (PLEG): container finished" podID="09c13759-50d4-40e6-b854-4691822b8c8a" containerID="caf9fc34319c58ec44f0f47e3b04f2501dfca1c85bb798c8fcb7f1823eecac04" exitCode=0
Oct 30 18:35:07.257485 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:35:07.257406    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-5bqdc" event={"ID":"09c13759-50d4-40e6-b854-4691822b8c8a","Type":"ContainerDied","Data":"caf9fc34319c58ec44f0f47e3b04f2501dfca1c85bb798c8fcb7f1823eecac04"}
Oct 30 18:35:07.257656 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:35:07.257644    4888 scope.go:122] "RemoveContainer" containerID="caf9fc34319c58ec44f0f47e3b04f2501dfca1c85bb798c8fcb7f1823eecac04"
Oct 30 18:35:08.263436 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:35:08.263399    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-5bqdc" event={"ID":"09c13759-50d4-40e6-b854-4691822b8c8a","Type":"ContainerStarted","Data":"451ff7225b73fef7f371398ff77076e2a189f35e4442d88b4f591840cf18ecca"}
Oct 30 18:35:17.297362 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:35:17.297328    4888 generic.go:356] "Generic (PLEG): container finished" podID="36a04851-81db-40cd-97ef-8e2e00a08604" containerID="2e0d65f31c3b3a319575ad6c5edc8c6071ebc79d492e57cf2b3edc5ede37edbe" exitCode=0
Oct 30 18:35:17.297827 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:35:17.297402    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-kr5qq" event={"ID":"36a04851-81db-40cd-97ef-8e2e00a08604","Type":"ContainerDied","Data":"2e0d65f31c3b3a319575ad6c5edc8c6071ebc79d492e57cf2b3edc5ede37edbe"}
Oct 30 18:35:17.297827 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:35:17.297777    4888 scope.go:122] "RemoveContainer" containerID="2e0d65f31c3b3a319575ad6c5edc8c6071ebc79d492e57cf2b3edc5ede37edbe"
Oct 30 18:35:18.305791 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:35:18.305756    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-kr5qq" event={"ID":"36a04851-81db-40cd-97ef-8e2e00a08604","Type":"ContainerStarted","Data":"66e79e3ecaf2ebd16da49369bc663e4cf5f98d72472215fee3a6449636a90be4"}
Oct 30 18:35:35.615888 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:35:35.615853    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-jbsw6" containerName="extractor"
Oct 30 18:36:05.616256 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:36:05.616178    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-jbsw6" containerName="extractor"
Oct 30 18:36:35.615934 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:36:35.615895    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-jbsw6" containerName="extractor"
Oct 30 18:37:05.617357 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:37:05.617313    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-jbsw6" containerName="extractor"
Oct 30 18:37:35.616226 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:37:35.616139    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-jbsw6" containerName="extractor"
Oct 30 18:37:42.649556 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:42.649520    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c6w88m"]
Oct 30 18:37:42.650018 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:42.649806    4888 cpu_manager.go:414] "containerMap: removing container" podUID="8789ec5b-226e-4551-bf68-3ac585006266" containerName="registry"
Oct 30 18:37:42.650018 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:42.649817    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8789ec5b-226e-4551-bf68-3ac585006266" containerName="registry"
Oct 30 18:37:42.650018 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:42.649830    4888 cpu_manager.go:414] "containerMap: removing container" podUID="9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351" containerName="registry"
Oct 30 18:37:42.650018 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:42.649834    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351" containerName="registry"
Oct 30 18:37:42.650018 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:42.649885    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="8789ec5b-226e-4551-bf68-3ac585006266" containerName="registry"
Oct 30 18:37:42.650018 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:42.649893    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="9bdb5a6c-6c32-4e70-b4cf-6e4d610a8351" containerName="registry"
Oct 30 18:37:42.653936 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:42.653912    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c6w88m"
Oct 30 18:37:42.713297 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:42.713257    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c6w88m"]
Oct 30 18:37:42.763218 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:42.763175    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Oct 30 18:37:42.763218 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:42.763197    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"c7537007944577f64df34960ace3c69136c416-139ec370-dockercfg-v4gpw\""
Oct 30 18:37:42.763218 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:42.763191    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Oct 30 18:37:42.807830 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:42.807800    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m9lh9\" (UniqueName: \"kubernetes.io/projected/bf03a161-80e8-4191-b562-661a7117035a-kube-api-access-m9lh9\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c6w88m\" (UID: \"bf03a161-80e8-4191-b562-661a7117035a\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c6w88m"
Oct 30 18:37:42.807977 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:42.807845    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bf03a161-80e8-4191-b562-661a7117035a-util\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c6w88m\" (UID: \"bf03a161-80e8-4191-b562-661a7117035a\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c6w88m"
Oct 30 18:37:42.808035 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:42.807986    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bf03a161-80e8-4191-b562-661a7117035a-bundle\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c6w88m\" (UID: \"bf03a161-80e8-4191-b562-661a7117035a\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c6w88m"
Oct 30 18:37:42.908553 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:42.908491    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bf03a161-80e8-4191-b562-661a7117035a-bundle\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c6w88m\" (UID: \"bf03a161-80e8-4191-b562-661a7117035a\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c6w88m"
Oct 30 18:37:42.908553 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:42.908528    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m9lh9\" (UniqueName: \"kubernetes.io/projected/bf03a161-80e8-4191-b562-661a7117035a-kube-api-access-m9lh9\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c6w88m\" (UID: \"bf03a161-80e8-4191-b562-661a7117035a\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c6w88m"
Oct 30 18:37:42.908716 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:42.908568    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bf03a161-80e8-4191-b562-661a7117035a-util\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c6w88m\" (UID: \"bf03a161-80e8-4191-b562-661a7117035a\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c6w88m"
Oct 30 18:37:42.908952 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:42.908932    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bf03a161-80e8-4191-b562-661a7117035a-bundle\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c6w88m\" (UID: \"bf03a161-80e8-4191-b562-661a7117035a\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c6w88m"
Oct 30 18:37:42.908994 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:42.908935    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bf03a161-80e8-4191-b562-661a7117035a-util\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c6w88m\" (UID: \"bf03a161-80e8-4191-b562-661a7117035a\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c6w88m"
Oct 30 18:37:42.971064 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:42.971033    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m9lh9\" (UniqueName: \"kubernetes.io/projected/bf03a161-80e8-4191-b562-661a7117035a-kube-api-access-m9lh9\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c6w88m\" (UID: \"bf03a161-80e8-4191-b562-661a7117035a\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c6w88m"
Oct 30 18:37:43.055604 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:43.055576    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqspzw"]
Oct 30 18:37:43.058914 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:43.058899    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqspzw"
Oct 30 18:37:43.115780 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:43.115755    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"e3d011e69d9573cabb8c359e5ac709eb5ac240-be78d160-dockercfg-sgs22\""
Oct 30 18:37:43.120604 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:43.120583    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqspzw"]
Oct 30 18:37:43.163656 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:43.163590    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6mwnhv"]
Oct 30 18:37:43.166901 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:43.166885    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6mwnhv"
Oct 30 18:37:43.211492 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:43.211465    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7b3553c3-4af3-49ce-9cd9-b22198c1e7be-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqspzw\" (UID: \"7b3553c3-4af3-49ce-9cd9-b22198c1e7be\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqspzw"
Oct 30 18:37:43.211634 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:43.211497    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fmv6d\" (UniqueName: \"kubernetes.io/projected/7b3553c3-4af3-49ce-9cd9-b22198c1e7be-kube-api-access-fmv6d\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqspzw\" (UID: \"7b3553c3-4af3-49ce-9cd9-b22198c1e7be\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqspzw"
Oct 30 18:37:43.211634 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:43.211530    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7b3553c3-4af3-49ce-9cd9-b22198c1e7be-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqspzw\" (UID: \"7b3553c3-4af3-49ce-9cd9-b22198c1e7be\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqspzw"
Oct 30 18:37:43.220762 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:43.220743    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3-b83c8cba-dockercfg-dfd7w\""
Oct 30 18:37:43.228014 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:43.227990    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6mwnhv"]
Oct 30 18:37:43.245114 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:43.245091    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847acr7cc"]
Oct 30 18:37:43.248318 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:43.248299    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847acr7cc"
Oct 30 18:37:43.269600 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:43.269578    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c6w88m"
Oct 30 18:37:43.307208 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:43.307170    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847acr7cc"]
Oct 30 18:37:43.312545 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:43.312516    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7b3553c3-4af3-49ce-9cd9-b22198c1e7be-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqspzw\" (UID: \"7b3553c3-4af3-49ce-9cd9-b22198c1e7be\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqspzw"
Oct 30 18:37:43.312676 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:43.312581    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t6v8t\" (UniqueName: \"kubernetes.io/projected/95475eb2-3d8f-469d-b1a1-e5cdf7567012-kube-api-access-t6v8t\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6mwnhv\" (UID: \"95475eb2-3d8f-469d-b1a1-e5cdf7567012\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6mwnhv"
Oct 30 18:37:43.312676 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:43.312620    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/95475eb2-3d8f-469d-b1a1-e5cdf7567012-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6mwnhv\" (UID: \"95475eb2-3d8f-469d-b1a1-e5cdf7567012\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6mwnhv"
Oct 30 18:37:43.312775 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:43.312751    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/12fe33f9-270e-41c7-b3ab-2ba8b0678a37-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847acr7cc\" (UID: \"12fe33f9-270e-41c7-b3ab-2ba8b0678a37\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847acr7cc"
Oct 30 18:37:43.312815 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:43.312801    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/95475eb2-3d8f-469d-b1a1-e5cdf7567012-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6mwnhv\" (UID: \"95475eb2-3d8f-469d-b1a1-e5cdf7567012\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6mwnhv"
Oct 30 18:37:43.312851 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:43.312833    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/12fe33f9-270e-41c7-b3ab-2ba8b0678a37-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847acr7cc\" (UID: \"12fe33f9-270e-41c7-b3ab-2ba8b0678a37\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847acr7cc"
Oct 30 18:37:43.312897 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:43.312856    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7b3553c3-4af3-49ce-9cd9-b22198c1e7be-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqspzw\" (UID: \"7b3553c3-4af3-49ce-9cd9-b22198c1e7be\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqspzw"
Oct 30 18:37:43.312897 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:43.312863    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5vknr\" (UniqueName: \"kubernetes.io/projected/12fe33f9-270e-41c7-b3ab-2ba8b0678a37-kube-api-access-5vknr\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847acr7cc\" (UID: \"12fe33f9-270e-41c7-b3ab-2ba8b0678a37\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847acr7cc"
Oct 30 18:37:43.313019 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:43.313005    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7b3553c3-4af3-49ce-9cd9-b22198c1e7be-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqspzw\" (UID: \"7b3553c3-4af3-49ce-9cd9-b22198c1e7be\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqspzw"
Oct 30 18:37:43.313053 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:43.313032    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fmv6d\" (UniqueName: \"kubernetes.io/projected/7b3553c3-4af3-49ce-9cd9-b22198c1e7be-kube-api-access-fmv6d\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqspzw\" (UID: \"7b3553c3-4af3-49ce-9cd9-b22198c1e7be\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqspzw"
Oct 30 18:37:43.313433 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:43.313414    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7b3553c3-4af3-49ce-9cd9-b22198c1e7be-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqspzw\" (UID: \"7b3553c3-4af3-49ce-9cd9-b22198c1e7be\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqspzw"
Oct 30 18:37:43.321165 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:43.320839    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"6541d93aaee335d6227a8778ab4e840bbe7f52-f98fbcf8-dockercfg-5qrsv\""
Oct 30 18:37:43.355820 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:43.355794    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7pmzc2"]
Oct 30 18:37:43.360221 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:43.360203    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7pmzc2"
Oct 30 18:37:43.375191 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:43.375167    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fmv6d\" (UniqueName: \"kubernetes.io/projected/7b3553c3-4af3-49ce-9cd9-b22198c1e7be-kube-api-access-fmv6d\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqspzw\" (UID: \"7b3553c3-4af3-49ce-9cd9-b22198c1e7be\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqspzw"
Oct 30 18:37:43.413696 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:43.413632    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t6v8t\" (UniqueName: \"kubernetes.io/projected/95475eb2-3d8f-469d-b1a1-e5cdf7567012-kube-api-access-t6v8t\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6mwnhv\" (UID: \"95475eb2-3d8f-469d-b1a1-e5cdf7567012\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6mwnhv"
Oct 30 18:37:43.413696 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:43.413668    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/95475eb2-3d8f-469d-b1a1-e5cdf7567012-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6mwnhv\" (UID: \"95475eb2-3d8f-469d-b1a1-e5cdf7567012\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6mwnhv"
Oct 30 18:37:43.413815 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:43.413702    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/12fe33f9-270e-41c7-b3ab-2ba8b0678a37-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847acr7cc\" (UID: \"12fe33f9-270e-41c7-b3ab-2ba8b0678a37\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847acr7cc"
Oct 30 18:37:43.413815 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:43.413749    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/95475eb2-3d8f-469d-b1a1-e5cdf7567012-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6mwnhv\" (UID: \"95475eb2-3d8f-469d-b1a1-e5cdf7567012\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6mwnhv"
Oct 30 18:37:43.413815 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:43.413776    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/12fe33f9-270e-41c7-b3ab-2ba8b0678a37-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847acr7cc\" (UID: \"12fe33f9-270e-41c7-b3ab-2ba8b0678a37\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847acr7cc"
Oct 30 18:37:43.413815 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:43.413804    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5vknr\" (UniqueName: \"kubernetes.io/projected/12fe33f9-270e-41c7-b3ab-2ba8b0678a37-kube-api-access-5vknr\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847acr7cc\" (UID: \"12fe33f9-270e-41c7-b3ab-2ba8b0678a37\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847acr7cc"
Oct 30 18:37:43.414112 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:43.414077    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/95475eb2-3d8f-469d-b1a1-e5cdf7567012-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6mwnhv\" (UID: \"95475eb2-3d8f-469d-b1a1-e5cdf7567012\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6mwnhv"
Oct 30 18:37:43.414112 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:43.414102    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/95475eb2-3d8f-469d-b1a1-e5cdf7567012-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6mwnhv\" (UID: \"95475eb2-3d8f-469d-b1a1-e5cdf7567012\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6mwnhv"
Oct 30 18:37:43.414239 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:43.414111    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/12fe33f9-270e-41c7-b3ab-2ba8b0678a37-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847acr7cc\" (UID: \"12fe33f9-270e-41c7-b3ab-2ba8b0678a37\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847acr7cc"
Oct 30 18:37:43.414239 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:43.414121    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/12fe33f9-270e-41c7-b3ab-2ba8b0678a37-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847acr7cc\" (UID: \"12fe33f9-270e-41c7-b3ab-2ba8b0678a37\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847acr7cc"
Oct 30 18:37:43.418462 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:43.418433    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7pmzc2"]
Oct 30 18:37:43.421189 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:43.421168    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"da112bd609bff87f7bacd2ba941225d7cff15c-bd73b2fc-dockercfg-dx28q\""
Oct 30 18:37:43.476315 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:43.476272    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t6v8t\" (UniqueName: \"kubernetes.io/projected/95475eb2-3d8f-469d-b1a1-e5cdf7567012-kube-api-access-t6v8t\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6mwnhv\" (UID: \"95475eb2-3d8f-469d-b1a1-e5cdf7567012\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6mwnhv"
Oct 30 18:37:43.476488 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:43.476468    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5vknr\" (UniqueName: \"kubernetes.io/projected/12fe33f9-270e-41c7-b3ab-2ba8b0678a37-kube-api-access-5vknr\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847acr7cc\" (UID: \"12fe33f9-270e-41c7-b3ab-2ba8b0678a37\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847acr7cc"
Oct 30 18:37:43.482217 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:43.482195    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6mwnhv"
Oct 30 18:37:43.514705 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:43.514670    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6138d789-ca22-4a27-93df-95c27e6c4dc7-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7pmzc2\" (UID: \"6138d789-ca22-4a27-93df-95c27e6c4dc7\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7pmzc2"
Oct 30 18:37:43.514868 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:43.514725    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rpmpq\" (UniqueName: \"kubernetes.io/projected/6138d789-ca22-4a27-93df-95c27e6c4dc7-kube-api-access-rpmpq\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7pmzc2\" (UID: \"6138d789-ca22-4a27-93df-95c27e6c4dc7\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7pmzc2"
Oct 30 18:37:43.514868 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:43.514839    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6138d789-ca22-4a27-93df-95c27e6c4dc7-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7pmzc2\" (UID: \"6138d789-ca22-4a27-93df-95c27e6c4dc7\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7pmzc2"
Oct 30 18:37:43.564108 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:43.564082    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847acr7cc"
Oct 30 18:37:43.615749 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:43.615695    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6138d789-ca22-4a27-93df-95c27e6c4dc7-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7pmzc2\" (UID: \"6138d789-ca22-4a27-93df-95c27e6c4dc7\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7pmzc2"
Oct 30 18:37:43.615876 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:43.615779    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rpmpq\" (UniqueName: \"kubernetes.io/projected/6138d789-ca22-4a27-93df-95c27e6c4dc7-kube-api-access-rpmpq\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7pmzc2\" (UID: \"6138d789-ca22-4a27-93df-95c27e6c4dc7\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7pmzc2"
Oct 30 18:37:43.616079 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:43.616055    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6138d789-ca22-4a27-93df-95c27e6c4dc7-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7pmzc2\" (UID: \"6138d789-ca22-4a27-93df-95c27e6c4dc7\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7pmzc2"
Oct 30 18:37:43.616538 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:43.616499    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6138d789-ca22-4a27-93df-95c27e6c4dc7-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7pmzc2\" (UID: \"6138d789-ca22-4a27-93df-95c27e6c4dc7\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7pmzc2"
Oct 30 18:37:43.616712 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:43.616510    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6138d789-ca22-4a27-93df-95c27e6c4dc7-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7pmzc2\" (UID: \"6138d789-ca22-4a27-93df-95c27e6c4dc7\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7pmzc2"
Oct 30 18:37:43.674773 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:43.674705    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqspzw"
Oct 30 18:37:43.679075 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:43.679056    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rpmpq\" (UniqueName: \"kubernetes.io/projected/6138d789-ca22-4a27-93df-95c27e6c4dc7-kube-api-access-rpmpq\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7pmzc2\" (UID: \"6138d789-ca22-4a27-93df-95c27e6c4dc7\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7pmzc2"
Oct 30 18:37:43.691255 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:43.691205    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c6w88m"]
Oct 30 18:37:43.693733 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:37:43.693706    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf03a161_80e8_4191_b562_661a7117035a.slice/crio-909e8d2686a0a7f7db5cc39631faa3c18b0d5d67767c3c72bf790488f639f33f WatchSource:0}: Error finding container 909e8d2686a0a7f7db5cc39631faa3c18b0d5d67767c3c72bf790488f639f33f: Status 404 returned error can't find the container with id 909e8d2686a0a7f7db5cc39631faa3c18b0d5d67767c3c72bf790488f639f33f
Oct 30 18:37:43.805355 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:43.805321    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c6w88m" event={"ID":"bf03a161-80e8-4191-b562-661a7117035a","Type":"ContainerStarted","Data":"909e8d2686a0a7f7db5cc39631faa3c18b0d5d67767c3c72bf790488f639f33f"}
Oct 30 18:37:43.855112 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:43.855091    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6mwnhv"]
Oct 30 18:37:43.857398 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:37:43.857374    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95475eb2_3d8f_469d_b1a1_e5cdf7567012.slice/crio-38186fc1f7f9188ad813f86c0f04e219aa9daf600592caa81600c36b549a8ab0 WatchSource:0}: Error finding container 38186fc1f7f9188ad813f86c0f04e219aa9daf600592caa81600c36b549a8ab0: Status 404 returned error can't find the container with id 38186fc1f7f9188ad813f86c0f04e219aa9daf600592caa81600c36b549a8ab0
Oct 30 18:37:43.936494 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:43.936457    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847acr7cc"]
Oct 30 18:37:43.938477 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:37:43.938448    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12fe33f9_270e_41c7_b3ab_2ba8b0678a37.slice/crio-8c20676f23ab41ae16f0132b86168818327afcce473c87e3027d5c1486e51468 WatchSource:0}: Error finding container 8c20676f23ab41ae16f0132b86168818327afcce473c87e3027d5c1486e51468: Status 404 returned error can't find the container with id 8c20676f23ab41ae16f0132b86168818327afcce473c87e3027d5c1486e51468
Oct 30 18:37:43.975615 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:43.975592    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7pmzc2"
Oct 30 18:37:44.048239 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:44.048149    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqspzw"]
Oct 30 18:37:44.050404 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:37:44.050379    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b3553c3_4af3_49ce_9cd9_b22198c1e7be.slice/crio-eac659f6fbb833d4f6cab8c06e361650fa3acfc118bf6126ce4824de6cccc4d1 WatchSource:0}: Error finding container eac659f6fbb833d4f6cab8c06e361650fa3acfc118bf6126ce4824de6cccc4d1: Status 404 returned error can't find the container with id eac659f6fbb833d4f6cab8c06e361650fa3acfc118bf6126ce4824de6cccc4d1
Oct 30 18:37:44.347307 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:44.347263    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7pmzc2"]
Oct 30 18:37:44.350614 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:37:44.350456    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6138d789_ca22_4a27_93df_95c27e6c4dc7.slice/crio-585419b236afdbccda06c8c7eb66b128d3a777bc6981c2eeef020403f783dc51 WatchSource:0}: Error finding container 585419b236afdbccda06c8c7eb66b128d3a777bc6981c2eeef020403f783dc51: Status 404 returned error can't find the container with id 585419b236afdbccda06c8c7eb66b128d3a777bc6981c2eeef020403f783dc51
Oct 30 18:37:44.811592 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:44.811505    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7pmzc2" event={"ID":"6138d789-ca22-4a27-93df-95c27e6c4dc7","Type":"ContainerStarted","Data":"585419b236afdbccda06c8c7eb66b128d3a777bc6981c2eeef020403f783dc51"}
Oct 30 18:37:44.813173 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:44.813142    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6mwnhv" event={"ID":"95475eb2-3d8f-469d-b1a1-e5cdf7567012","Type":"ContainerStarted","Data":"38186fc1f7f9188ad813f86c0f04e219aa9daf600592caa81600c36b549a8ab0"}
Oct 30 18:37:44.814845 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:44.814794    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqspzw" event={"ID":"7b3553c3-4af3-49ce-9cd9-b22198c1e7be","Type":"ContainerStarted","Data":"eac659f6fbb833d4f6cab8c06e361650fa3acfc118bf6126ce4824de6cccc4d1"}
Oct 30 18:37:44.816452 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:44.816427    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847acr7cc" event={"ID":"12fe33f9-270e-41c7-b3ab-2ba8b0678a37","Type":"ContainerStarted","Data":"8c20676f23ab41ae16f0132b86168818327afcce473c87e3027d5c1486e51468"}
Oct 30 18:37:45.425258 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:45.425217    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644thm68"]
Oct 30 18:37:45.429524 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:45.429486    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644thm68"
Oct 30 18:37:45.533558 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:45.533522    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ec4e00bf-5598-40ca-ac8b-874419ab0f33-bundle\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644thm68\" (UID: \"ec4e00bf-5598-40ca-ac8b-874419ab0f33\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644thm68"
Oct 30 18:37:45.533722 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:45.533576    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6vxld\" (UniqueName: \"kubernetes.io/projected/ec4e00bf-5598-40ca-ac8b-874419ab0f33-kube-api-access-6vxld\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644thm68\" (UID: \"ec4e00bf-5598-40ca-ac8b-874419ab0f33\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644thm68"
Oct 30 18:37:45.533722 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:45.533645    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ec4e00bf-5598-40ca-ac8b-874419ab0f33-util\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644thm68\" (UID: \"ec4e00bf-5598-40ca-ac8b-874419ab0f33\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644thm68"
Oct 30 18:37:45.535472 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:45.535445    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"c4959aeb960c8e20c88ac2de924a35b8221494-145dcd5a-dockercfg-zk7vp\""
Oct 30 18:37:45.535865 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:45.535844    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644thm68"]
Oct 30 18:37:45.634793 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:45.634752    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ec4e00bf-5598-40ca-ac8b-874419ab0f33-bundle\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644thm68\" (UID: \"ec4e00bf-5598-40ca-ac8b-874419ab0f33\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644thm68"
Oct 30 18:37:45.634968 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:45.634809    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6vxld\" (UniqueName: \"kubernetes.io/projected/ec4e00bf-5598-40ca-ac8b-874419ab0f33-kube-api-access-6vxld\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644thm68\" (UID: \"ec4e00bf-5598-40ca-ac8b-874419ab0f33\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644thm68"
Oct 30 18:37:45.634968 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:45.634883    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ec4e00bf-5598-40ca-ac8b-874419ab0f33-util\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644thm68\" (UID: \"ec4e00bf-5598-40ca-ac8b-874419ab0f33\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644thm68"
Oct 30 18:37:45.635341 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:45.635321    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ec4e00bf-5598-40ca-ac8b-874419ab0f33-util\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644thm68\" (UID: \"ec4e00bf-5598-40ca-ac8b-874419ab0f33\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644thm68"
Oct 30 18:37:45.635580 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:45.635563    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ec4e00bf-5598-40ca-ac8b-874419ab0f33-bundle\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644thm68\" (UID: \"ec4e00bf-5598-40ca-ac8b-874419ab0f33\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644thm68"
Oct 30 18:37:45.709635 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:45.709568    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6vxld\" (UniqueName: \"kubernetes.io/projected/ec4e00bf-5598-40ca-ac8b-874419ab0f33-kube-api-access-6vxld\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644thm68\" (UID: \"ec4e00bf-5598-40ca-ac8b-874419ab0f33\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644thm68"
Oct 30 18:37:45.750863 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:45.750829    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644thm68"
Oct 30 18:37:46.169374 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:46.169187    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644thm68"]
Oct 30 18:37:46.178118 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:37:46.178079    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec4e00bf_5598_40ca_ac8b_874419ab0f33.slice/crio-4233672615f7f235b468a37c972e2e6495c7e226e83fb65c5435afd082030dc0 WatchSource:0}: Error finding container 4233672615f7f235b468a37c972e2e6495c7e226e83fb65c5435afd082030dc0: Status 404 returned error can't find the container with id 4233672615f7f235b468a37c972e2e6495c7e226e83fb65c5435afd082030dc0
Oct 30 18:37:46.820038 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:46.818106    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71r5qv6"]
Oct 30 18:37:46.824398 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:46.823302    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71r5qv6"
Oct 30 18:37:46.835039 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:46.834912    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644thm68" event={"ID":"ec4e00bf-5598-40ca-ac8b-874419ab0f33","Type":"ContainerStarted","Data":"4233672615f7f235b468a37c972e2e6495c7e226e83fb65c5435afd082030dc0"}
Oct 30 18:37:46.931587 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:46.931523    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"e644421527c169d876fdb59235c927986d6bd2-9d2815fc-dockercfg-hkslx\""
Oct 30 18:37:46.931796 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:46.931772    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71r5qv6"]
Oct 30 18:37:46.946098 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:46.946014    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3861ca20-0760-4301-a912-95920a9601ec-bundle\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71r5qv6\" (UID: \"3861ca20-0760-4301-a912-95920a9601ec\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71r5qv6"
Oct 30 18:37:46.946260 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:46.946193    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wthn2\" (UniqueName: \"kubernetes.io/projected/3861ca20-0760-4301-a912-95920a9601ec-kube-api-access-wthn2\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71r5qv6\" (UID: \"3861ca20-0760-4301-a912-95920a9601ec\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71r5qv6"
Oct 30 18:37:46.946260 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:46.946250    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3861ca20-0760-4301-a912-95920a9601ec-util\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71r5qv6\" (UID: \"3861ca20-0760-4301-a912-95920a9601ec\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71r5qv6"
Oct 30 18:37:47.047562 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:47.047523    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3861ca20-0760-4301-a912-95920a9601ec-util\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71r5qv6\" (UID: \"3861ca20-0760-4301-a912-95920a9601ec\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71r5qv6"
Oct 30 18:37:47.047754 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:47.047595    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3861ca20-0760-4301-a912-95920a9601ec-bundle\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71r5qv6\" (UID: \"3861ca20-0760-4301-a912-95920a9601ec\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71r5qv6"
Oct 30 18:37:47.047754 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:47.047720    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wthn2\" (UniqueName: \"kubernetes.io/projected/3861ca20-0760-4301-a912-95920a9601ec-kube-api-access-wthn2\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71r5qv6\" (UID: \"3861ca20-0760-4301-a912-95920a9601ec\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71r5qv6"
Oct 30 18:37:47.048456 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:47.048430    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3861ca20-0760-4301-a912-95920a9601ec-util\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71r5qv6\" (UID: \"3861ca20-0760-4301-a912-95920a9601ec\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71r5qv6"
Oct 30 18:37:47.048663 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:47.048648    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3861ca20-0760-4301-a912-95920a9601ec-bundle\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71r5qv6\" (UID: \"3861ca20-0760-4301-a912-95920a9601ec\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71r5qv6"
Oct 30 18:37:47.121223 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:47.121167    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wthn2\" (UniqueName: \"kubernetes.io/projected/3861ca20-0760-4301-a912-95920a9601ec-kube-api-access-wthn2\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71r5qv6\" (UID: \"3861ca20-0760-4301-a912-95920a9601ec\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71r5qv6"
Oct 30 18:37:47.145194 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:47.144957    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71r5qv6"
Oct 30 18:37:47.550397 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:47.550327    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71r5qv6"]
Oct 30 18:37:47.560729 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:37:47.560695    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3861ca20_0760_4301_a912_95920a9601ec.slice/crio-54ef21074d7aa07007d0b9632c03ba5bf6896bfcd945421ccc663f3d2cd7aa6d WatchSource:0}: Error finding container 54ef21074d7aa07007d0b9632c03ba5bf6896bfcd945421ccc663f3d2cd7aa6d: Status 404 returned error can't find the container with id 54ef21074d7aa07007d0b9632c03ba5bf6896bfcd945421ccc663f3d2cd7aa6d
Oct 30 18:37:47.844089 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:47.843993    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71r5qv6" event={"ID":"3861ca20-0760-4301-a912-95920a9601ec","Type":"ContainerStarted","Data":"54ef21074d7aa07007d0b9632c03ba5bf6896bfcd945421ccc663f3d2cd7aa6d"}
Oct 30 18:37:48.135450 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:48.133772    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-66db998887-mhzc5"]
Oct 30 18:37:48.139004 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:48.138981    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-mhzc5"
Oct 30 18:37:48.193619 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:48.192924    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Oct 30 18:37:48.209955 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:48.209884    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-s874n\""
Oct 30 18:37:48.245883 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:48.245729    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-66db998887-mhzc5"]
Oct 30 18:37:48.260837 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:48.260303    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bf0f21cf-8198-4fe4-8983-db083c82b34a-istio-podinfo\") pod \"maas-default-gateway-openshift-default-66db998887-mhzc5\" (UID: \"bf0f21cf-8198-4fe4-8983-db083c82b34a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-mhzc5"
Oct 30 18:37:48.260837 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:48.260343    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bf0f21cf-8198-4fe4-8983-db083c82b34a-istio-token\") pod \"maas-default-gateway-openshift-default-66db998887-mhzc5\" (UID: \"bf0f21cf-8198-4fe4-8983-db083c82b34a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-mhzc5"
Oct 30 18:37:48.260837 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:48.260380    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bf0f21cf-8198-4fe4-8983-db083c82b34a-credential-socket\") pod \"maas-default-gateway-openshift-default-66db998887-mhzc5\" (UID: \"bf0f21cf-8198-4fe4-8983-db083c82b34a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-mhzc5"
Oct 30 18:37:48.260837 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:48.260407    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bf0f21cf-8198-4fe4-8983-db083c82b34a-istio-envoy\") pod \"maas-default-gateway-openshift-default-66db998887-mhzc5\" (UID: \"bf0f21cf-8198-4fe4-8983-db083c82b34a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-mhzc5"
Oct 30 18:37:48.260837 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:48.260446    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bf0f21cf-8198-4fe4-8983-db083c82b34a-workload-certs\") pod \"maas-default-gateway-openshift-default-66db998887-mhzc5\" (UID: \"bf0f21cf-8198-4fe4-8983-db083c82b34a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-mhzc5"
Oct 30 18:37:48.260837 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:48.260479    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bf0f21cf-8198-4fe4-8983-db083c82b34a-workload-socket\") pod \"maas-default-gateway-openshift-default-66db998887-mhzc5\" (UID: \"bf0f21cf-8198-4fe4-8983-db083c82b34a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-mhzc5"
Oct 30 18:37:48.260837 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:48.260514    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bf0f21cf-8198-4fe4-8983-db083c82b34a-istio-data\") pod \"maas-default-gateway-openshift-default-66db998887-mhzc5\" (UID: \"bf0f21cf-8198-4fe4-8983-db083c82b34a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-mhzc5"
Oct 30 18:37:48.260837 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:48.260540    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jszwr\" (UniqueName: \"kubernetes.io/projected/bf0f21cf-8198-4fe4-8983-db083c82b34a-kube-api-access-jszwr\") pod \"maas-default-gateway-openshift-default-66db998887-mhzc5\" (UID: \"bf0f21cf-8198-4fe4-8983-db083c82b34a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-mhzc5"
Oct 30 18:37:48.260837 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:48.260640    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bf0f21cf-8198-4fe4-8983-db083c82b34a-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-66db998887-mhzc5\" (UID: \"bf0f21cf-8198-4fe4-8983-db083c82b34a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-mhzc5"
Oct 30 18:37:48.361711 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:48.361676    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bf0f21cf-8198-4fe4-8983-db083c82b34a-workload-socket\") pod \"maas-default-gateway-openshift-default-66db998887-mhzc5\" (UID: \"bf0f21cf-8198-4fe4-8983-db083c82b34a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-mhzc5"
Oct 30 18:37:48.361887 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:48.361732    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bf0f21cf-8198-4fe4-8983-db083c82b34a-istio-data\") pod \"maas-default-gateway-openshift-default-66db998887-mhzc5\" (UID: \"bf0f21cf-8198-4fe4-8983-db083c82b34a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-mhzc5"
Oct 30 18:37:48.361887 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:48.361761    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jszwr\" (UniqueName: \"kubernetes.io/projected/bf0f21cf-8198-4fe4-8983-db083c82b34a-kube-api-access-jszwr\") pod \"maas-default-gateway-openshift-default-66db998887-mhzc5\" (UID: \"bf0f21cf-8198-4fe4-8983-db083c82b34a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-mhzc5"
Oct 30 18:37:48.361887 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:48.361850    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bf0f21cf-8198-4fe4-8983-db083c82b34a-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-66db998887-mhzc5\" (UID: \"bf0f21cf-8198-4fe4-8983-db083c82b34a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-mhzc5"
Oct 30 18:37:48.361887 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:48.361880    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bf0f21cf-8198-4fe4-8983-db083c82b34a-istio-podinfo\") pod \"maas-default-gateway-openshift-default-66db998887-mhzc5\" (UID: \"bf0f21cf-8198-4fe4-8983-db083c82b34a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-mhzc5"
Oct 30 18:37:48.362143 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:48.361902    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bf0f21cf-8198-4fe4-8983-db083c82b34a-istio-token\") pod \"maas-default-gateway-openshift-default-66db998887-mhzc5\" (UID: \"bf0f21cf-8198-4fe4-8983-db083c82b34a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-mhzc5"
Oct 30 18:37:48.362143 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:48.361934    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bf0f21cf-8198-4fe4-8983-db083c82b34a-credential-socket\") pod \"maas-default-gateway-openshift-default-66db998887-mhzc5\" (UID: \"bf0f21cf-8198-4fe4-8983-db083c82b34a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-mhzc5"
Oct 30 18:37:48.362143 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:48.361970    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bf0f21cf-8198-4fe4-8983-db083c82b34a-istio-envoy\") pod \"maas-default-gateway-openshift-default-66db998887-mhzc5\" (UID: \"bf0f21cf-8198-4fe4-8983-db083c82b34a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-mhzc5"
Oct 30 18:37:48.362143 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:48.362016    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bf0f21cf-8198-4fe4-8983-db083c82b34a-workload-certs\") pod \"maas-default-gateway-openshift-default-66db998887-mhzc5\" (UID: \"bf0f21cf-8198-4fe4-8983-db083c82b34a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-mhzc5"
Oct 30 18:37:48.362510 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:48.362487    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bf0f21cf-8198-4fe4-8983-db083c82b34a-workload-certs\") pod \"maas-default-gateway-openshift-default-66db998887-mhzc5\" (UID: \"bf0f21cf-8198-4fe4-8983-db083c82b34a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-mhzc5"
Oct 30 18:37:48.363663 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:48.363636    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bf0f21cf-8198-4fe4-8983-db083c82b34a-workload-socket\") pod \"maas-default-gateway-openshift-default-66db998887-mhzc5\" (UID: \"bf0f21cf-8198-4fe4-8983-db083c82b34a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-mhzc5"
Oct 30 18:37:48.363956 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:48.363935    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bf0f21cf-8198-4fe4-8983-db083c82b34a-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-66db998887-mhzc5\" (UID: \"bf0f21cf-8198-4fe4-8983-db083c82b34a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-mhzc5"
Oct 30 18:37:48.364046 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:48.363950    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bf0f21cf-8198-4fe4-8983-db083c82b34a-credential-socket\") pod \"maas-default-gateway-openshift-default-66db998887-mhzc5\" (UID: \"bf0f21cf-8198-4fe4-8983-db083c82b34a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-mhzc5"
Oct 30 18:37:48.364245 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:48.364158    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bf0f21cf-8198-4fe4-8983-db083c82b34a-istio-data\") pod \"maas-default-gateway-openshift-default-66db998887-mhzc5\" (UID: \"bf0f21cf-8198-4fe4-8983-db083c82b34a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-mhzc5"
Oct 30 18:37:48.371987 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:48.371925    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bf0f21cf-8198-4fe4-8983-db083c82b34a-istio-podinfo\") pod \"maas-default-gateway-openshift-default-66db998887-mhzc5\" (UID: \"bf0f21cf-8198-4fe4-8983-db083c82b34a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-mhzc5"
Oct 30 18:37:48.373755 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:48.373692    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bf0f21cf-8198-4fe4-8983-db083c82b34a-istio-envoy\") pod \"maas-default-gateway-openshift-default-66db998887-mhzc5\" (UID: \"bf0f21cf-8198-4fe4-8983-db083c82b34a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-mhzc5"
Oct 30 18:37:48.429954 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:48.429864    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jszwr\" (UniqueName: \"kubernetes.io/projected/bf0f21cf-8198-4fe4-8983-db083c82b34a-kube-api-access-jszwr\") pod \"maas-default-gateway-openshift-default-66db998887-mhzc5\" (UID: \"bf0f21cf-8198-4fe4-8983-db083c82b34a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-mhzc5"
Oct 30 18:37:48.436634 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:48.436510    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bf0f21cf-8198-4fe4-8983-db083c82b34a-istio-token\") pod \"maas-default-gateway-openshift-default-66db998887-mhzc5\" (UID: \"bf0f21cf-8198-4fe4-8983-db083c82b34a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-mhzc5"
Oct 30 18:37:48.456331 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:48.455720    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-ai-inference-6d5bbf59c5-wjwcj"]
Oct 30 18:37:48.460953 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:48.460654    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-mhzc5"
Oct 30 18:37:48.465506 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:48.464958    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-6d5bbf59c5-wjwcj"
Oct 30 18:37:48.521665 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:48.521579    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-ai-inference-6d5bbf59c5-wjwcj"]
Oct 30 18:37:48.529549 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:48.529491    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"openshift-ai-inference-openshift-ai-inference-dockercfg-kzf6l\""
Oct 30 18:37:48.564231 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:48.564192    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f434df49-9016-4649-a140-82f23de7cdfc-workload-certs\") pod \"openshift-ai-inference-openshift-ai-inference-6d5bbf59c5-wjwcj\" (UID: \"f434df49-9016-4649-a140-82f23de7cdfc\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-6d5bbf59c5-wjwcj"
Oct 30 18:37:48.564711 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:48.564257    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f434df49-9016-4649-a140-82f23de7cdfc-workload-socket\") pod \"openshift-ai-inference-openshift-ai-inference-6d5bbf59c5-wjwcj\" (UID: \"f434df49-9016-4649-a140-82f23de7cdfc\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-6d5bbf59c5-wjwcj"
Oct 30 18:37:48.564711 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:48.564304    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f434df49-9016-4649-a140-82f23de7cdfc-istio-data\") pod \"openshift-ai-inference-openshift-ai-inference-6d5bbf59c5-wjwcj\" (UID: \"f434df49-9016-4649-a140-82f23de7cdfc\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-6d5bbf59c5-wjwcj"
Oct 30 18:37:48.564711 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:48.564481    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f434df49-9016-4649-a140-82f23de7cdfc-istio-token\") pod \"openshift-ai-inference-openshift-ai-inference-6d5bbf59c5-wjwcj\" (UID: \"f434df49-9016-4649-a140-82f23de7cdfc\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-6d5bbf59c5-wjwcj"
Oct 30 18:37:48.564711 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:48.564504    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f434df49-9016-4649-a140-82f23de7cdfc-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-ai-inference-6d5bbf59c5-wjwcj\" (UID: \"f434df49-9016-4649-a140-82f23de7cdfc\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-6d5bbf59c5-wjwcj"
Oct 30 18:37:48.564711 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:48.564641    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tspnk\" (UniqueName: \"kubernetes.io/projected/f434df49-9016-4649-a140-82f23de7cdfc-kube-api-access-tspnk\") pod \"openshift-ai-inference-openshift-ai-inference-6d5bbf59c5-wjwcj\" (UID: \"f434df49-9016-4649-a140-82f23de7cdfc\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-6d5bbf59c5-wjwcj"
Oct 30 18:37:48.564711 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:48.564688    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f434df49-9016-4649-a140-82f23de7cdfc-istio-podinfo\") pod \"openshift-ai-inference-openshift-ai-inference-6d5bbf59c5-wjwcj\" (UID: \"f434df49-9016-4649-a140-82f23de7cdfc\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-6d5bbf59c5-wjwcj"
Oct 30 18:37:48.565113 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:48.564747    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f434df49-9016-4649-a140-82f23de7cdfc-credential-socket\") pod \"openshift-ai-inference-openshift-ai-inference-6d5bbf59c5-wjwcj\" (UID: \"f434df49-9016-4649-a140-82f23de7cdfc\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-6d5bbf59c5-wjwcj"
Oct 30 18:37:48.565113 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:48.564773    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f434df49-9016-4649-a140-82f23de7cdfc-istio-envoy\") pod \"openshift-ai-inference-openshift-ai-inference-6d5bbf59c5-wjwcj\" (UID: \"f434df49-9016-4649-a140-82f23de7cdfc\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-6d5bbf59c5-wjwcj"
Oct 30 18:37:48.665543 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:48.665453    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tspnk\" (UniqueName: \"kubernetes.io/projected/f434df49-9016-4649-a140-82f23de7cdfc-kube-api-access-tspnk\") pod \"openshift-ai-inference-openshift-ai-inference-6d5bbf59c5-wjwcj\" (UID: \"f434df49-9016-4649-a140-82f23de7cdfc\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-6d5bbf59c5-wjwcj"
Oct 30 18:37:48.665543 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:48.665506    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f434df49-9016-4649-a140-82f23de7cdfc-istio-podinfo\") pod \"openshift-ai-inference-openshift-ai-inference-6d5bbf59c5-wjwcj\" (UID: \"f434df49-9016-4649-a140-82f23de7cdfc\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-6d5bbf59c5-wjwcj"
Oct 30 18:37:48.665771 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:48.665567    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f434df49-9016-4649-a140-82f23de7cdfc-credential-socket\") pod \"openshift-ai-inference-openshift-ai-inference-6d5bbf59c5-wjwcj\" (UID: \"f434df49-9016-4649-a140-82f23de7cdfc\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-6d5bbf59c5-wjwcj"
Oct 30 18:37:48.665771 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:48.665591    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f434df49-9016-4649-a140-82f23de7cdfc-istio-envoy\") pod \"openshift-ai-inference-openshift-ai-inference-6d5bbf59c5-wjwcj\" (UID: \"f434df49-9016-4649-a140-82f23de7cdfc\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-6d5bbf59c5-wjwcj"
Oct 30 18:37:48.665771 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:48.665650    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f434df49-9016-4649-a140-82f23de7cdfc-workload-certs\") pod \"openshift-ai-inference-openshift-ai-inference-6d5bbf59c5-wjwcj\" (UID: \"f434df49-9016-4649-a140-82f23de7cdfc\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-6d5bbf59c5-wjwcj"
Oct 30 18:37:48.665771 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:48.665708    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f434df49-9016-4649-a140-82f23de7cdfc-workload-socket\") pod \"openshift-ai-inference-openshift-ai-inference-6d5bbf59c5-wjwcj\" (UID: \"f434df49-9016-4649-a140-82f23de7cdfc\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-6d5bbf59c5-wjwcj"
Oct 30 18:37:48.665771 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:48.665738    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f434df49-9016-4649-a140-82f23de7cdfc-istio-data\") pod \"openshift-ai-inference-openshift-ai-inference-6d5bbf59c5-wjwcj\" (UID: \"f434df49-9016-4649-a140-82f23de7cdfc\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-6d5bbf59c5-wjwcj"
Oct 30 18:37:48.666024 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:48.665781    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f434df49-9016-4649-a140-82f23de7cdfc-istio-token\") pod \"openshift-ai-inference-openshift-ai-inference-6d5bbf59c5-wjwcj\" (UID: \"f434df49-9016-4649-a140-82f23de7cdfc\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-6d5bbf59c5-wjwcj"
Oct 30 18:37:48.666024 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:48.665806    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f434df49-9016-4649-a140-82f23de7cdfc-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-ai-inference-6d5bbf59c5-wjwcj\" (UID: \"f434df49-9016-4649-a140-82f23de7cdfc\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-6d5bbf59c5-wjwcj"
Oct 30 18:37:48.666752 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:48.666697    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f434df49-9016-4649-a140-82f23de7cdfc-workload-socket\") pod \"openshift-ai-inference-openshift-ai-inference-6d5bbf59c5-wjwcj\" (UID: \"f434df49-9016-4649-a140-82f23de7cdfc\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-6d5bbf59c5-wjwcj"
Oct 30 18:37:48.667048 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:48.666982    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f434df49-9016-4649-a140-82f23de7cdfc-istio-data\") pod \"openshift-ai-inference-openshift-ai-inference-6d5bbf59c5-wjwcj\" (UID: \"f434df49-9016-4649-a140-82f23de7cdfc\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-6d5bbf59c5-wjwcj"
Oct 30 18:37:48.667048 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:48.667023    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f434df49-9016-4649-a140-82f23de7cdfc-workload-certs\") pod \"openshift-ai-inference-openshift-ai-inference-6d5bbf59c5-wjwcj\" (UID: \"f434df49-9016-4649-a140-82f23de7cdfc\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-6d5bbf59c5-wjwcj"
Oct 30 18:37:48.667215 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:48.667049    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f434df49-9016-4649-a140-82f23de7cdfc-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-ai-inference-6d5bbf59c5-wjwcj\" (UID: \"f434df49-9016-4649-a140-82f23de7cdfc\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-6d5bbf59c5-wjwcj"
Oct 30 18:37:48.667623 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:48.667507    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f434df49-9016-4649-a140-82f23de7cdfc-credential-socket\") pod \"openshift-ai-inference-openshift-ai-inference-6d5bbf59c5-wjwcj\" (UID: \"f434df49-9016-4649-a140-82f23de7cdfc\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-6d5bbf59c5-wjwcj"
Oct 30 18:37:48.678710 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:48.678668    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f434df49-9016-4649-a140-82f23de7cdfc-istio-envoy\") pod \"openshift-ai-inference-openshift-ai-inference-6d5bbf59c5-wjwcj\" (UID: \"f434df49-9016-4649-a140-82f23de7cdfc\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-6d5bbf59c5-wjwcj"
Oct 30 18:37:48.678932 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:48.678902    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f434df49-9016-4649-a140-82f23de7cdfc-istio-podinfo\") pod \"openshift-ai-inference-openshift-ai-inference-6d5bbf59c5-wjwcj\" (UID: \"f434df49-9016-4649-a140-82f23de7cdfc\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-6d5bbf59c5-wjwcj"
Oct 30 18:37:48.733617 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:48.733554    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f434df49-9016-4649-a140-82f23de7cdfc-istio-token\") pod \"openshift-ai-inference-openshift-ai-inference-6d5bbf59c5-wjwcj\" (UID: \"f434df49-9016-4649-a140-82f23de7cdfc\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-6d5bbf59c5-wjwcj"
Oct 30 18:37:48.738125 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:48.738098    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tspnk\" (UniqueName: \"kubernetes.io/projected/f434df49-9016-4649-a140-82f23de7cdfc-kube-api-access-tspnk\") pod \"openshift-ai-inference-openshift-ai-inference-6d5bbf59c5-wjwcj\" (UID: \"f434df49-9016-4649-a140-82f23de7cdfc\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-6d5bbf59c5-wjwcj"
Oct 30 18:37:48.791866 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:48.791823    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-6d5bbf59c5-wjwcj"
Oct 30 18:37:48.987271 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:48.987184    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-66db998887-mhzc5"]
Oct 30 18:37:49.001219 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:37:49.001186    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf0f21cf_8198_4fe4_8983_db083c82b34a.slice/crio-bfc080a2db45bcab96ef48a9e600c9e467611cce07e8128b8ab5ea7619fd2b43 WatchSource:0}: Error finding container bfc080a2db45bcab96ef48a9e600c9e467611cce07e8128b8ab5ea7619fd2b43: Status 404 returned error can't find the container with id bfc080a2db45bcab96ef48a9e600c9e467611cce07e8128b8ab5ea7619fd2b43
Oct 30 18:37:49.229506 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:49.229472    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-ai-inference-6d5bbf59c5-wjwcj"]
Oct 30 18:37:49.237135 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:37:49.237104    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf434df49_9016_4649_a140_82f23de7cdfc.slice/crio-d0bb8b37cdddf0b5a3ba5f071717bce2ead007d2308c2079ade86c8164c6c8a4 WatchSource:0}: Error finding container d0bb8b37cdddf0b5a3ba5f071717bce2ead007d2308c2079ade86c8164c6c8a4: Status 404 returned error can't find the container with id d0bb8b37cdddf0b5a3ba5f071717bce2ead007d2308c2079ade86c8164c6c8a4
Oct 30 18:37:49.860423 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:49.860373    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-6d5bbf59c5-wjwcj" event={"ID":"f434df49-9016-4649-a140-82f23de7cdfc","Type":"ContainerStarted","Data":"d0bb8b37cdddf0b5a3ba5f071717bce2ead007d2308c2079ade86c8164c6c8a4"}
Oct 30 18:37:49.863168 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:49.863077    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-mhzc5" event={"ID":"bf0f21cf-8198-4fe4-8983-db083c82b34a","Type":"ContainerStarted","Data":"bfc080a2db45bcab96ef48a9e600c9e467611cce07e8128b8ab5ea7619fd2b43"}
Oct 30 18:37:59.836580 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:59.836551    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:37:59.837053 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:59.836619    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:37:59.837053 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:59.836658    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:37:59.837172 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:59.837076    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:37:59.837172 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:59.837132    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:37:59.837172 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:37:59.837161    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:38:00.952031 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:00.951993    4888 generic.go:356] "Generic (PLEG): container finished" podID="bf03a161-80e8-4191-b562-661a7117035a" containerID="ad074a55036d08da04b5642b924257393390cb22857d283e5096baa4f46ce9e6" exitCode=0
Oct 30 18:38:00.952533 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:00.952318    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c6w88m" event={"ID":"bf03a161-80e8-4191-b562-661a7117035a","Type":"ContainerDied","Data":"ad074a55036d08da04b5642b924257393390cb22857d283e5096baa4f46ce9e6"}
Oct 30 18:38:00.954141 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:00.954116    4888 generic.go:356] "Generic (PLEG): container finished" podID="7b3553c3-4af3-49ce-9cd9-b22198c1e7be" containerID="9e0188aca4e2a2db39f77c11f2fb0e382be6c387e22363a361b56bd371d3f7f5" exitCode=0
Oct 30 18:38:00.955345 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:00.954188    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqspzw" event={"ID":"7b3553c3-4af3-49ce-9cd9-b22198c1e7be","Type":"ContainerDied","Data":"9e0188aca4e2a2db39f77c11f2fb0e382be6c387e22363a361b56bd371d3f7f5"}
Oct 30 18:38:00.956245 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:00.956129    4888 generic.go:356] "Generic (PLEG): container finished" podID="12fe33f9-270e-41c7-b3ab-2ba8b0678a37" containerID="1f01bb0436a88c1ee34cb897610cb7b1ba09b1e284e158e2a3c05d3d893bec0f" exitCode=0
Oct 30 18:38:00.956245 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:00.956168    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847acr7cc" event={"ID":"12fe33f9-270e-41c7-b3ab-2ba8b0678a37","Type":"ContainerDied","Data":"1f01bb0436a88c1ee34cb897610cb7b1ba09b1e284e158e2a3c05d3d893bec0f"}
Oct 30 18:38:00.958133 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:00.958112    4888 generic.go:356] "Generic (PLEG): container finished" podID="6138d789-ca22-4a27-93df-95c27e6c4dc7" containerID="e9e5353460210c2792d6caf15868898fe53e91879ca02017a0a88c26457e02ca" exitCode=0
Oct 30 18:38:00.958234 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:00.958193    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7pmzc2" event={"ID":"6138d789-ca22-4a27-93df-95c27e6c4dc7","Type":"ContainerDied","Data":"e9e5353460210c2792d6caf15868898fe53e91879ca02017a0a88c26457e02ca"}
Oct 30 18:38:00.960354 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:00.960165    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-6d5bbf59c5-wjwcj" event={"ID":"f434df49-9016-4649-a140-82f23de7cdfc","Type":"ContainerStarted","Data":"b697b9a9f32e06931c17f2aafceb94573107e12e82956b02ba9a1ada14a9a080"}
Oct 30 18:38:00.962690 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:00.962665    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-mhzc5" event={"ID":"bf0f21cf-8198-4fe4-8983-db083c82b34a","Type":"ContainerStarted","Data":"bf28cd527b037444b211a9ce7cbb22d6e612330c035e394c25bba819000865be"}
Oct 30 18:38:00.964605 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:00.964506    4888 generic.go:356] "Generic (PLEG): container finished" podID="95475eb2-3d8f-469d-b1a1-e5cdf7567012" containerID="5acb1af8b6e69953fa85b858044714ee5d2938e635215198ec2deef96db214ce" exitCode=0
Oct 30 18:38:00.964605 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:00.964563    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6mwnhv" event={"ID":"95475eb2-3d8f-469d-b1a1-e5cdf7567012","Type":"ContainerDied","Data":"5acb1af8b6e69953fa85b858044714ee5d2938e635215198ec2deef96db214ce"}
Oct 30 18:38:01.203074 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:01.202962    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-mhzc5" podStartSLOduration=2.373229243 podStartE2EDuration="13.202947547s" totalImagesPullingTime="10.829718304s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:37:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:38:01.201847746 +0000 UTC m=+245.449253773" watchObservedRunningTime="2026-10-30 18:38:01.202947547 +0000 UTC m=+245.450353564"
Oct 30 18:38:01.325219 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:01.325174    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-6d5bbf59c5-wjwcj" podStartSLOduration=2.73726217 podStartE2EDuration="13.325160787s" totalImagesPullingTime="10.587898617s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:37:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:38:01.324369878 +0000 UTC m=+245.571775905" watchObservedRunningTime="2026-10-30 18:38:01.325160787 +0000 UTC m=+245.572566813"
Oct 30 18:38:01.461666 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:01.461588    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-mhzc5"
Oct 30 18:38:01.466830 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:01.466806    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-mhzc5"
Oct 30 18:38:01.792878 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:01.792810    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-6d5bbf59c5-wjwcj"
Oct 30 18:38:01.797480 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:01.797452    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-6d5bbf59c5-wjwcj"
Oct 30 18:38:01.970192 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:01.970158    4888 generic.go:356] "Generic (PLEG): container finished" podID="3861ca20-0760-4301-a912-95920a9601ec" containerID="a842eedbe8d1179e96dd3a5cf0dfd7eea5bf51f330f62642be1cad2809ba0e8f" exitCode=0
Oct 30 18:38:01.970566 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:01.970240    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71r5qv6" event={"ID":"3861ca20-0760-4301-a912-95920a9601ec","Type":"ContainerDied","Data":"a842eedbe8d1179e96dd3a5cf0dfd7eea5bf51f330f62642be1cad2809ba0e8f"}
Oct 30 18:38:01.971864 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:01.971843    4888 generic.go:356] "Generic (PLEG): container finished" podID="ec4e00bf-5598-40ca-ac8b-874419ab0f33" containerID="9816744a35a36cbb2be85f023af8feb14cbdf497ff61a5896bde014ce68963be" exitCode=0
Oct 30 18:38:01.972021 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:01.971929    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644thm68" event={"ID":"ec4e00bf-5598-40ca-ac8b-874419ab0f33","Type":"ContainerDied","Data":"9816744a35a36cbb2be85f023af8feb14cbdf497ff61a5896bde014ce68963be"}
Oct 30 18:38:01.972393 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:01.972371    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-mhzc5"
Oct 30 18:38:01.972499 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:01.972419    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-6d5bbf59c5-wjwcj"
Oct 30 18:38:01.973434 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:01.973413    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-6d5bbf59c5-wjwcj"
Oct 30 18:38:01.973535 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:01.973471    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-mhzc5"
Oct 30 18:38:02.978683 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:02.978654    4888 generic.go:356] "Generic (PLEG): container finished" podID="7b3553c3-4af3-49ce-9cd9-b22198c1e7be" containerID="0c48dbf9c2f1c8c84bafa68d26e660609737b09911e612a3cbdd98925ea9b443" exitCode=0
Oct 30 18:38:02.979129 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:02.978736    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqspzw" event={"ID":"7b3553c3-4af3-49ce-9cd9-b22198c1e7be","Type":"ContainerDied","Data":"0c48dbf9c2f1c8c84bafa68d26e660609737b09911e612a3cbdd98925ea9b443"}
Oct 30 18:38:02.980932 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:02.980917    4888 generic.go:356] "Generic (PLEG): container finished" podID="12fe33f9-270e-41c7-b3ab-2ba8b0678a37" containerID="887822b025bdfa0d775e67abadbc745f4b2126cbcf2862cccfae2090b3df9e26" exitCode=0
Oct 30 18:38:02.981026 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:02.981005    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847acr7cc" event={"ID":"12fe33f9-270e-41c7-b3ab-2ba8b0678a37","Type":"ContainerDied","Data":"887822b025bdfa0d775e67abadbc745f4b2126cbcf2862cccfae2090b3df9e26"}
Oct 30 18:38:02.983672 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:02.983653    4888 generic.go:356] "Generic (PLEG): container finished" podID="95475eb2-3d8f-469d-b1a1-e5cdf7567012" containerID="89b1f98b8b90ee0f5804473935dca84297a46cc04c41ed8db73282ae0acfafe2" exitCode=0
Oct 30 18:38:02.985090 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:02.984352    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6mwnhv" event={"ID":"95475eb2-3d8f-469d-b1a1-e5cdf7567012","Type":"ContainerDied","Data":"89b1f98b8b90ee0f5804473935dca84297a46cc04c41ed8db73282ae0acfafe2"}
Oct 30 18:38:03.994235 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:03.994204    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c6w88m" event={"ID":"bf03a161-80e8-4191-b562-661a7117035a","Type":"ContainerStarted","Data":"8919722785f653d9b818c37c9af4dada7247dc049de68b5acf9c2051d0c2647f"}
Oct 30 18:38:03.998012 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:03.997241    4888 generic.go:356] "Generic (PLEG): container finished" podID="3861ca20-0760-4301-a912-95920a9601ec" containerID="43f7a28ee0031a34a60e6f7afc52e9d3d32fd67a476f155f14c3d2d10a46c035" exitCode=0
Oct 30 18:38:03.998012 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:03.997350    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71r5qv6" event={"ID":"3861ca20-0760-4301-a912-95920a9601ec","Type":"ContainerDied","Data":"43f7a28ee0031a34a60e6f7afc52e9d3d32fd67a476f155f14c3d2d10a46c035"}
Oct 30 18:38:04.004595 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:04.001912    4888 generic.go:356] "Generic (PLEG): container finished" podID="6138d789-ca22-4a27-93df-95c27e6c4dc7" containerID="3352e60cc9d6030b58441404410b09cc6da746e6e90f1e221c1c1f0597800c29" exitCode=0
Oct 30 18:38:04.004595 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:04.003135    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7pmzc2" event={"ID":"6138d789-ca22-4a27-93df-95c27e6c4dc7","Type":"ContainerDied","Data":"3352e60cc9d6030b58441404410b09cc6da746e6e90f1e221c1c1f0597800c29"}
Oct 30 18:38:05.015358 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:05.015220    4888 generic.go:356] "Generic (PLEG): container finished" podID="bf03a161-80e8-4191-b562-661a7117035a" containerID="8919722785f653d9b818c37c9af4dada7247dc049de68b5acf9c2051d0c2647f" exitCode=0
Oct 30 18:38:05.016465 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:05.016340    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c6w88m" event={"ID":"bf03a161-80e8-4191-b562-661a7117035a","Type":"ContainerDied","Data":"8919722785f653d9b818c37c9af4dada7247dc049de68b5acf9c2051d0c2647f"}
Oct 30 18:38:05.633466 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:38:05.633426    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-jbsw6" containerName="extractor"
Oct 30 18:38:07.032600 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:07.032518    4888 generic.go:356] "Generic (PLEG): container finished" podID="ec4e00bf-5598-40ca-ac8b-874419ab0f33" containerID="b1144add109f85ee11f39900e7fb0ee7b4da530a4ef8eea0572f6763f05a0ca1" exitCode=0
Oct 30 18:38:07.033052 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:07.032598    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644thm68" event={"ID":"ec4e00bf-5598-40ca-ac8b-874419ab0f33","Type":"ContainerDied","Data":"b1144add109f85ee11f39900e7fb0ee7b4da530a4ef8eea0572f6763f05a0ca1"}
Oct 30 18:38:18.118980 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:18.118941    4888 generic.go:356] "Generic (PLEG): container finished" podID="12fe33f9-270e-41c7-b3ab-2ba8b0678a37" containerID="bdfd9f41b63c3a566e64131acb314b138c9aed7c8456b794acc4f93defba57cf" exitCode=0
Oct 30 18:38:18.119548 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:18.119037    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847acr7cc" event={"ID":"12fe33f9-270e-41c7-b3ab-2ba8b0678a37","Type":"ContainerDied","Data":"bdfd9f41b63c3a566e64131acb314b138c9aed7c8456b794acc4f93defba57cf"}
Oct 30 18:38:19.129179 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:19.129153    4888 generic.go:356] "Generic (PLEG): container finished" podID="95475eb2-3d8f-469d-b1a1-e5cdf7567012" containerID="85bb241d304e3d06a7bbf0a7e59b9226e6a3eb906ce95d07674d58debbb26aa6" exitCode=0
Oct 30 18:38:19.130314 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:19.129666    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6mwnhv" event={"ID":"95475eb2-3d8f-469d-b1a1-e5cdf7567012","Type":"ContainerDied","Data":"85bb241d304e3d06a7bbf0a7e59b9226e6a3eb906ce95d07674d58debbb26aa6"}
Oct 30 18:38:19.300188 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:19.299902    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847acr7cc"
Oct 30 18:38:19.396486 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:19.396450    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/12fe33f9-270e-41c7-b3ab-2ba8b0678a37-bundle\" (UniqueName: \"kubernetes.io/empty-dir/12fe33f9-270e-41c7-b3ab-2ba8b0678a37-bundle\") pod \"12fe33f9-270e-41c7-b3ab-2ba8b0678a37\" (UID: \"12fe33f9-270e-41c7-b3ab-2ba8b0678a37\") "
Oct 30 18:38:19.396671 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:19.396558    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/12fe33f9-270e-41c7-b3ab-2ba8b0678a37-util\" (UniqueName: \"kubernetes.io/empty-dir/12fe33f9-270e-41c7-b3ab-2ba8b0678a37-util\") pod \"12fe33f9-270e-41c7-b3ab-2ba8b0678a37\" (UID: \"12fe33f9-270e-41c7-b3ab-2ba8b0678a37\") "
Oct 30 18:38:19.396671 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:19.396582    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/12fe33f9-270e-41c7-b3ab-2ba8b0678a37-kube-api-access-5vknr\" (UniqueName: \"kubernetes.io/projected/12fe33f9-270e-41c7-b3ab-2ba8b0678a37-kube-api-access-5vknr\") pod \"12fe33f9-270e-41c7-b3ab-2ba8b0678a37\" (UID: \"12fe33f9-270e-41c7-b3ab-2ba8b0678a37\") "
Oct 30 18:38:19.397763 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:19.397724    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/12fe33f9-270e-41c7-b3ab-2ba8b0678a37-bundle" pod "12fe33f9-270e-41c7-b3ab-2ba8b0678a37" (UID: "12fe33f9-270e-41c7-b3ab-2ba8b0678a37"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:38:19.399550 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:19.399518    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/12fe33f9-270e-41c7-b3ab-2ba8b0678a37-kube-api-access-5vknr" pod "12fe33f9-270e-41c7-b3ab-2ba8b0678a37" (UID: "12fe33f9-270e-41c7-b3ab-2ba8b0678a37"). InnerVolumeSpecName "kube-api-access-5vknr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:38:19.402750 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:19.402722    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/12fe33f9-270e-41c7-b3ab-2ba8b0678a37-util" pod "12fe33f9-270e-41c7-b3ab-2ba8b0678a37" (UID: "12fe33f9-270e-41c7-b3ab-2ba8b0678a37"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:38:19.498223 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:19.498183    4888 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/12fe33f9-270e-41c7-b3ab-2ba8b0678a37-util\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:38:19.498223 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:19.498222    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5vknr\" (UniqueName: \"kubernetes.io/projected/12fe33f9-270e-41c7-b3ab-2ba8b0678a37-kube-api-access-5vknr\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:38:19.498528 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:19.498236    4888 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/12fe33f9-270e-41c7-b3ab-2ba8b0678a37-bundle\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:38:20.136911 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:20.136874    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c6w88m" event={"ID":"bf03a161-80e8-4191-b562-661a7117035a","Type":"ContainerStarted","Data":"6f1402ea3df41514a05441d01bbd07f06501194d441d0bff3376c03b02155220"}
Oct 30 18:38:20.139395 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:20.139366    4888 generic.go:356] "Generic (PLEG): container finished" podID="7b3553c3-4af3-49ce-9cd9-b22198c1e7be" containerID="378b297f62d5ffee4b814bff7414be1b0fd286d13f1be0020f6cc12146d5b7fe" exitCode=0
Oct 30 18:38:20.139527 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:20.139441    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqspzw" event={"ID":"7b3553c3-4af3-49ce-9cd9-b22198c1e7be","Type":"ContainerDied","Data":"378b297f62d5ffee4b814bff7414be1b0fd286d13f1be0020f6cc12146d5b7fe"}
Oct 30 18:38:20.141576 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:20.141550    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847acr7cc" event={"ID":"12fe33f9-270e-41c7-b3ab-2ba8b0678a37","Type":"ContainerDied","Data":"8c20676f23ab41ae16f0132b86168818327afcce473c87e3027d5c1486e51468"}
Oct 30 18:38:20.141764 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:20.141578    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847acr7cc"
Oct 30 18:38:20.141764 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:20.141592    4888 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8c20676f23ab41ae16f0132b86168818327afcce473c87e3027d5c1486e51468"
Oct 30 18:38:20.144399 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:20.144374    4888 generic.go:356] "Generic (PLEG): container finished" podID="3861ca20-0760-4301-a912-95920a9601ec" containerID="8f82678cc0e70fc834bf25d52614df4f862d6366466547ffb6a07bdfd0a9aa44" exitCode=0
Oct 30 18:38:20.144515 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:20.144425    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71r5qv6" event={"ID":"3861ca20-0760-4301-a912-95920a9601ec","Type":"ContainerDied","Data":"8f82678cc0e70fc834bf25d52614df4f862d6366466547ffb6a07bdfd0a9aa44"}
Oct 30 18:38:20.146994 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:20.146975    4888 generic.go:356] "Generic (PLEG): container finished" podID="6138d789-ca22-4a27-93df-95c27e6c4dc7" containerID="7e09f955aac7a871a0d12c9df7dab8c4e10834aca1d79f69b18c2eef26a8aa9e" exitCode=0
Oct 30 18:38:20.147095 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:20.147060    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7pmzc2" event={"ID":"6138d789-ca22-4a27-93df-95c27e6c4dc7","Type":"ContainerDied","Data":"7e09f955aac7a871a0d12c9df7dab8c4e10834aca1d79f69b18c2eef26a8aa9e"}
Oct 30 18:38:20.257947 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:20.257894    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c6w88m" podStartSLOduration=4.561046561 podStartE2EDuration="38.257878089s" totalImagesPullingTime="33.696831528s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:37:42 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:38:20.257773997 +0000 UTC m=+264.505180032" watchObservedRunningTime="2026-10-30 18:38:20.257878089 +0000 UTC m=+264.505284116"
Oct 30 18:38:20.294868 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:20.294847    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6mwnhv"
Oct 30 18:38:20.407783 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:20.407754    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/95475eb2-3d8f-469d-b1a1-e5cdf7567012-kube-api-access-t6v8t\" (UniqueName: \"kubernetes.io/projected/95475eb2-3d8f-469d-b1a1-e5cdf7567012-kube-api-access-t6v8t\") pod \"95475eb2-3d8f-469d-b1a1-e5cdf7567012\" (UID: \"95475eb2-3d8f-469d-b1a1-e5cdf7567012\") "
Oct 30 18:38:20.407915 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:20.407889    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/95475eb2-3d8f-469d-b1a1-e5cdf7567012-util\" (UniqueName: \"kubernetes.io/empty-dir/95475eb2-3d8f-469d-b1a1-e5cdf7567012-util\") pod \"95475eb2-3d8f-469d-b1a1-e5cdf7567012\" (UID: \"95475eb2-3d8f-469d-b1a1-e5cdf7567012\") "
Oct 30 18:38:20.407976 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:20.407914    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/95475eb2-3d8f-469d-b1a1-e5cdf7567012-bundle\" (UniqueName: \"kubernetes.io/empty-dir/95475eb2-3d8f-469d-b1a1-e5cdf7567012-bundle\") pod \"95475eb2-3d8f-469d-b1a1-e5cdf7567012\" (UID: \"95475eb2-3d8f-469d-b1a1-e5cdf7567012\") "
Oct 30 18:38:20.408545 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:20.408518    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/95475eb2-3d8f-469d-b1a1-e5cdf7567012-bundle" pod "95475eb2-3d8f-469d-b1a1-e5cdf7567012" (UID: "95475eb2-3d8f-469d-b1a1-e5cdf7567012"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:38:20.410139 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:20.410113    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/95475eb2-3d8f-469d-b1a1-e5cdf7567012-kube-api-access-t6v8t" pod "95475eb2-3d8f-469d-b1a1-e5cdf7567012" (UID: "95475eb2-3d8f-469d-b1a1-e5cdf7567012"). InnerVolumeSpecName "kube-api-access-t6v8t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:38:20.414451 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:20.414421    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/95475eb2-3d8f-469d-b1a1-e5cdf7567012-util" pod "95475eb2-3d8f-469d-b1a1-e5cdf7567012" (UID: "95475eb2-3d8f-469d-b1a1-e5cdf7567012"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:38:20.509324 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:20.509242    4888 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/95475eb2-3d8f-469d-b1a1-e5cdf7567012-util\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:38:20.509324 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:20.509271    4888 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/95475eb2-3d8f-469d-b1a1-e5cdf7567012-bundle\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:38:20.509324 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:20.509308    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t6v8t\" (UniqueName: \"kubernetes.io/projected/95475eb2-3d8f-469d-b1a1-e5cdf7567012-kube-api-access-t6v8t\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:38:21.153194 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:21.153149    4888 generic.go:356] "Generic (PLEG): container finished" podID="ec4e00bf-5598-40ca-ac8b-874419ab0f33" containerID="cd545b6ac1ed3083aec57b810656f23605a3032b528bd94f760ee3a16294f466" exitCode=0
Oct 30 18:38:21.153630 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:21.153243    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644thm68" event={"ID":"ec4e00bf-5598-40ca-ac8b-874419ab0f33","Type":"ContainerDied","Data":"cd545b6ac1ed3083aec57b810656f23605a3032b528bd94f760ee3a16294f466"}
Oct 30 18:38:21.155159 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:21.155136    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6mwnhv"
Oct 30 18:38:21.155310 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:21.155162    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6mwnhv" event={"ID":"95475eb2-3d8f-469d-b1a1-e5cdf7567012","Type":"ContainerDied","Data":"38186fc1f7f9188ad813f86c0f04e219aa9daf600592caa81600c36b549a8ab0"}
Oct 30 18:38:21.155310 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:21.155209    4888 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="38186fc1f7f9188ad813f86c0f04e219aa9daf600592caa81600c36b549a8ab0"
Oct 30 18:38:21.157197 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:21.157179    4888 generic.go:356] "Generic (PLEG): container finished" podID="bf03a161-80e8-4191-b562-661a7117035a" containerID="6f1402ea3df41514a05441d01bbd07f06501194d441d0bff3376c03b02155220" exitCode=0
Oct 30 18:38:21.157324 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:21.157276    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c6w88m" event={"ID":"bf03a161-80e8-4191-b562-661a7117035a","Type":"ContainerDied","Data":"6f1402ea3df41514a05441d01bbd07f06501194d441d0bff3376c03b02155220"}
Oct 30 18:38:21.296350 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:21.293695    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71r5qv6"
Oct 30 18:38:21.328515 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:21.328491    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqspzw"
Oct 30 18:38:21.338153 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:21.338131    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7pmzc2"
Oct 30 18:38:21.417417 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:21.417392    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7b3553c3-4af3-49ce-9cd9-b22198c1e7be-bundle\" (UniqueName: \"kubernetes.io/empty-dir/7b3553c3-4af3-49ce-9cd9-b22198c1e7be-bundle\") pod \"7b3553c3-4af3-49ce-9cd9-b22198c1e7be\" (UID: \"7b3553c3-4af3-49ce-9cd9-b22198c1e7be\") "
Oct 30 18:38:21.417556 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:21.417437    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3861ca20-0760-4301-a912-95920a9601ec-util\" (UniqueName: \"kubernetes.io/empty-dir/3861ca20-0760-4301-a912-95920a9601ec-util\") pod \"3861ca20-0760-4301-a912-95920a9601ec\" (UID: \"3861ca20-0760-4301-a912-95920a9601ec\") "
Oct 30 18:38:21.417556 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:21.417461    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7b3553c3-4af3-49ce-9cd9-b22198c1e7be-util\" (UniqueName: \"kubernetes.io/empty-dir/7b3553c3-4af3-49ce-9cd9-b22198c1e7be-util\") pod \"7b3553c3-4af3-49ce-9cd9-b22198c1e7be\" (UID: \"7b3553c3-4af3-49ce-9cd9-b22198c1e7be\") "
Oct 30 18:38:21.417556 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:21.417524    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7b3553c3-4af3-49ce-9cd9-b22198c1e7be-kube-api-access-fmv6d\" (UniqueName: \"kubernetes.io/projected/7b3553c3-4af3-49ce-9cd9-b22198c1e7be-kube-api-access-fmv6d\") pod \"7b3553c3-4af3-49ce-9cd9-b22198c1e7be\" (UID: \"7b3553c3-4af3-49ce-9cd9-b22198c1e7be\") "
Oct 30 18:38:21.417556 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:21.417540    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3861ca20-0760-4301-a912-95920a9601ec-kube-api-access-wthn2\" (UniqueName: \"kubernetes.io/projected/3861ca20-0760-4301-a912-95920a9601ec-kube-api-access-wthn2\") pod \"3861ca20-0760-4301-a912-95920a9601ec\" (UID: \"3861ca20-0760-4301-a912-95920a9601ec\") "
Oct 30 18:38:21.417763 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:21.417564    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6138d789-ca22-4a27-93df-95c27e6c4dc7-util\" (UniqueName: \"kubernetes.io/empty-dir/6138d789-ca22-4a27-93df-95c27e6c4dc7-util\") pod \"6138d789-ca22-4a27-93df-95c27e6c4dc7\" (UID: \"6138d789-ca22-4a27-93df-95c27e6c4dc7\") "
Oct 30 18:38:21.417763 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:21.417593    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6138d789-ca22-4a27-93df-95c27e6c4dc7-kube-api-access-rpmpq\" (UniqueName: \"kubernetes.io/projected/6138d789-ca22-4a27-93df-95c27e6c4dc7-kube-api-access-rpmpq\") pod \"6138d789-ca22-4a27-93df-95c27e6c4dc7\" (UID: \"6138d789-ca22-4a27-93df-95c27e6c4dc7\") "
Oct 30 18:38:21.417763 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:21.417631    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6138d789-ca22-4a27-93df-95c27e6c4dc7-bundle\" (UniqueName: \"kubernetes.io/empty-dir/6138d789-ca22-4a27-93df-95c27e6c4dc7-bundle\") pod \"6138d789-ca22-4a27-93df-95c27e6c4dc7\" (UID: \"6138d789-ca22-4a27-93df-95c27e6c4dc7\") "
Oct 30 18:38:21.417763 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:21.417662    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3861ca20-0760-4301-a912-95920a9601ec-bundle\" (UniqueName: \"kubernetes.io/empty-dir/3861ca20-0760-4301-a912-95920a9601ec-bundle\") pod \"3861ca20-0760-4301-a912-95920a9601ec\" (UID: \"3861ca20-0760-4301-a912-95920a9601ec\") "
Oct 30 18:38:21.418628 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:21.418593    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7b3553c3-4af3-49ce-9cd9-b22198c1e7be-bundle" pod "7b3553c3-4af3-49ce-9cd9-b22198c1e7be" (UID: "7b3553c3-4af3-49ce-9cd9-b22198c1e7be"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:38:21.418748 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:21.418720    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3861ca20-0760-4301-a912-95920a9601ec-bundle" pod "3861ca20-0760-4301-a912-95920a9601ec" (UID: "3861ca20-0760-4301-a912-95920a9601ec"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:38:21.419448 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:21.419419    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6138d789-ca22-4a27-93df-95c27e6c4dc7-bundle" pod "6138d789-ca22-4a27-93df-95c27e6c4dc7" (UID: "6138d789-ca22-4a27-93df-95c27e6c4dc7"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:38:21.420042 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:21.420010    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3861ca20-0760-4301-a912-95920a9601ec-kube-api-access-wthn2" pod "3861ca20-0760-4301-a912-95920a9601ec" (UID: "3861ca20-0760-4301-a912-95920a9601ec"). InnerVolumeSpecName "kube-api-access-wthn2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:38:21.420138 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:21.420114    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7b3553c3-4af3-49ce-9cd9-b22198c1e7be-kube-api-access-fmv6d" pod "7b3553c3-4af3-49ce-9cd9-b22198c1e7be" (UID: "7b3553c3-4af3-49ce-9cd9-b22198c1e7be"). InnerVolumeSpecName "kube-api-access-fmv6d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:38:21.420235 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:21.420216    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6138d789-ca22-4a27-93df-95c27e6c4dc7-kube-api-access-rpmpq" pod "6138d789-ca22-4a27-93df-95c27e6c4dc7" (UID: "6138d789-ca22-4a27-93df-95c27e6c4dc7"). InnerVolumeSpecName "kube-api-access-rpmpq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:38:21.424470 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:21.424451    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3861ca20-0760-4301-a912-95920a9601ec-util" pod "3861ca20-0760-4301-a912-95920a9601ec" (UID: "3861ca20-0760-4301-a912-95920a9601ec"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:38:21.425801 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:21.425775    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7b3553c3-4af3-49ce-9cd9-b22198c1e7be-util" pod "7b3553c3-4af3-49ce-9cd9-b22198c1e7be" (UID: "7b3553c3-4af3-49ce-9cd9-b22198c1e7be"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:38:21.425940 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:21.425921    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6138d789-ca22-4a27-93df-95c27e6c4dc7-util" pod "6138d789-ca22-4a27-93df-95c27e6c4dc7" (UID: "6138d789-ca22-4a27-93df-95c27e6c4dc7"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:38:21.518335 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:21.518308    4888 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3861ca20-0760-4301-a912-95920a9601ec-util\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:38:21.518335 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:21.518327    4888 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7b3553c3-4af3-49ce-9cd9-b22198c1e7be-util\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:38:21.518335 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:21.518338    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fmv6d\" (UniqueName: \"kubernetes.io/projected/7b3553c3-4af3-49ce-9cd9-b22198c1e7be-kube-api-access-fmv6d\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:38:21.518489 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:21.518347    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wthn2\" (UniqueName: \"kubernetes.io/projected/3861ca20-0760-4301-a912-95920a9601ec-kube-api-access-wthn2\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:38:21.518489 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:21.518356    4888 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6138d789-ca22-4a27-93df-95c27e6c4dc7-util\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:38:21.518489 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:21.518365    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rpmpq\" (UniqueName: \"kubernetes.io/projected/6138d789-ca22-4a27-93df-95c27e6c4dc7-kube-api-access-rpmpq\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:38:21.518489 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:21.518372    4888 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6138d789-ca22-4a27-93df-95c27e6c4dc7-bundle\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:38:21.518489 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:21.518381    4888 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3861ca20-0760-4301-a912-95920a9601ec-bundle\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:38:21.518489 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:21.518388    4888 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7b3553c3-4af3-49ce-9cd9-b22198c1e7be-bundle\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:38:22.163308 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:22.163209    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7pmzc2"
Oct 30 18:38:22.163308 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:22.163225    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7pmzc2" event={"ID":"6138d789-ca22-4a27-93df-95c27e6c4dc7","Type":"ContainerDied","Data":"585419b236afdbccda06c8c7eb66b128d3a777bc6981c2eeef020403f783dc51"}
Oct 30 18:38:22.163308 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:22.163271    4888 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="585419b236afdbccda06c8c7eb66b128d3a777bc6981c2eeef020403f783dc51"
Oct 30 18:38:22.165341 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:22.165321    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqspzw"
Oct 30 18:38:22.165531 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:22.165321    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqspzw" event={"ID":"7b3553c3-4af3-49ce-9cd9-b22198c1e7be","Type":"ContainerDied","Data":"eac659f6fbb833d4f6cab8c06e361650fa3acfc118bf6126ce4824de6cccc4d1"}
Oct 30 18:38:22.165531 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:22.165463    4888 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="eac659f6fbb833d4f6cab8c06e361650fa3acfc118bf6126ce4824de6cccc4d1"
Oct 30 18:38:22.167498 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:22.167479    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71r5qv6" event={"ID":"3861ca20-0760-4301-a912-95920a9601ec","Type":"ContainerDied","Data":"54ef21074d7aa07007d0b9632c03ba5bf6896bfcd945421ccc663f3d2cd7aa6d"}
Oct 30 18:38:22.167597 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:22.167512    4888 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="54ef21074d7aa07007d0b9632c03ba5bf6896bfcd945421ccc663f3d2cd7aa6d"
Oct 30 18:38:22.167652 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:22.167629    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71r5qv6"
Oct 30 18:38:22.297484 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:22.297462    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c6w88m"
Oct 30 18:38:22.321511 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:22.321492    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644thm68"
Oct 30 18:38:22.425788 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:22.425724    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bf03a161-80e8-4191-b562-661a7117035a-bundle\" (UniqueName: \"kubernetes.io/empty-dir/bf03a161-80e8-4191-b562-661a7117035a-bundle\") pod \"bf03a161-80e8-4191-b562-661a7117035a\" (UID: \"bf03a161-80e8-4191-b562-661a7117035a\") "
Oct 30 18:38:22.425788 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:22.425776    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ec4e00bf-5598-40ca-ac8b-874419ab0f33-kube-api-access-6vxld\" (UniqueName: \"kubernetes.io/projected/ec4e00bf-5598-40ca-ac8b-874419ab0f33-kube-api-access-6vxld\") pod \"ec4e00bf-5598-40ca-ac8b-874419ab0f33\" (UID: \"ec4e00bf-5598-40ca-ac8b-874419ab0f33\") "
Oct 30 18:38:22.425981 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:22.425821    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bf03a161-80e8-4191-b562-661a7117035a-util\" (UniqueName: \"kubernetes.io/empty-dir/bf03a161-80e8-4191-b562-661a7117035a-util\") pod \"bf03a161-80e8-4191-b562-661a7117035a\" (UID: \"bf03a161-80e8-4191-b562-661a7117035a\") "
Oct 30 18:38:22.425981 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:22.425844    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ec4e00bf-5598-40ca-ac8b-874419ab0f33-util\" (UniqueName: \"kubernetes.io/empty-dir/ec4e00bf-5598-40ca-ac8b-874419ab0f33-util\") pod \"ec4e00bf-5598-40ca-ac8b-874419ab0f33\" (UID: \"ec4e00bf-5598-40ca-ac8b-874419ab0f33\") "
Oct 30 18:38:22.425981 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:22.425889    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ec4e00bf-5598-40ca-ac8b-874419ab0f33-bundle\" (UniqueName: \"kubernetes.io/empty-dir/ec4e00bf-5598-40ca-ac8b-874419ab0f33-bundle\") pod \"ec4e00bf-5598-40ca-ac8b-874419ab0f33\" (UID: \"ec4e00bf-5598-40ca-ac8b-874419ab0f33\") "
Oct 30 18:38:22.425981 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:22.425928    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bf03a161-80e8-4191-b562-661a7117035a-kube-api-access-m9lh9\" (UniqueName: \"kubernetes.io/projected/bf03a161-80e8-4191-b562-661a7117035a-kube-api-access-m9lh9\") pod \"bf03a161-80e8-4191-b562-661a7117035a\" (UID: \"bf03a161-80e8-4191-b562-661a7117035a\") "
Oct 30 18:38:22.426341 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:22.426314    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec4e00bf-5598-40ca-ac8b-874419ab0f33-bundle" pod "ec4e00bf-5598-40ca-ac8b-874419ab0f33" (UID: "ec4e00bf-5598-40ca-ac8b-874419ab0f33"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:38:22.426572 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:22.426551    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bf03a161-80e8-4191-b562-661a7117035a-bundle" pod "bf03a161-80e8-4191-b562-661a7117035a" (UID: "bf03a161-80e8-4191-b562-661a7117035a"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:38:22.428662 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:22.428640    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ec4e00bf-5598-40ca-ac8b-874419ab0f33-kube-api-access-6vxld" pod "ec4e00bf-5598-40ca-ac8b-874419ab0f33" (UID: "ec4e00bf-5598-40ca-ac8b-874419ab0f33"). InnerVolumeSpecName "kube-api-access-6vxld". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:38:22.428662 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:22.428644    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bf03a161-80e8-4191-b562-661a7117035a-kube-api-access-m9lh9" pod "bf03a161-80e8-4191-b562-661a7117035a" (UID: "bf03a161-80e8-4191-b562-661a7117035a"). InnerVolumeSpecName "kube-api-access-m9lh9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:38:22.432033 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:22.432014    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bf03a161-80e8-4191-b562-661a7117035a-util" pod "bf03a161-80e8-4191-b562-661a7117035a" (UID: "bf03a161-80e8-4191-b562-661a7117035a"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:38:22.434043 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:22.434016    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec4e00bf-5598-40ca-ac8b-874419ab0f33-util" pod "ec4e00bf-5598-40ca-ac8b-874419ab0f33" (UID: "ec4e00bf-5598-40ca-ac8b-874419ab0f33"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:38:22.526983 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:22.526960    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m9lh9\" (UniqueName: \"kubernetes.io/projected/bf03a161-80e8-4191-b562-661a7117035a-kube-api-access-m9lh9\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:38:22.527092 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:22.526984    4888 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bf03a161-80e8-4191-b562-661a7117035a-bundle\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:38:22.527092 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:22.526998    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6vxld\" (UniqueName: \"kubernetes.io/projected/ec4e00bf-5598-40ca-ac8b-874419ab0f33-kube-api-access-6vxld\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:38:22.527092 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:22.527006    4888 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bf03a161-80e8-4191-b562-661a7117035a-util\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:38:22.527092 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:22.527014    4888 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ec4e00bf-5598-40ca-ac8b-874419ab0f33-util\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:38:22.527092 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:22.527021    4888 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ec4e00bf-5598-40ca-ac8b-874419ab0f33-bundle\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:38:23.178353 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:23.178313    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c6w88m" event={"ID":"bf03a161-80e8-4191-b562-661a7117035a","Type":"ContainerDied","Data":"909e8d2686a0a7f7db5cc39631faa3c18b0d5d67767c3c72bf790488f639f33f"}
Oct 30 18:38:23.178353 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:23.178350    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c6w88m"
Oct 30 18:38:23.178822 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:23.178397    4888 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="909e8d2686a0a7f7db5cc39631faa3c18b0d5d67767c3c72bf790488f639f33f"
Oct 30 18:38:23.180275 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:23.180245    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644thm68" event={"ID":"ec4e00bf-5598-40ca-ac8b-874419ab0f33","Type":"ContainerDied","Data":"4233672615f7f235b468a37c972e2e6495c7e226e83fb65c5435afd082030dc0"}
Oct 30 18:38:23.180275 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:23.180292    4888 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4233672615f7f235b468a37c972e2e6495c7e226e83fb65c5435afd082030dc0"
Oct 30 18:38:23.180449 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:23.180252    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644thm68"
Oct 30 18:38:25.328369 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:25.328335    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-67f4db5dc9-nqp9q"]
Oct 30 18:38:25.328984 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:25.328761    4888 cpu_manager.go:414] "containerMap: removing container" podUID="12fe33f9-270e-41c7-b3ab-2ba8b0678a37" containerName="pull"
Oct 30 18:38:25.328984 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:25.328777    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="12fe33f9-270e-41c7-b3ab-2ba8b0678a37" containerName="pull"
Oct 30 18:38:25.328984 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:25.328788    4888 cpu_manager.go:414] "containerMap: removing container" podUID="7b3553c3-4af3-49ce-9cd9-b22198c1e7be" containerName="pull"
Oct 30 18:38:25.328984 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:25.328797    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7b3553c3-4af3-49ce-9cd9-b22198c1e7be" containerName="pull"
Oct 30 18:38:25.328984 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:25.328810    4888 cpu_manager.go:414] "containerMap: removing container" podUID="7b3553c3-4af3-49ce-9cd9-b22198c1e7be" containerName="util"
Oct 30 18:38:25.328984 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:25.328816    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7b3553c3-4af3-49ce-9cd9-b22198c1e7be" containerName="util"
Oct 30 18:38:25.328984 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:25.328828    4888 cpu_manager.go:414] "containerMap: removing container" podUID="3861ca20-0760-4301-a912-95920a9601ec" containerName="extract"
Oct 30 18:38:25.328984 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:25.328835    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3861ca20-0760-4301-a912-95920a9601ec" containerName="extract"
Oct 30 18:38:25.328984 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:25.328851    4888 cpu_manager.go:414] "containerMap: removing container" podUID="95475eb2-3d8f-469d-b1a1-e5cdf7567012" containerName="pull"
Oct 30 18:38:25.328984 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:25.328857    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="95475eb2-3d8f-469d-b1a1-e5cdf7567012" containerName="pull"
Oct 30 18:38:25.328984 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:25.328872    4888 cpu_manager.go:414] "containerMap: removing container" podUID="ec4e00bf-5598-40ca-ac8b-874419ab0f33" containerName="util"
Oct 30 18:38:25.328984 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:25.328879    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ec4e00bf-5598-40ca-ac8b-874419ab0f33" containerName="util"
Oct 30 18:38:25.328984 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:25.328889    4888 cpu_manager.go:414] "containerMap: removing container" podUID="12fe33f9-270e-41c7-b3ab-2ba8b0678a37" containerName="util"
Oct 30 18:38:25.328984 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:25.328896    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="12fe33f9-270e-41c7-b3ab-2ba8b0678a37" containerName="util"
Oct 30 18:38:25.328984 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:25.328905    4888 cpu_manager.go:414] "containerMap: removing container" podUID="ec4e00bf-5598-40ca-ac8b-874419ab0f33" containerName="pull"
Oct 30 18:38:25.328984 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:25.328912    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ec4e00bf-5598-40ca-ac8b-874419ab0f33" containerName="pull"
Oct 30 18:38:25.328984 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:25.328922    4888 cpu_manager.go:414] "containerMap: removing container" podUID="95475eb2-3d8f-469d-b1a1-e5cdf7567012" containerName="extract"
Oct 30 18:38:25.328984 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:25.328928    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="95475eb2-3d8f-469d-b1a1-e5cdf7567012" containerName="extract"
Oct 30 18:38:25.328984 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:25.328940    4888 cpu_manager.go:414] "containerMap: removing container" podUID="6138d789-ca22-4a27-93df-95c27e6c4dc7" containerName="extract"
Oct 30 18:38:25.328984 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:25.328947    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6138d789-ca22-4a27-93df-95c27e6c4dc7" containerName="extract"
Oct 30 18:38:25.328984 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:25.328963    4888 cpu_manager.go:414] "containerMap: removing container" podUID="6138d789-ca22-4a27-93df-95c27e6c4dc7" containerName="util"
Oct 30 18:38:25.328984 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:25.328970    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6138d789-ca22-4a27-93df-95c27e6c4dc7" containerName="util"
Oct 30 18:38:25.328984 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:25.328980    4888 cpu_manager.go:414] "containerMap: removing container" podUID="12fe33f9-270e-41c7-b3ab-2ba8b0678a37" containerName="extract"
Oct 30 18:38:25.328984 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:25.328987    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="12fe33f9-270e-41c7-b3ab-2ba8b0678a37" containerName="extract"
Oct 30 18:38:25.330132 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:25.329000    4888 cpu_manager.go:414] "containerMap: removing container" podUID="3861ca20-0760-4301-a912-95920a9601ec" containerName="util"
Oct 30 18:38:25.330132 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:25.329006    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3861ca20-0760-4301-a912-95920a9601ec" containerName="util"
Oct 30 18:38:25.330132 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:25.329015    4888 cpu_manager.go:414] "containerMap: removing container" podUID="3861ca20-0760-4301-a912-95920a9601ec" containerName="pull"
Oct 30 18:38:25.330132 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:25.329021    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3861ca20-0760-4301-a912-95920a9601ec" containerName="pull"
Oct 30 18:38:25.330132 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:25.329032    4888 cpu_manager.go:414] "containerMap: removing container" podUID="7b3553c3-4af3-49ce-9cd9-b22198c1e7be" containerName="extract"
Oct 30 18:38:25.330132 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:25.329038    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7b3553c3-4af3-49ce-9cd9-b22198c1e7be" containerName="extract"
Oct 30 18:38:25.330132 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:25.329050    4888 cpu_manager.go:414] "containerMap: removing container" podUID="bf03a161-80e8-4191-b562-661a7117035a" containerName="pull"
Oct 30 18:38:25.330132 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:25.329056    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bf03a161-80e8-4191-b562-661a7117035a" containerName="pull"
Oct 30 18:38:25.330132 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:25.329071    4888 cpu_manager.go:414] "containerMap: removing container" podUID="bf03a161-80e8-4191-b562-661a7117035a" containerName="util"
Oct 30 18:38:25.330132 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:25.329076    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bf03a161-80e8-4191-b562-661a7117035a" containerName="util"
Oct 30 18:38:25.330132 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:25.329085    4888 cpu_manager.go:414] "containerMap: removing container" podUID="95475eb2-3d8f-469d-b1a1-e5cdf7567012" containerName="util"
Oct 30 18:38:25.330132 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:25.329092    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="95475eb2-3d8f-469d-b1a1-e5cdf7567012" containerName="util"
Oct 30 18:38:25.330132 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:25.329100    4888 cpu_manager.go:414] "containerMap: removing container" podUID="bf03a161-80e8-4191-b562-661a7117035a" containerName="extract"
Oct 30 18:38:25.330132 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:25.329107    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bf03a161-80e8-4191-b562-661a7117035a" containerName="extract"
Oct 30 18:38:25.330132 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:25.329121    4888 cpu_manager.go:414] "containerMap: removing container" podUID="ec4e00bf-5598-40ca-ac8b-874419ab0f33" containerName="extract"
Oct 30 18:38:25.330132 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:25.329128    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ec4e00bf-5598-40ca-ac8b-874419ab0f33" containerName="extract"
Oct 30 18:38:25.330132 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:25.329143    4888 cpu_manager.go:414] "containerMap: removing container" podUID="6138d789-ca22-4a27-93df-95c27e6c4dc7" containerName="pull"
Oct 30 18:38:25.330132 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:25.329149    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6138d789-ca22-4a27-93df-95c27e6c4dc7" containerName="pull"
Oct 30 18:38:25.330132 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:25.329232    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="3861ca20-0760-4301-a912-95920a9601ec" containerName="extract"
Oct 30 18:38:25.330132 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:25.329248    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="6138d789-ca22-4a27-93df-95c27e6c4dc7" containerName="extract"
Oct 30 18:38:25.330132 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:25.329258    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="95475eb2-3d8f-469d-b1a1-e5cdf7567012" containerName="extract"
Oct 30 18:38:25.330132 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:25.329266    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="12fe33f9-270e-41c7-b3ab-2ba8b0678a37" containerName="extract"
Oct 30 18:38:25.330132 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:25.329274    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="7b3553c3-4af3-49ce-9cd9-b22198c1e7be" containerName="extract"
Oct 30 18:38:25.330132 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:25.329304    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="bf03a161-80e8-4191-b562-661a7117035a" containerName="extract"
Oct 30 18:38:25.330132 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:25.329313    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="ec4e00bf-5598-40ca-ac8b-874419ab0f33" containerName="extract"
Oct 30 18:38:25.368009 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:25.367982    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-67f4db5dc9-nqp9q"
Oct 30 18:38:25.421206 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:25.421176    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-manager-service-cert\""
Oct 30 18:38:25.421710 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:25.421691    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub-operator-system\"/\"openshift-service-ca.crt\""
Oct 30 18:38:25.421841 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:25.421824    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-manager-dockercfg-xt8v5\""
Oct 30 18:38:25.422022 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:25.422006    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub-operator-system\"/\"kube-root-ca.crt\""
Oct 30 18:38:25.422082 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:25.422026    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-webhook-cert\""
Oct 30 18:38:25.422360 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:25.422341    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-metrics-tls\""
Oct 30 18:38:25.458492 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:25.458463    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/1b42d455-85e3-45de-ab37-79a951e4c6dc-apiservice-cert\") pod \"opendatahub-operator-controller-manager-67f4db5dc9-nqp9q\" (UID: \"1b42d455-85e3-45de-ab37-79a951e4c6dc\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-67f4db5dc9-nqp9q"
Oct 30 18:38:25.458613 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:25.458502    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/1b42d455-85e3-45de-ab37-79a951e4c6dc-manifests\") pod \"opendatahub-operator-controller-manager-67f4db5dc9-nqp9q\" (UID: \"1b42d455-85e3-45de-ab37-79a951e4c6dc\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-67f4db5dc9-nqp9q"
Oct 30 18:38:25.458613 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:25.458527    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1b42d455-85e3-45de-ab37-79a951e4c6dc-webhook-cert\") pod \"opendatahub-operator-controller-manager-67f4db5dc9-nqp9q\" (UID: \"1b42d455-85e3-45de-ab37-79a951e4c6dc\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-67f4db5dc9-nqp9q"
Oct 30 18:38:25.458613 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:25.458590    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/1b42d455-85e3-45de-ab37-79a951e4c6dc-metrics-cert\") pod \"opendatahub-operator-controller-manager-67f4db5dc9-nqp9q\" (UID: \"1b42d455-85e3-45de-ab37-79a951e4c6dc\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-67f4db5dc9-nqp9q"
Oct 30 18:38:25.458735 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:25.458691    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d7zzs\" (UniqueName: \"kubernetes.io/projected/1b42d455-85e3-45de-ab37-79a951e4c6dc-kube-api-access-d7zzs\") pod \"opendatahub-operator-controller-manager-67f4db5dc9-nqp9q\" (UID: \"1b42d455-85e3-45de-ab37-79a951e4c6dc\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-67f4db5dc9-nqp9q"
Oct 30 18:38:25.458787 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:25.458740    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1b42d455-85e3-45de-ab37-79a951e4c6dc-tmp\") pod \"opendatahub-operator-controller-manager-67f4db5dc9-nqp9q\" (UID: \"1b42d455-85e3-45de-ab37-79a951e4c6dc\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-67f4db5dc9-nqp9q"
Oct 30 18:38:25.475342 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:25.475319    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-67f4db5dc9-nqp9q"]
Oct 30 18:38:25.559542 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:25.559513    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1b42d455-85e3-45de-ab37-79a951e4c6dc-tmp\") pod \"opendatahub-operator-controller-manager-67f4db5dc9-nqp9q\" (UID: \"1b42d455-85e3-45de-ab37-79a951e4c6dc\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-67f4db5dc9-nqp9q"
Oct 30 18:38:25.559718 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:25.559550    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/1b42d455-85e3-45de-ab37-79a951e4c6dc-apiservice-cert\") pod \"opendatahub-operator-controller-manager-67f4db5dc9-nqp9q\" (UID: \"1b42d455-85e3-45de-ab37-79a951e4c6dc\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-67f4db5dc9-nqp9q"
Oct 30 18:38:25.559718 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:25.559573    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/1b42d455-85e3-45de-ab37-79a951e4c6dc-manifests\") pod \"opendatahub-operator-controller-manager-67f4db5dc9-nqp9q\" (UID: \"1b42d455-85e3-45de-ab37-79a951e4c6dc\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-67f4db5dc9-nqp9q"
Oct 30 18:38:25.559718 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:25.559587    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1b42d455-85e3-45de-ab37-79a951e4c6dc-webhook-cert\") pod \"opendatahub-operator-controller-manager-67f4db5dc9-nqp9q\" (UID: \"1b42d455-85e3-45de-ab37-79a951e4c6dc\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-67f4db5dc9-nqp9q"
Oct 30 18:38:25.559718 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:25.559603    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/1b42d455-85e3-45de-ab37-79a951e4c6dc-metrics-cert\") pod \"opendatahub-operator-controller-manager-67f4db5dc9-nqp9q\" (UID: \"1b42d455-85e3-45de-ab37-79a951e4c6dc\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-67f4db5dc9-nqp9q"
Oct 30 18:38:25.559718 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:25.559661    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d7zzs\" (UniqueName: \"kubernetes.io/projected/1b42d455-85e3-45de-ab37-79a951e4c6dc-kube-api-access-d7zzs\") pod \"opendatahub-operator-controller-manager-67f4db5dc9-nqp9q\" (UID: \"1b42d455-85e3-45de-ab37-79a951e4c6dc\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-67f4db5dc9-nqp9q"
Oct 30 18:38:25.560011 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:25.559984    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/1b42d455-85e3-45de-ab37-79a951e4c6dc-manifests\") pod \"opendatahub-operator-controller-manager-67f4db5dc9-nqp9q\" (UID: \"1b42d455-85e3-45de-ab37-79a951e4c6dc\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-67f4db5dc9-nqp9q"
Oct 30 18:38:25.562243 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:25.562215    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1b42d455-85e3-45de-ab37-79a951e4c6dc-tmp\") pod \"opendatahub-operator-controller-manager-67f4db5dc9-nqp9q\" (UID: \"1b42d455-85e3-45de-ab37-79a951e4c6dc\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-67f4db5dc9-nqp9q"
Oct 30 18:38:25.562348 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:25.562323    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1b42d455-85e3-45de-ab37-79a951e4c6dc-webhook-cert\") pod \"opendatahub-operator-controller-manager-67f4db5dc9-nqp9q\" (UID: \"1b42d455-85e3-45de-ab37-79a951e4c6dc\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-67f4db5dc9-nqp9q"
Oct 30 18:38:25.562348 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:25.562336    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/1b42d455-85e3-45de-ab37-79a951e4c6dc-metrics-cert\") pod \"opendatahub-operator-controller-manager-67f4db5dc9-nqp9q\" (UID: \"1b42d455-85e3-45de-ab37-79a951e4c6dc\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-67f4db5dc9-nqp9q"
Oct 30 18:38:25.562428 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:25.562407    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/1b42d455-85e3-45de-ab37-79a951e4c6dc-apiservice-cert\") pod \"opendatahub-operator-controller-manager-67f4db5dc9-nqp9q\" (UID: \"1b42d455-85e3-45de-ab37-79a951e4c6dc\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-67f4db5dc9-nqp9q"
Oct 30 18:38:25.622512 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:25.622453    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d7zzs\" (UniqueName: \"kubernetes.io/projected/1b42d455-85e3-45de-ab37-79a951e4c6dc-kube-api-access-d7zzs\") pod \"opendatahub-operator-controller-manager-67f4db5dc9-nqp9q\" (UID: \"1b42d455-85e3-45de-ab37-79a951e4c6dc\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-67f4db5dc9-nqp9q"
Oct 30 18:38:25.721687 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:25.721660    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-67f4db5dc9-nqp9q"
Oct 30 18:38:26.248439 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:26.248381    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-67f4db5dc9-nqp9q"]
Oct 30 18:38:26.250598 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:38:26.250569    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b42d455_85e3_45de_ab37_79a951e4c6dc.slice/crio-d971cf93942fec855098de37535e4b2522bf0ab1220a7a4ab75709b7732187e6 WatchSource:0}: Error finding container d971cf93942fec855098de37535e4b2522bf0ab1220a7a4ab75709b7732187e6: Status 404 returned error can't find the container with id d971cf93942fec855098de37535e4b2522bf0ab1220a7a4ab75709b7732187e6
Oct 30 18:38:27.198320 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:27.198268    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-67f4db5dc9-nqp9q" event={"ID":"1b42d455-85e3-45de-ab37-79a951e4c6dc","Type":"ContainerStarted","Data":"d971cf93942fec855098de37535e4b2522bf0ab1220a7a4ab75709b7732187e6"}
Oct 30 18:38:30.214577 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:30.214538    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-67f4db5dc9-nqp9q" event={"ID":"1b42d455-85e3-45de-ab37-79a951e4c6dc","Type":"ContainerStarted","Data":"969462fa659ac46354adf97bc2d257f8aa7f78874fa1d5fa1facc0691d8db0ea"}
Oct 30 18:38:30.374439 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:30.374404    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-kw5ft"]
Oct 30 18:38:30.378745 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:30.378729    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-kw5ft"
Oct 30 18:38:30.432361 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:30.432338    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-5hf2l\""
Oct 30 18:38:30.432487 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:30.432393    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Oct 30 18:38:30.432613 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:30.432599    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Oct 30 18:38:30.453178 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:30.453155    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-kw5ft"]
Oct 30 18:38:30.504508 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:30.504487    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e9326b0d-0c69-428c-8806-3dd6792c0646-tmp\") pod \"cert-manager-operator-controller-manager-7d87bcd5fd-kw5ft\" (UID: \"e9326b0d-0c69-428c-8806-3dd6792c0646\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-kw5ft"
Oct 30 18:38:30.504622 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:30.504529    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fk75s\" (UniqueName: \"kubernetes.io/projected/e9326b0d-0c69-428c-8806-3dd6792c0646-kube-api-access-fk75s\") pod \"cert-manager-operator-controller-manager-7d87bcd5fd-kw5ft\" (UID: \"e9326b0d-0c69-428c-8806-3dd6792c0646\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-kw5ft"
Oct 30 18:38:30.605654 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:30.605628    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fk75s\" (UniqueName: \"kubernetes.io/projected/e9326b0d-0c69-428c-8806-3dd6792c0646-kube-api-access-fk75s\") pod \"cert-manager-operator-controller-manager-7d87bcd5fd-kw5ft\" (UID: \"e9326b0d-0c69-428c-8806-3dd6792c0646\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-kw5ft"
Oct 30 18:38:30.605793 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:30.605698    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e9326b0d-0c69-428c-8806-3dd6792c0646-tmp\") pod \"cert-manager-operator-controller-manager-7d87bcd5fd-kw5ft\" (UID: \"e9326b0d-0c69-428c-8806-3dd6792c0646\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-kw5ft"
Oct 30 18:38:30.606042 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:30.606026    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e9326b0d-0c69-428c-8806-3dd6792c0646-tmp\") pod \"cert-manager-operator-controller-manager-7d87bcd5fd-kw5ft\" (UID: \"e9326b0d-0c69-428c-8806-3dd6792c0646\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-kw5ft"
Oct 30 18:38:30.668577 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:30.668509    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fk75s\" (UniqueName: \"kubernetes.io/projected/e9326b0d-0c69-428c-8806-3dd6792c0646-kube-api-access-fk75s\") pod \"cert-manager-operator-controller-manager-7d87bcd5fd-kw5ft\" (UID: \"e9326b0d-0c69-428c-8806-3dd6792c0646\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-kw5ft"
Oct 30 18:38:30.689342 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:30.689317    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-kw5ft"
Oct 30 18:38:31.125358 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:31.125331    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-kw5ft"]
Oct 30 18:38:31.126169 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:38:31.126145    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9326b0d_0c69_428c_8806_3dd6792c0646.slice/crio-614d8d5f2f3c3bc09bc161119b380a3d8c421145cab492b4755aeb88c851cd56 WatchSource:0}: Error finding container 614d8d5f2f3c3bc09bc161119b380a3d8c421145cab492b4755aeb88c851cd56: Status 404 returned error can't find the container with id 614d8d5f2f3c3bc09bc161119b380a3d8c421145cab492b4755aeb88c851cd56
Oct 30 18:38:31.219964 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:31.219936    4888 generic.go:356] "Generic (PLEG): container finished" podID="1b42d455-85e3-45de-ab37-79a951e4c6dc" containerID="969462fa659ac46354adf97bc2d257f8aa7f78874fa1d5fa1facc0691d8db0ea" exitCode=0
Oct 30 18:38:31.220368 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:31.220017    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-67f4db5dc9-nqp9q" event={"ID":"1b42d455-85e3-45de-ab37-79a951e4c6dc","Type":"ContainerDied","Data":"969462fa659ac46354adf97bc2d257f8aa7f78874fa1d5fa1facc0691d8db0ea"}
Oct 30 18:38:31.223150 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:31.223105    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-kw5ft" event={"ID":"e9326b0d-0c69-428c-8806-3dd6792c0646","Type":"ContainerStarted","Data":"614d8d5f2f3c3bc09bc161119b380a3d8c421145cab492b4755aeb88c851cd56"}
Oct 30 18:38:33.235371 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:33.235328    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-67f4db5dc9-nqp9q" event={"ID":"1b42d455-85e3-45de-ab37-79a951e4c6dc","Type":"ContainerStarted","Data":"95b846c454bd68d4f3df114154e32adb540a1d638f2fcda9134457bea7edef9d"}
Oct 30 18:38:33.235842 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:33.235493    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-67f4db5dc9-nqp9q"
Oct 30 18:38:33.501162 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:33.497647    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-67f4db5dc9-nqp9q" podStartSLOduration=3.149601561 podStartE2EDuration="8.497628801s" totalImagesPullingTime="4.899214747s" totalInitContainerRuntime="448.812493ms" isStatefulPod=false podCreationTimestamp="2026-10-30 18:38:25 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:38:33.444525637 +0000 UTC m=+277.691931695" watchObservedRunningTime="2026-10-30 18:38:33.497628801 +0000 UTC m=+277.745034827"
Oct 30 18:38:34.241466 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:34.241422    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-kw5ft" event={"ID":"e9326b0d-0c69-428c-8806-3dd6792c0646","Type":"ContainerStarted","Data":"76f96618973649f83badadf26d1b5d8d0f416bafbf36ed8b4747dd93c349d771"}
Oct 30 18:38:34.366694 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:34.366639    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-kw5ft" podStartSLOduration=2.024145102 podStartE2EDuration="4.366623898s" totalImagesPullingTime="2.342478796s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:38:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:38:34.365478814 +0000 UTC m=+278.612884839" watchObservedRunningTime="2026-10-30 18:38:34.366623898 +0000 UTC m=+278.614029923"
Oct 30 18:38:35.617406 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:38:35.617365    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-jbsw6" containerName="extractor"
Oct 30 18:38:36.966017 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:36.965987    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-8557cbdb84-8dfbp"]
Oct 30 18:38:36.968036 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:36.968021    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-8dfbp"
Oct 30 18:38:37.022069 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:37.022044    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Oct 30 18:38:37.058738 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:37.058714    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r7dfj\" (UniqueName: \"kubernetes.io/projected/d78a842d-920b-4619-9cd7-61d8e86ae880-kube-api-access-r7dfj\") pod \"dns-operator-controller-manager-8557cbdb84-8dfbp\" (UID: \"d78a842d-920b-4619-9cd7-61d8e86ae880\") " pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-8dfbp"
Oct 30 18:38:37.072511 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:37.072492    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Oct 30 18:38:37.072511 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:37.072502    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Oct 30 18:38:37.074303 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:37.074258    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-8557cbdb84-8dfbp"]
Oct 30 18:38:37.078198 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:37.078185    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-9htrl\""
Oct 30 18:38:37.159186 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:37.159160    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r7dfj\" (UniqueName: \"kubernetes.io/projected/d78a842d-920b-4619-9cd7-61d8e86ae880-kube-api-access-r7dfj\") pod \"dns-operator-controller-manager-8557cbdb84-8dfbp\" (UID: \"d78a842d-920b-4619-9cd7-61d8e86ae880\") " pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-8dfbp"
Oct 30 18:38:37.221594 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:37.221536    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r7dfj\" (UniqueName: \"kubernetes.io/projected/d78a842d-920b-4619-9cd7-61d8e86ae880-kube-api-access-r7dfj\") pod \"dns-operator-controller-manager-8557cbdb84-8dfbp\" (UID: \"d78a842d-920b-4619-9cd7-61d8e86ae880\") " pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-8dfbp"
Oct 30 18:38:37.278884 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:37.278862    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-8dfbp"
Oct 30 18:38:37.657132 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:37.657112    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-8557cbdb84-8dfbp"]
Oct 30 18:38:37.659376 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:38:37.659339    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd78a842d_920b_4619_9cd7_61d8e86ae880.slice/crio-60d6a646221483475ad263221ab45d2b7b6c9933c5e979abe23681e13b9085fd WatchSource:0}: Error finding container 60d6a646221483475ad263221ab45d2b7b6c9933c5e979abe23681e13b9085fd: Status 404 returned error can't find the container with id 60d6a646221483475ad263221ab45d2b7b6c9933c5e979abe23681e13b9085fd
Oct 30 18:38:37.819951 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:37.819921    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-5c57bbd4fc-2xhfh"]
Oct 30 18:38:37.822862 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:37.822845    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-2xhfh"
Oct 30 18:38:37.864277 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:37.864247    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g5t6p\" (UniqueName: \"kubernetes.io/projected/4998fa50-93ae-4af0-abe8-2e70d475f40b-kube-api-access-g5t6p\") pod \"cert-manager-cainjector-5c57bbd4fc-2xhfh\" (UID: \"4998fa50-93ae-4af0-abe8-2e70d475f40b\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-2xhfh"
Oct 30 18:38:37.864432 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:37.864326    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4998fa50-93ae-4af0-abe8-2e70d475f40b-bound-sa-token\") pod \"cert-manager-cainjector-5c57bbd4fc-2xhfh\" (UID: \"4998fa50-93ae-4af0-abe8-2e70d475f40b\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-2xhfh"
Oct 30 18:38:37.884663 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:37.884635    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Oct 30 18:38:37.884885 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:37.884865    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Oct 30 18:38:37.884994 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:37.884914    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-4kz5z\""
Oct 30 18:38:37.887944 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:37.887923    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-5c57bbd4fc-2xhfh"]
Oct 30 18:38:37.965746 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:37.965666    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g5t6p\" (UniqueName: \"kubernetes.io/projected/4998fa50-93ae-4af0-abe8-2e70d475f40b-kube-api-access-g5t6p\") pod \"cert-manager-cainjector-5c57bbd4fc-2xhfh\" (UID: \"4998fa50-93ae-4af0-abe8-2e70d475f40b\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-2xhfh"
Oct 30 18:38:37.965915 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:37.965752    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4998fa50-93ae-4af0-abe8-2e70d475f40b-bound-sa-token\") pod \"cert-manager-cainjector-5c57bbd4fc-2xhfh\" (UID: \"4998fa50-93ae-4af0-abe8-2e70d475f40b\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-2xhfh"
Oct 30 18:38:38.033814 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:38.033782    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4998fa50-93ae-4af0-abe8-2e70d475f40b-bound-sa-token\") pod \"cert-manager-cainjector-5c57bbd4fc-2xhfh\" (UID: \"4998fa50-93ae-4af0-abe8-2e70d475f40b\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-2xhfh"
Oct 30 18:38:38.034139 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:38.034058    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g5t6p\" (UniqueName: \"kubernetes.io/projected/4998fa50-93ae-4af0-abe8-2e70d475f40b-kube-api-access-g5t6p\") pod \"cert-manager-cainjector-5c57bbd4fc-2xhfh\" (UID: \"4998fa50-93ae-4af0-abe8-2e70d475f40b\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-2xhfh"
Oct 30 18:38:38.155694 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:38.155661    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-2xhfh"
Oct 30 18:38:38.260659 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:38.260591    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-8dfbp" event={"ID":"d78a842d-920b-4619-9cd7-61d8e86ae880","Type":"ContainerStarted","Data":"60d6a646221483475ad263221ab45d2b7b6c9933c5e979abe23681e13b9085fd"}
Oct 30 18:38:38.538926 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:38.536876    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-5c57bbd4fc-2xhfh"]
Oct 30 18:38:39.265909 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:39.265868    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-2xhfh" event={"ID":"4998fa50-93ae-4af0-abe8-2e70d475f40b","Type":"ContainerStarted","Data":"6794fd98b58699f981441fce1a7455704c4873d97f0bd84883190856d25eff27"}
Oct 30 18:38:41.742871 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:41.742800    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-7b7bb4bc77-rvtlv"]
Oct 30 18:38:41.745707 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:41.745676    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-7b7bb4bc77-rvtlv"
Oct 30 18:38:41.795357 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:41.795306    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hkhnq\" (UniqueName: \"kubernetes.io/projected/ea157aed-6416-4a50-af9f-c9ebbc6ee044-kube-api-access-hkhnq\") pod \"authorino-operator-7b7bb4bc77-rvtlv\" (UID: \"ea157aed-6416-4a50-af9f-c9ebbc6ee044\") " pod="kuadrant-system/authorino-operator-7b7bb4bc77-rvtlv"
Oct 30 18:38:41.804261 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:41.804229    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-sskpr\""
Oct 30 18:38:41.857008 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:41.856977    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-7b7bb4bc77-rvtlv"]
Oct 30 18:38:41.895643 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:41.895608    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hkhnq\" (UniqueName: \"kubernetes.io/projected/ea157aed-6416-4a50-af9f-c9ebbc6ee044-kube-api-access-hkhnq\") pod \"authorino-operator-7b7bb4bc77-rvtlv\" (UID: \"ea157aed-6416-4a50-af9f-c9ebbc6ee044\") " pod="kuadrant-system/authorino-operator-7b7bb4bc77-rvtlv"
Oct 30 18:38:41.962929 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:41.962897    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hkhnq\" (UniqueName: \"kubernetes.io/projected/ea157aed-6416-4a50-af9f-c9ebbc6ee044-kube-api-access-hkhnq\") pod \"authorino-operator-7b7bb4bc77-rvtlv\" (UID: \"ea157aed-6416-4a50-af9f-c9ebbc6ee044\") " pod="kuadrant-system/authorino-operator-7b7bb4bc77-rvtlv"
Oct 30 18:38:42.058802 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:42.058713    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-7b7bb4bc77-rvtlv"
Oct 30 18:38:42.284444 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:42.284400    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-8dfbp" event={"ID":"d78a842d-920b-4619-9cd7-61d8e86ae880","Type":"ContainerStarted","Data":"61abd235fcf9baf562f5cd50e5df56176dbab1a6d3eeab155d03acb9a2f9fd8e"}
Oct 30 18:38:42.284629 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:42.284501    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-8dfbp"
Oct 30 18:38:42.406354 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:42.406305    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-8dfbp" podStartSLOduration=2.677337545 podStartE2EDuration="6.406272027s" totalImagesPullingTime="3.728934482s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:38:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:38:42.404934812 +0000 UTC m=+286.652340837" watchObservedRunningTime="2026-10-30 18:38:42.406272027 +0000 UTC m=+286.653678051"
Oct 30 18:38:42.439185 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:42.439165    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-7b7bb4bc77-rvtlv"]
Oct 30 18:38:42.460361 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:38:42.460339    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea157aed_6416_4a50_af9f_c9ebbc6ee044.slice/crio-489d0edb7b8290ba6894137435ad810bc28949b41fc90e36a67b54a6f4b88c1b WatchSource:0}: Error finding container 489d0edb7b8290ba6894137435ad810bc28949b41fc90e36a67b54a6f4b88c1b: Status 404 returned error can't find the container with id 489d0edb7b8290ba6894137435ad810bc28949b41fc90e36a67b54a6f4b88c1b
Oct 30 18:38:43.291101 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:43.291070    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-7b7bb4bc77-rvtlv" event={"ID":"ea157aed-6416-4a50-af9f-c9ebbc6ee044","Type":"ContainerStarted","Data":"489d0edb7b8290ba6894137435ad810bc28949b41fc90e36a67b54a6f4b88c1b"}
Oct 30 18:38:43.292778 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:43.292751    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-2xhfh" event={"ID":"4998fa50-93ae-4af0-abe8-2e70d475f40b","Type":"ContainerStarted","Data":"030eedbc260d730f8ddceb1152f13865b0727750ee26adaac5f7520a35c99922"}
Oct 30 18:38:43.423131 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:43.423078    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-2xhfh" podStartSLOduration=2.5384263369999998 podStartE2EDuration="6.423065297s" totalImagesPullingTime="3.88463896s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:38:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:38:43.422112322 +0000 UTC m=+287.669518365" watchObservedRunningTime="2026-10-30 18:38:43.423065297 +0000 UTC m=+287.670471321"
Oct 30 18:38:44.244452 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:44.244417    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-67f4db5dc9-nqp9q"
Oct 30 18:38:45.303997 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:45.303956    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-7b7bb4bc77-rvtlv" event={"ID":"ea157aed-6416-4a50-af9f-c9ebbc6ee044","Type":"ContainerStarted","Data":"162e799dab359d3cd830c43e1419976b5e44ca3b1894751c664cc60b64874f00"}
Oct 30 18:38:45.304402 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:45.304093    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-7b7bb4bc77-rvtlv"
Oct 30 18:38:45.426612 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:45.426521    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-7b7bb4bc77-rvtlv" podStartSLOduration=1.8113617739999999 podStartE2EDuration="4.426509038s" totalImagesPullingTime="2.615147264s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:38:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:38:45.424800709 +0000 UTC m=+289.672206734" watchObservedRunningTime="2026-10-30 18:38:45.426509038 +0000 UTC m=+289.673915065"
Oct 30 18:38:47.062602 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:47.062563    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-5778bb4449-f9rvt"]
Oct 30 18:38:47.065298 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:47.065260    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-f9rvt"
Oct 30 18:38:47.142653 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:47.142617    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g66tl\" (UniqueName: \"kubernetes.io/projected/db5d7a86-9e39-413e-8628-9a7e77270908-kube-api-access-g66tl\") pod \"limitador-operator-controller-manager-5778bb4449-f9rvt\" (UID: \"db5d7a86-9e39-413e-8628-9a7e77270908\") " pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-f9rvt"
Oct 30 18:38:47.176989 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:47.176959    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-n9gnx\""
Oct 30 18:38:47.178856 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:47.178829    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-5778bb4449-f9rvt"]
Oct 30 18:38:47.243942 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:47.243919    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g66tl\" (UniqueName: \"kubernetes.io/projected/db5d7a86-9e39-413e-8628-9a7e77270908-kube-api-access-g66tl\") pod \"limitador-operator-controller-manager-5778bb4449-f9rvt\" (UID: \"db5d7a86-9e39-413e-8628-9a7e77270908\") " pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-f9rvt"
Oct 30 18:38:47.316962 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:47.316885    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g66tl\" (UniqueName: \"kubernetes.io/projected/db5d7a86-9e39-413e-8628-9a7e77270908-kube-api-access-g66tl\") pod \"limitador-operator-controller-manager-5778bb4449-f9rvt\" (UID: \"db5d7a86-9e39-413e-8628-9a7e77270908\") " pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-f9rvt"
Oct 30 18:38:47.377629 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:47.377602    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-f9rvt"
Oct 30 18:38:47.769746 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:47.769712    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-5778bb4449-f9rvt"]
Oct 30 18:38:47.797947 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:38:47.797920    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb5d7a86_9e39_413e_8628_9a7e77270908.slice/crio-85264a1a8b3db86725cd21206082f6f40606eb859c717bfebe102f33d68636e8 WatchSource:0}: Error finding container 85264a1a8b3db86725cd21206082f6f40606eb859c717bfebe102f33d68636e8: Status 404 returned error can't find the container with id 85264a1a8b3db86725cd21206082f6f40606eb859c717bfebe102f33d68636e8
Oct 30 18:38:48.323000 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:48.322965    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-f9rvt" event={"ID":"db5d7a86-9e39-413e-8628-9a7e77270908","Type":"ContainerStarted","Data":"85264a1a8b3db86725cd21206082f6f40606eb859c717bfebe102f33d68636e8"}
Oct 30 18:38:51.347509 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:51.347476    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-f9rvt" event={"ID":"db5d7a86-9e39-413e-8628-9a7e77270908","Type":"ContainerStarted","Data":"9d359c81bb3d5d5ed625cfab0039aea00b74112bc31dd3791427e92ace556bb9"}
Oct 30 18:38:51.347955 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:51.347579    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-f9rvt"
Oct 30 18:38:51.473787 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:51.473736    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-f9rvt" podStartSLOduration=1.599552562 podStartE2EDuration="4.473724162s" totalImagesPullingTime="2.8741716s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:38:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:38:51.472106394 +0000 UTC m=+295.719512419" watchObservedRunningTime="2026-10-30 18:38:51.473724162 +0000 UTC m=+295.721130187"
Oct 30 18:38:53.295509 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:53.295477    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-8dfbp"
Oct 30 18:38:53.325225 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:53.325190    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-678d87766f-mx24q"]
Oct 30 18:38:53.327777 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:53.327758    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-mx24q"
Oct 30 18:38:53.395353 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:53.395324    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8cqrp\" (UniqueName: \"kubernetes.io/projected/e31ecc7e-2afe-4777-9ae4-50954433a58e-kube-api-access-8cqrp\") pod \"kuadrant-operator-controller-manager-678d87766f-mx24q\" (UID: \"e31ecc7e-2afe-4777-9ae4-50954433a58e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-mx24q"
Oct 30 18:38:53.395488 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:53.395367    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e31ecc7e-2afe-4777-9ae4-50954433a58e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-678d87766f-mx24q\" (UID: \"e31ecc7e-2afe-4777-9ae4-50954433a58e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-mx24q"
Oct 30 18:38:53.400262 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:53.400232    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-jkvtc\""
Oct 30 18:38:53.402085 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:53.402068    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-678d87766f-mx24q"]
Oct 30 18:38:53.496554 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:53.496526    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8cqrp\" (UniqueName: \"kubernetes.io/projected/e31ecc7e-2afe-4777-9ae4-50954433a58e-kube-api-access-8cqrp\") pod \"kuadrant-operator-controller-manager-678d87766f-mx24q\" (UID: \"e31ecc7e-2afe-4777-9ae4-50954433a58e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-mx24q"
Oct 30 18:38:53.496718 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:53.496575    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e31ecc7e-2afe-4777-9ae4-50954433a58e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-678d87766f-mx24q\" (UID: \"e31ecc7e-2afe-4777-9ae4-50954433a58e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-mx24q"
Oct 30 18:38:53.496962 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:53.496945    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e31ecc7e-2afe-4777-9ae4-50954433a58e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-678d87766f-mx24q\" (UID: \"e31ecc7e-2afe-4777-9ae4-50954433a58e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-mx24q"
Oct 30 18:38:53.558835 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:53.558775    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8cqrp\" (UniqueName: \"kubernetes.io/projected/e31ecc7e-2afe-4777-9ae4-50954433a58e-kube-api-access-8cqrp\") pod \"kuadrant-operator-controller-manager-678d87766f-mx24q\" (UID: \"e31ecc7e-2afe-4777-9ae4-50954433a58e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-mx24q"
Oct 30 18:38:53.638874 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:53.638848    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-mx24q"
Oct 30 18:38:54.067722 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:54.067697    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-678d87766f-mx24q"]
Oct 30 18:38:54.069066 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:38:54.069042    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode31ecc7e_2afe_4777_9ae4_50954433a58e.slice/crio-620ee337e8b847e6a825b06f31d4ad23f5f4cb7c3e0aad349e1499e608272b83 WatchSource:0}: Error finding container 620ee337e8b847e6a825b06f31d4ad23f5f4cb7c3e0aad349e1499e608272b83: Status 404 returned error can't find the container with id 620ee337e8b847e6a825b06f31d4ad23f5f4cb7c3e0aad349e1499e608272b83
Oct 30 18:38:54.362713 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:54.362678    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-mx24q" event={"ID":"e31ecc7e-2afe-4777-9ae4-50954433a58e","Type":"ContainerStarted","Data":"620ee337e8b847e6a825b06f31d4ad23f5f4cb7c3e0aad349e1499e608272b83"}
Oct 30 18:38:55.908660 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:55.908631    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-7x4rb_382e8b31-a7fa-43f3-9f15-4c61627b0c04/console-operator/2.log"
Oct 30 18:38:55.909144 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:55.908836    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-7x4rb_382e8b31-a7fa-43f3-9f15-4c61627b0c04/console-operator/2.log"
Oct 30 18:38:55.916469 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:55.916447    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xfcdl_3f3fe764-2652-46c9-8e54-7882c0e6bcf1/ovn-acl-logging/0.log"
Oct 30 18:38:55.916718 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:55.916704    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xfcdl_3f3fe764-2652-46c9-8e54-7882c0e6bcf1/ovn-acl-logging/0.log"
Oct 30 18:38:56.313476 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:38:56.313411    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-7b7bb4bc77-rvtlv"
Oct 30 18:39:00.395545 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:39:00.395510    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-mx24q" event={"ID":"e31ecc7e-2afe-4777-9ae4-50954433a58e","Type":"ContainerStarted","Data":"a3fc5f2a7a80619e55c355916cb70040f34b6d55882df15c35a40c5d42e68135"}
Oct 30 18:39:00.395945 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:39:00.395695    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-mx24q"
Oct 30 18:39:00.620814 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:39:00.620768    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-mx24q" podStartSLOduration=1.8300473529999999 podStartE2EDuration="7.620754045s" totalImagesPullingTime="5.790706692s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:38:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:39:00.568592031 +0000 UTC m=+304.815998057" watchObservedRunningTime="2026-10-30 18:39:00.620754045 +0000 UTC m=+304.868160070"
Oct 30 18:39:01.509493 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:39:01.509461    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-8f76c657-bfbg6"]
Oct 30 18:39:01.517354 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:39:01.517321    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-8f76c657-bfbg6"
Oct 30 18:39:01.567767 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:39:01.567735    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h56qb\" (UniqueName: \"kubernetes.io/projected/e1f8438b-6784-4805-adbb-9d45026b12b6-kube-api-access-h56qb\") pod \"data-science-gateway-data-science-gateway-class-8f76c657-bfbg6\" (UID: \"e1f8438b-6784-4805-adbb-9d45026b12b6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-8f76c657-bfbg6"
Oct 30 18:39:01.567936 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:39:01.567780    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e1f8438b-6784-4805-adbb-9d45026b12b6-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-8f76c657-bfbg6\" (UID: \"e1f8438b-6784-4805-adbb-9d45026b12b6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-8f76c657-bfbg6"
Oct 30 18:39:01.567936 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:39:01.567863    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e1f8438b-6784-4805-adbb-9d45026b12b6-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-8f76c657-bfbg6\" (UID: \"e1f8438b-6784-4805-adbb-9d45026b12b6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-8f76c657-bfbg6"
Oct 30 18:39:01.567936 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:39:01.567901    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e1f8438b-6784-4805-adbb-9d45026b12b6-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-8f76c657-bfbg6\" (UID: \"e1f8438b-6784-4805-adbb-9d45026b12b6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-8f76c657-bfbg6"
Oct 30 18:39:01.567936 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:39:01.567925    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e1f8438b-6784-4805-adbb-9d45026b12b6-istio-token\") pod \"data-science-gateway-data-science-gateway-class-8f76c657-bfbg6\" (UID: \"e1f8438b-6784-4805-adbb-9d45026b12b6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-8f76c657-bfbg6"
Oct 30 18:39:01.568156 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:39:01.567998    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e1f8438b-6784-4805-adbb-9d45026b12b6-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-8f76c657-bfbg6\" (UID: \"e1f8438b-6784-4805-adbb-9d45026b12b6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-8f76c657-bfbg6"
Oct 30 18:39:01.568156 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:39:01.568036    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e1f8438b-6784-4805-adbb-9d45026b12b6-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-8f76c657-bfbg6\" (UID: \"e1f8438b-6784-4805-adbb-9d45026b12b6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-8f76c657-bfbg6"
Oct 30 18:39:01.568156 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:39:01.568071    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e1f8438b-6784-4805-adbb-9d45026b12b6-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-8f76c657-bfbg6\" (UID: \"e1f8438b-6784-4805-adbb-9d45026b12b6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-8f76c657-bfbg6"
Oct 30 18:39:01.568156 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:39:01.568086    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e1f8438b-6784-4805-adbb-9d45026b12b6-istio-data\") pod \"data-science-gateway-data-science-gateway-class-8f76c657-bfbg6\" (UID: \"e1f8438b-6784-4805-adbb-9d45026b12b6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-8f76c657-bfbg6"
Oct 30 18:39:01.592971 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:39:01.592942    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-8f76c657-bfbg6"]
Oct 30 18:39:01.600237 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:39:01.600214    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-5mz2r\""
Oct 30 18:39:01.668455 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:39:01.668425    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e1f8438b-6784-4805-adbb-9d45026b12b6-istio-token\") pod \"data-science-gateway-data-science-gateway-class-8f76c657-bfbg6\" (UID: \"e1f8438b-6784-4805-adbb-9d45026b12b6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-8f76c657-bfbg6"
Oct 30 18:39:01.668651 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:39:01.668462    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e1f8438b-6784-4805-adbb-9d45026b12b6-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-8f76c657-bfbg6\" (UID: \"e1f8438b-6784-4805-adbb-9d45026b12b6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-8f76c657-bfbg6"
Oct 30 18:39:01.668651 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:39:01.668481    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e1f8438b-6784-4805-adbb-9d45026b12b6-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-8f76c657-bfbg6\" (UID: \"e1f8438b-6784-4805-adbb-9d45026b12b6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-8f76c657-bfbg6"
Oct 30 18:39:01.668651 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:39:01.668525    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e1f8438b-6784-4805-adbb-9d45026b12b6-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-8f76c657-bfbg6\" (UID: \"e1f8438b-6784-4805-adbb-9d45026b12b6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-8f76c657-bfbg6"
Oct 30 18:39:01.668651 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:39:01.668552    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e1f8438b-6784-4805-adbb-9d45026b12b6-istio-data\") pod \"data-science-gateway-data-science-gateway-class-8f76c657-bfbg6\" (UID: \"e1f8438b-6784-4805-adbb-9d45026b12b6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-8f76c657-bfbg6"
Oct 30 18:39:01.668821 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:39:01.668723    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h56qb\" (UniqueName: \"kubernetes.io/projected/e1f8438b-6784-4805-adbb-9d45026b12b6-kube-api-access-h56qb\") pod \"data-science-gateway-data-science-gateway-class-8f76c657-bfbg6\" (UID: \"e1f8438b-6784-4805-adbb-9d45026b12b6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-8f76c657-bfbg6"
Oct 30 18:39:01.668821 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:39:01.668771    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e1f8438b-6784-4805-adbb-9d45026b12b6-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-8f76c657-bfbg6\" (UID: \"e1f8438b-6784-4805-adbb-9d45026b12b6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-8f76c657-bfbg6"
Oct 30 18:39:01.668896 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:39:01.668817    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e1f8438b-6784-4805-adbb-9d45026b12b6-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-8f76c657-bfbg6\" (UID: \"e1f8438b-6784-4805-adbb-9d45026b12b6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-8f76c657-bfbg6"
Oct 30 18:39:01.668896 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:39:01.668853    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e1f8438b-6784-4805-adbb-9d45026b12b6-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-8f76c657-bfbg6\" (UID: \"e1f8438b-6784-4805-adbb-9d45026b12b6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-8f76c657-bfbg6"
Oct 30 18:39:01.669345 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:39:01.669319    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e1f8438b-6784-4805-adbb-9d45026b12b6-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-8f76c657-bfbg6\" (UID: \"e1f8438b-6784-4805-adbb-9d45026b12b6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-8f76c657-bfbg6"
Oct 30 18:39:01.669443 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:39:01.669423    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e1f8438b-6784-4805-adbb-9d45026b12b6-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-8f76c657-bfbg6\" (UID: \"e1f8438b-6784-4805-adbb-9d45026b12b6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-8f76c657-bfbg6"
Oct 30 18:39:01.669503 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:39:01.669455    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e1f8438b-6784-4805-adbb-9d45026b12b6-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-8f76c657-bfbg6\" (UID: \"e1f8438b-6784-4805-adbb-9d45026b12b6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-8f76c657-bfbg6"
Oct 30 18:39:01.669503 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:39:01.669468    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e1f8438b-6784-4805-adbb-9d45026b12b6-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-8f76c657-bfbg6\" (UID: \"e1f8438b-6784-4805-adbb-9d45026b12b6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-8f76c657-bfbg6"
Oct 30 18:39:01.669596 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:39:01.669541    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e1f8438b-6784-4805-adbb-9d45026b12b6-istio-data\") pod \"data-science-gateway-data-science-gateway-class-8f76c657-bfbg6\" (UID: \"e1f8438b-6784-4805-adbb-9d45026b12b6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-8f76c657-bfbg6"
Oct 30 18:39:01.672276 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:39:01.672249    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e1f8438b-6784-4805-adbb-9d45026b12b6-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-8f76c657-bfbg6\" (UID: \"e1f8438b-6784-4805-adbb-9d45026b12b6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-8f76c657-bfbg6"
Oct 30 18:39:01.672449 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:39:01.672378    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e1f8438b-6784-4805-adbb-9d45026b12b6-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-8f76c657-bfbg6\" (UID: \"e1f8438b-6784-4805-adbb-9d45026b12b6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-8f76c657-bfbg6"
Oct 30 18:39:01.750623 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:39:01.750595    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h56qb\" (UniqueName: \"kubernetes.io/projected/e1f8438b-6784-4805-adbb-9d45026b12b6-kube-api-access-h56qb\") pod \"data-science-gateway-data-science-gateway-class-8f76c657-bfbg6\" (UID: \"e1f8438b-6784-4805-adbb-9d45026b12b6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-8f76c657-bfbg6"
Oct 30 18:39:01.751324 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:39:01.751302    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e1f8438b-6784-4805-adbb-9d45026b12b6-istio-token\") pod \"data-science-gateway-data-science-gateway-class-8f76c657-bfbg6\" (UID: \"e1f8438b-6784-4805-adbb-9d45026b12b6\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-8f76c657-bfbg6"
Oct 30 18:39:01.830883 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:39:01.830846    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-8f76c657-bfbg6"
Oct 30 18:39:02.257981 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:39:02.257904    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-8f76c657-bfbg6"]
Oct 30 18:39:02.295312 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:39:02.295256    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1f8438b_6784_4805_adbb_9d45026b12b6.slice/crio-10a9be358be358216f7b3e79848bf177c726408078f8f8cfb5a1e93c3c938f58 WatchSource:0}: Error finding container 10a9be358be358216f7b3e79848bf177c726408078f8f8cfb5a1e93c3c938f58: Status 404 returned error can't find the container with id 10a9be358be358216f7b3e79848bf177c726408078f8f8cfb5a1e93c3c938f58
Oct 30 18:39:02.297444 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:39:02.297410    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:39:02.297540 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:39:02.297484    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:39:02.297540 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:39:02.297514    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:39:02.354852 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:39:02.354826    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-5778bb4449-f9rvt"
Oct 30 18:39:02.408240 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:39:02.408191    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-8f76c657-bfbg6" event={"ID":"e1f8438b-6784-4805-adbb-9d45026b12b6","Type":"ContainerStarted","Data":"0f39bd9996315eba0d85c39127c17d865227f258a6a9996c120ebfa0b1240593"}
Oct 30 18:39:02.408462 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:39:02.408276    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-8f76c657-bfbg6" event={"ID":"e1f8438b-6784-4805-adbb-9d45026b12b6","Type":"ContainerStarted","Data":"10a9be358be358216f7b3e79848bf177c726408078f8f8cfb5a1e93c3c938f58"}
Oct 30 18:39:02.605077 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:39:02.605026    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-8f76c657-bfbg6" podStartSLOduration=1.6050129640000002 podStartE2EDuration="1.605012964s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:39:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:39:02.603643588 +0000 UTC m=+306.851049612" watchObservedRunningTime="2026-10-30 18:39:02.605012964 +0000 UTC m=+306.852419032"
Oct 30 18:39:02.831136 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:39:02.831101    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-8f76c657-bfbg6"
Oct 30 18:39:03.835634 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:39:03.835608    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-8f76c657-bfbg6"
Oct 30 18:39:04.418569 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:39:04.418538    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-8f76c657-bfbg6"
Oct 30 18:39:04.419541 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:39:04.419522    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-8f76c657-bfbg6"
Oct 30 18:39:05.617590 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:39:05.617520    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-jbsw6" containerName="extractor"
Oct 30 18:39:10.406600 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:39:10.406569    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-57f7f8659f-6j5cr"]
Oct 30 18:39:10.410347 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:39:10.410331    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-57f7f8659f-6j5cr"
Oct 30 18:39:10.470690 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:39:10.470657    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-57f7f8659f-6j5cr"]
Oct 30 18:39:10.518680 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:39:10.518656    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-fl2x7\""
Oct 30 18:39:10.547383 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:39:10.547361    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4wwwt\" (UniqueName: \"kubernetes.io/projected/d0636781-6560-4ec3-82cf-140d88b71e1f-kube-api-access-4wwwt\") pod \"authorino-57f7f8659f-6j5cr\" (UID: \"d0636781-6560-4ec3-82cf-140d88b71e1f\") " pod="kuadrant-system/authorino-57f7f8659f-6j5cr"
Oct 30 18:39:10.648526 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:39:10.648498    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4wwwt\" (UniqueName: \"kubernetes.io/projected/d0636781-6560-4ec3-82cf-140d88b71e1f-kube-api-access-4wwwt\") pod \"authorino-57f7f8659f-6j5cr\" (UID: \"d0636781-6560-4ec3-82cf-140d88b71e1f\") " pod="kuadrant-system/authorino-57f7f8659f-6j5cr"
Oct 30 18:39:10.716257 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:39:10.716203    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4wwwt\" (UniqueName: \"kubernetes.io/projected/d0636781-6560-4ec3-82cf-140d88b71e1f-kube-api-access-4wwwt\") pod \"authorino-57f7f8659f-6j5cr\" (UID: \"d0636781-6560-4ec3-82cf-140d88b71e1f\") " pod="kuadrant-system/authorino-57f7f8659f-6j5cr"
Oct 30 18:39:10.730027 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:39:10.730007    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-57f7f8659f-6j5cr"
Oct 30 18:39:11.103776 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:39:11.103748    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-57f7f8659f-6j5cr"]
Oct 30 18:39:11.105157 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:39:11.105129    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd0636781_6560_4ec3_82cf_140d88b71e1f.slice/crio-f1835694ba7cad7f52f75b0a08a366619ef4ecfbaaeb16173ff59ba194fd7437 WatchSource:0}: Error finding container f1835694ba7cad7f52f75b0a08a366619ef4ecfbaaeb16173ff59ba194fd7437: Status 404 returned error can't find the container with id f1835694ba7cad7f52f75b0a08a366619ef4ecfbaaeb16173ff59ba194fd7437
Oct 30 18:39:11.106529 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:39:11.106511    4888 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 18:39:11.403864 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:39:11.403794    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-mx24q"
Oct 30 18:39:11.451835 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:39:11.451803    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-57f7f8659f-6j5cr" event={"ID":"d0636781-6560-4ec3-82cf-140d88b71e1f","Type":"ContainerStarted","Data":"f1835694ba7cad7f52f75b0a08a366619ef4ecfbaaeb16173ff59ba194fd7437"}
Oct 30 18:39:13.415858 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:39:13.415823    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-rjmdz"]
Oct 30 18:39:13.439541 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:39:13.439503    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-7c9b87d574-rjmdz"
Oct 30 18:39:13.483419 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:39:13.483388    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-rjmdz"]
Oct 30 18:39:13.531853 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:39:13.531827    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Oct 30 18:39:13.574636 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:39:13.574605    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/513b0b3f-06d8-42bf-b73c-89f3bae42d98-config-file\") pod \"limitador-limitador-7c9b87d574-rjmdz\" (UID: \"513b0b3f-06d8-42bf-b73c-89f3bae42d98\") " pod="kuadrant-system/limitador-limitador-7c9b87d574-rjmdz"
Oct 30 18:39:13.574772 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:39:13.574643    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zvvrd\" (UniqueName: \"kubernetes.io/projected/513b0b3f-06d8-42bf-b73c-89f3bae42d98-kube-api-access-zvvrd\") pod \"limitador-limitador-7c9b87d574-rjmdz\" (UID: \"513b0b3f-06d8-42bf-b73c-89f3bae42d98\") " pod="kuadrant-system/limitador-limitador-7c9b87d574-rjmdz"
Oct 30 18:39:13.586095 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:39:13.586074    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-4m2vx\""
Oct 30 18:39:13.675821 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:39:13.675739    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/513b0b3f-06d8-42bf-b73c-89f3bae42d98-config-file\") pod \"limitador-limitador-7c9b87d574-rjmdz\" (UID: \"513b0b3f-06d8-42bf-b73c-89f3bae42d98\") " pod="kuadrant-system/limitador-limitador-7c9b87d574-rjmdz"
Oct 30 18:39:13.675821 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:39:13.675797    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zvvrd\" (UniqueName: \"kubernetes.io/projected/513b0b3f-06d8-42bf-b73c-89f3bae42d98-kube-api-access-zvvrd\") pod \"limitador-limitador-7c9b87d574-rjmdz\" (UID: \"513b0b3f-06d8-42bf-b73c-89f3bae42d98\") " pod="kuadrant-system/limitador-limitador-7c9b87d574-rjmdz"
Oct 30 18:39:13.676603 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:39:13.676573    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/513b0b3f-06d8-42bf-b73c-89f3bae42d98-config-file\") pod \"limitador-limitador-7c9b87d574-rjmdz\" (UID: \"513b0b3f-06d8-42bf-b73c-89f3bae42d98\") " pod="kuadrant-system/limitador-limitador-7c9b87d574-rjmdz"
Oct 30 18:39:13.727151 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:39:13.727124    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-rjmdz"]
Oct 30 18:39:13.738521 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:39:13.738496    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zvvrd\" (UniqueName: \"kubernetes.io/projected/513b0b3f-06d8-42bf-b73c-89f3bae42d98-kube-api-access-zvvrd\") pod \"limitador-limitador-7c9b87d574-rjmdz\" (UID: \"513b0b3f-06d8-42bf-b73c-89f3bae42d98\") " pod="kuadrant-system/limitador-limitador-7c9b87d574-rjmdz"
Oct 30 18:39:13.774594 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:39:13.774567    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-7c9b87d574-rjmdz"
Oct 30 18:39:14.163388 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:39:14.163360    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-rjmdz"]
Oct 30 18:39:14.165145 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:39:14.165112    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod513b0b3f_06d8_42bf_b73c_89f3bae42d98.slice/crio-ec8681a4a396909e50081b4e4a5bc83fa12ae9f2b3df4a4156872638c4cbae1d WatchSource:0}: Error finding container ec8681a4a396909e50081b4e4a5bc83fa12ae9f2b3df4a4156872638c4cbae1d: Status 404 returned error can't find the container with id ec8681a4a396909e50081b4e4a5bc83fa12ae9f2b3df4a4156872638c4cbae1d
Oct 30 18:39:14.468544 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:39:14.468462    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-7c9b87d574-rjmdz" event={"ID":"513b0b3f-06d8-42bf-b73c-89f3bae42d98","Type":"ContainerStarted","Data":"ec8681a4a396909e50081b4e4a5bc83fa12ae9f2b3df4a4156872638c4cbae1d"}
Oct 30 18:39:14.469966 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:39:14.469928    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-57f7f8659f-6j5cr" event={"ID":"d0636781-6560-4ec3-82cf-140d88b71e1f","Type":"ContainerStarted","Data":"7aef495c6cf7f0d985a82337a4c779cee38c7318f928782b0fccd36c562edf00"}
Oct 30 18:39:14.589801 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:39:14.589751    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/authorino-57f7f8659f-6j5cr" podStartSLOduration=1.369411111 podStartE2EDuration="4.589738227s" totalImagesPullingTime="3.220327116s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:39:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:39:14.588439406 +0000 UTC m=+318.835845432" watchObservedRunningTime="2026-10-30 18:39:14.589738227 +0000 UTC m=+318.837144251"
Oct 30 18:39:17.491869 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:39:17.491833    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-7c9b87d574-rjmdz" event={"ID":"513b0b3f-06d8-42bf-b73c-89f3bae42d98","Type":"ContainerStarted","Data":"bef8dd7ec0e15aa3e0babb0282eb0d211e73b3cdee07f61b7f9b4e5020f0b14e"}
Oct 30 18:39:17.492273 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:39:17.491896    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-7c9b87d574-rjmdz"
Oct 30 18:39:17.615363 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:39:17.615248    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-7c9b87d574-rjmdz" podStartSLOduration=1.447393095 podStartE2EDuration="4.615235704s" totalImagesPullingTime="3.167842609s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:39:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:39:17.615075423 +0000 UTC m=+321.862481448" watchObservedRunningTime="2026-10-30 18:39:17.615235704 +0000 UTC m=+321.862641726"
Oct 30 18:39:17.830980 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:39:17.830949    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-57f7f8659f-6j5cr"]
Oct 30 18:39:17.831171 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:39:17.831103    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/authorino-57f7f8659f-6j5cr" podUID="d0636781-6560-4ec3-82cf-140d88b71e1f" containerName="authorino" containerID="cri-o://7aef495c6cf7f0d985a82337a4c779cee38c7318f928782b0fccd36c562edf00" gracePeriod=30
Oct 30 18:39:18.071506 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:39:18.071482    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-57f7f8659f-6j5cr"
Oct 30 18:39:18.216415 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:39:18.216344    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d0636781-6560-4ec3-82cf-140d88b71e1f-kube-api-access-4wwwt\" (UniqueName: \"kubernetes.io/projected/d0636781-6560-4ec3-82cf-140d88b71e1f-kube-api-access-4wwwt\") pod \"d0636781-6560-4ec3-82cf-140d88b71e1f\" (UID: \"d0636781-6560-4ec3-82cf-140d88b71e1f\") "
Oct 30 18:39:18.218643 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:39:18.218597    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d0636781-6560-4ec3-82cf-140d88b71e1f-kube-api-access-4wwwt" pod "d0636781-6560-4ec3-82cf-140d88b71e1f" (UID: "d0636781-6560-4ec3-82cf-140d88b71e1f"). InnerVolumeSpecName "kube-api-access-4wwwt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:39:18.317598 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:39:18.317571    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4wwwt\" (UniqueName: \"kubernetes.io/projected/d0636781-6560-4ec3-82cf-140d88b71e1f-kube-api-access-4wwwt\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:39:18.497653 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:39:18.497571    4888 generic.go:356] "Generic (PLEG): container finished" podID="d0636781-6560-4ec3-82cf-140d88b71e1f" containerID="7aef495c6cf7f0d985a82337a4c779cee38c7318f928782b0fccd36c562edf00" exitCode=0
Oct 30 18:39:18.497653 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:39:18.497614    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-57f7f8659f-6j5cr"
Oct 30 18:39:18.498170 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:39:18.497662    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-57f7f8659f-6j5cr" event={"ID":"d0636781-6560-4ec3-82cf-140d88b71e1f","Type":"ContainerDied","Data":"7aef495c6cf7f0d985a82337a4c779cee38c7318f928782b0fccd36c562edf00"}
Oct 30 18:39:18.498170 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:39:18.497709    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-57f7f8659f-6j5cr" event={"ID":"d0636781-6560-4ec3-82cf-140d88b71e1f","Type":"ContainerDied","Data":"f1835694ba7cad7f52f75b0a08a366619ef4ecfbaaeb16173ff59ba194fd7437"}
Oct 30 18:39:18.498170 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:39:18.497732    4888 scope.go:122] "RemoveContainer" containerID="7aef495c6cf7f0d985a82337a4c779cee38c7318f928782b0fccd36c562edf00"
Oct 30 18:39:18.507268 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:39:18.507251    4888 scope.go:122] "RemoveContainer" containerID="7aef495c6cf7f0d985a82337a4c779cee38c7318f928782b0fccd36c562edf00"
Oct 30 18:39:18.507557 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:39:18.507537    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7aef495c6cf7f0d985a82337a4c779cee38c7318f928782b0fccd36c562edf00\": container with ID starting with 7aef495c6cf7f0d985a82337a4c779cee38c7318f928782b0fccd36c562edf00 not found: ID does not exist" containerID="7aef495c6cf7f0d985a82337a4c779cee38c7318f928782b0fccd36c562edf00"
Oct 30 18:39:18.507644 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:39:18.507562    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7aef495c6cf7f0d985a82337a4c779cee38c7318f928782b0fccd36c562edf00"} err="failed to get container status \"7aef495c6cf7f0d985a82337a4c779cee38c7318f928782b0fccd36c562edf00\": rpc error: code = NotFound desc = could not find container \"7aef495c6cf7f0d985a82337a4c779cee38c7318f928782b0fccd36c562edf00\": container with ID starting with 7aef495c6cf7f0d985a82337a4c779cee38c7318f928782b0fccd36c562edf00 not found: ID does not exist"
Oct 30 18:39:18.666432 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:39:18.666400    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-57f7f8659f-6j5cr"]
Oct 30 18:39:18.670530 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:39:18.670503    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-57f7f8659f-6j5cr"]
Oct 30 18:39:19.888329 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:39:19.888267    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d0636781-6560-4ec3-82cf-140d88b71e1f" path="/var/lib/kubelet/pods/d0636781-6560-4ec3-82cf-140d88b71e1f/volumes"
Oct 30 18:39:28.498694 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:39:28.498667    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-7c9b87d574-rjmdz"
Oct 30 18:39:35.616549 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:39:35.616508    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-jbsw6" containerName="extractor"
Oct 30 18:39:58.376342 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:39:58.376310    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5757c987cd-n6lrf"]
Oct 30 18:39:58.376733 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:39:58.376660    4888 cpu_manager.go:414] "containerMap: removing container" podUID="d0636781-6560-4ec3-82cf-140d88b71e1f" containerName="authorino"
Oct 30 18:39:58.376733 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:39:58.376671    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d0636781-6560-4ec3-82cf-140d88b71e1f" containerName="authorino"
Oct 30 18:39:58.376805 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:39:58.376741    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="d0636781-6560-4ec3-82cf-140d88b71e1f" containerName="authorino"
Oct 30 18:39:58.381841 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:39:58.381825    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5757c987cd-n6lrf"
Oct 30 18:39:58.451680 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:39:58.451644    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1e1a2be4-6d12-483e-907d-9aa9ea15c712-webhook-cert\") pod \"maas-controller-5757c987cd-n6lrf\" (UID: \"1e1a2be4-6d12-483e-907d-9aa9ea15c712\") " pod="opendatahub/maas-controller-5757c987cd-n6lrf"
Oct 30 18:39:58.451844 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:39:58.451688    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1e1a2be4-6d12-483e-907d-9aa9ea15c712-metrics-certs\") pod \"maas-controller-5757c987cd-n6lrf\" (UID: \"1e1a2be4-6d12-483e-907d-9aa9ea15c712\") " pod="opendatahub/maas-controller-5757c987cd-n6lrf"
Oct 30 18:39:58.451892 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:39:58.451845    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-876sl\" (UniqueName: \"kubernetes.io/projected/1e1a2be4-6d12-483e-907d-9aa9ea15c712-kube-api-access-876sl\") pod \"maas-controller-5757c987cd-n6lrf\" (UID: \"1e1a2be4-6d12-483e-907d-9aa9ea15c712\") " pod="opendatahub/maas-controller-5757c987cd-n6lrf"
Oct 30 18:39:58.465900 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:39:58.465878    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Oct 30 18:39:58.466032 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:39:58.465990    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Oct 30 18:39:58.487435 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:39:58.487410    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Oct 30 18:39:58.487748 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:39:58.487730    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Oct 30 18:39:58.487856 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:39:58.487840    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Oct 30 18:39:58.488216 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:39:58.488201    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-7dphn\""
Oct 30 18:39:58.489803 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:39:58.489778    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5757c987cd-n6lrf"]
Oct 30 18:39:58.552329 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:39:58.552275    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-876sl\" (UniqueName: \"kubernetes.io/projected/1e1a2be4-6d12-483e-907d-9aa9ea15c712-kube-api-access-876sl\") pod \"maas-controller-5757c987cd-n6lrf\" (UID: \"1e1a2be4-6d12-483e-907d-9aa9ea15c712\") " pod="opendatahub/maas-controller-5757c987cd-n6lrf"
Oct 30 18:39:58.552492 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:39:58.552356    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1e1a2be4-6d12-483e-907d-9aa9ea15c712-webhook-cert\") pod \"maas-controller-5757c987cd-n6lrf\" (UID: \"1e1a2be4-6d12-483e-907d-9aa9ea15c712\") " pod="opendatahub/maas-controller-5757c987cd-n6lrf"
Oct 30 18:39:58.552492 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:39:58.552382    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1e1a2be4-6d12-483e-907d-9aa9ea15c712-metrics-certs\") pod \"maas-controller-5757c987cd-n6lrf\" (UID: \"1e1a2be4-6d12-483e-907d-9aa9ea15c712\") " pod="opendatahub/maas-controller-5757c987cd-n6lrf"
Oct 30 18:39:58.555003 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:39:58.554977    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1e1a2be4-6d12-483e-907d-9aa9ea15c712-webhook-cert\") pod \"maas-controller-5757c987cd-n6lrf\" (UID: \"1e1a2be4-6d12-483e-907d-9aa9ea15c712\") " pod="opendatahub/maas-controller-5757c987cd-n6lrf"
Oct 30 18:39:58.555127 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:39:58.555106    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1e1a2be4-6d12-483e-907d-9aa9ea15c712-metrics-certs\") pod \"maas-controller-5757c987cd-n6lrf\" (UID: \"1e1a2be4-6d12-483e-907d-9aa9ea15c712\") " pod="opendatahub/maas-controller-5757c987cd-n6lrf"
Oct 30 18:39:58.615119 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:39:58.615079    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-876sl\" (UniqueName: \"kubernetes.io/projected/1e1a2be4-6d12-483e-907d-9aa9ea15c712-kube-api-access-876sl\") pod \"maas-controller-5757c987cd-n6lrf\" (UID: \"1e1a2be4-6d12-483e-907d-9aa9ea15c712\") " pod="opendatahub/maas-controller-5757c987cd-n6lrf"
Oct 30 18:39:58.694791 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:39:58.694721    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5757c987cd-n6lrf"
Oct 30 18:39:58.997344 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:39:58.997261    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-86b6bf7f47-w2sgd"]
Oct 30 18:39:59.002215 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:39:59.002194    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-86b6bf7f47-w2sgd"
Oct 30 18:39:59.056475 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:39:59.056448    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dtw6b\" (UniqueName: \"kubernetes.io/projected/149ec7c2-df24-4c6e-81ff-121bad25f86d-kube-api-access-dtw6b\") pod \"maas-controller-86b6bf7f47-w2sgd\" (UID: \"149ec7c2-df24-4c6e-81ff-121bad25f86d\") " pod="opendatahub/maas-controller-86b6bf7f47-w2sgd"
Oct 30 18:39:59.056606 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:39:59.056484    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/149ec7c2-df24-4c6e-81ff-121bad25f86d-metrics-certs\") pod \"maas-controller-86b6bf7f47-w2sgd\" (UID: \"149ec7c2-df24-4c6e-81ff-121bad25f86d\") " pod="opendatahub/maas-controller-86b6bf7f47-w2sgd"
Oct 30 18:39:59.056606 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:39:59.056588    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/149ec7c2-df24-4c6e-81ff-121bad25f86d-webhook-cert\") pod \"maas-controller-86b6bf7f47-w2sgd\" (UID: \"149ec7c2-df24-4c6e-81ff-121bad25f86d\") " pod="opendatahub/maas-controller-86b6bf7f47-w2sgd"
Oct 30 18:39:59.060325 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:39:59.060299    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-86b6bf7f47-w2sgd"]
Oct 30 18:39:59.122893 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:39:59.122866    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5757c987cd-n6lrf"]
Oct 30 18:39:59.124785 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:39:59.124750    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e1a2be4_6d12_483e_907d_9aa9ea15c712.slice/crio-a71cab0d36374060aea8caeeba3a2d4d9909a414fff10d3f16a293056470fa19 WatchSource:0}: Error finding container a71cab0d36374060aea8caeeba3a2d4d9909a414fff10d3f16a293056470fa19: Status 404 returned error can't find the container with id a71cab0d36374060aea8caeeba3a2d4d9909a414fff10d3f16a293056470fa19
Oct 30 18:39:59.157240 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:39:59.157219    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/149ec7c2-df24-4c6e-81ff-121bad25f86d-webhook-cert\") pod \"maas-controller-86b6bf7f47-w2sgd\" (UID: \"149ec7c2-df24-4c6e-81ff-121bad25f86d\") " pod="opendatahub/maas-controller-86b6bf7f47-w2sgd"
Oct 30 18:39:59.157349 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:39:59.157278    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dtw6b\" (UniqueName: \"kubernetes.io/projected/149ec7c2-df24-4c6e-81ff-121bad25f86d-kube-api-access-dtw6b\") pod \"maas-controller-86b6bf7f47-w2sgd\" (UID: \"149ec7c2-df24-4c6e-81ff-121bad25f86d\") " pod="opendatahub/maas-controller-86b6bf7f47-w2sgd"
Oct 30 18:39:59.157405 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:39:59.157346    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/149ec7c2-df24-4c6e-81ff-121bad25f86d-metrics-certs\") pod \"maas-controller-86b6bf7f47-w2sgd\" (UID: \"149ec7c2-df24-4c6e-81ff-121bad25f86d\") " pod="opendatahub/maas-controller-86b6bf7f47-w2sgd"
Oct 30 18:39:59.159867 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:39:59.159844    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/149ec7c2-df24-4c6e-81ff-121bad25f86d-webhook-cert\") pod \"maas-controller-86b6bf7f47-w2sgd\" (UID: \"149ec7c2-df24-4c6e-81ff-121bad25f86d\") " pod="opendatahub/maas-controller-86b6bf7f47-w2sgd"
Oct 30 18:39:59.159867 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:39:59.159863    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/149ec7c2-df24-4c6e-81ff-121bad25f86d-metrics-certs\") pod \"maas-controller-86b6bf7f47-w2sgd\" (UID: \"149ec7c2-df24-4c6e-81ff-121bad25f86d\") " pod="opendatahub/maas-controller-86b6bf7f47-w2sgd"
Oct 30 18:39:59.219510 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:39:59.219492    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dtw6b\" (UniqueName: \"kubernetes.io/projected/149ec7c2-df24-4c6e-81ff-121bad25f86d-kube-api-access-dtw6b\") pod \"maas-controller-86b6bf7f47-w2sgd\" (UID: \"149ec7c2-df24-4c6e-81ff-121bad25f86d\") " pod="opendatahub/maas-controller-86b6bf7f47-w2sgd"
Oct 30 18:39:59.315723 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:39:59.315673    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-86b6bf7f47-w2sgd"
Oct 30 18:39:59.687306 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:39:59.687267    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-86b6bf7f47-w2sgd"]
Oct 30 18:39:59.688897 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:39:59.688874    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod149ec7c2_df24_4c6e_81ff_121bad25f86d.slice/crio-4bb2d11173c5d4c7ae12ad47d758f0609b439e5b5a92bff7de65eaa3ec025d50 WatchSource:0}: Error finding container 4bb2d11173c5d4c7ae12ad47d758f0609b439e5b5a92bff7de65eaa3ec025d50: Status 404 returned error can't find the container with id 4bb2d11173c5d4c7ae12ad47d758f0609b439e5b5a92bff7de65eaa3ec025d50
Oct 30 18:39:59.702966 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:39:59.702942    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-86b6bf7f47-w2sgd" event={"ID":"149ec7c2-df24-4c6e-81ff-121bad25f86d","Type":"ContainerStarted","Data":"4bb2d11173c5d4c7ae12ad47d758f0609b439e5b5a92bff7de65eaa3ec025d50"}
Oct 30 18:39:59.704076 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:39:59.704056    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5757c987cd-n6lrf" event={"ID":"1e1a2be4-6d12-483e-907d-9aa9ea15c712","Type":"ContainerStarted","Data":"a71cab0d36374060aea8caeeba3a2d4d9909a414fff10d3f16a293056470fa19"}
Oct 30 18:40:05.616911 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:40:05.616864    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-jbsw6" containerName="extractor"
Oct 30 18:40:05.737975 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:40:05.737888    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5757c987cd-n6lrf" event={"ID":"1e1a2be4-6d12-483e-907d-9aa9ea15c712","Type":"ContainerStarted","Data":"fc19d53b0d5b808c5eb2320830f63e25149981aebd4d0e0ae84578f21573b480"}
Oct 30 18:40:05.738129 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:40:05.737987    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5757c987cd-n6lrf"
Oct 30 18:40:05.739512 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:40:05.739490    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-86b6bf7f47-w2sgd" event={"ID":"149ec7c2-df24-4c6e-81ff-121bad25f86d","Type":"ContainerStarted","Data":"4ad76ac53237330f04d2a3e83f5c49b7a07fe18dd0a7a6302ab435d924555ddd"}
Oct 30 18:40:05.739623 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:40:05.739600    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-86b6bf7f47-w2sgd"
Oct 30 18:40:05.917255 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:40:05.912499    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/maas-controller-5757c987cd-n6lrf" podStartSLOduration=1.583541476 podStartE2EDuration="7.912482003s" totalImagesPullingTime="6.328940527s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:39:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:40:05.906960962 +0000 UTC m=+370.154366986" watchObservedRunningTime="2026-10-30 18:40:05.912482003 +0000 UTC m=+370.159888029"
Oct 30 18:40:06.120613 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:40:06.120560    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/maas-controller-86b6bf7f47-w2sgd" podStartSLOduration=2.346032947 podStartE2EDuration="8.120545153s" totalImagesPullingTime="5.774512206s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:39:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:40:06.120472229 +0000 UTC m=+370.367878255" watchObservedRunningTime="2026-10-30 18:40:06.120545153 +0000 UTC m=+370.367951179"
Oct 30 18:40:16.746762 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:40:16.746730    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5757c987cd-n6lrf"
Oct 30 18:40:16.747229 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:40:16.746805    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-86b6bf7f47-w2sgd"
Oct 30 18:40:17.031127 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:40:17.031040    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5757c987cd-n6lrf"]
Oct 30 18:40:17.031362 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:40:17.031311    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/maas-controller-5757c987cd-n6lrf" podUID="1e1a2be4-6d12-483e-907d-9aa9ea15c712" containerName="manager" containerID="cri-o://fc19d53b0d5b808c5eb2320830f63e25149981aebd4d0e0ae84578f21573b480" gracePeriod=30
Oct 30 18:40:17.389233 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:40:17.389208    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5757c987cd-n6lrf"
Oct 30 18:40:17.522724 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:40:17.522699    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1e1a2be4-6d12-483e-907d-9aa9ea15c712-webhook-cert\" (UniqueName: \"kubernetes.io/secret/1e1a2be4-6d12-483e-907d-9aa9ea15c712-webhook-cert\") pod \"1e1a2be4-6d12-483e-907d-9aa9ea15c712\" (UID: \"1e1a2be4-6d12-483e-907d-9aa9ea15c712\") "
Oct 30 18:40:17.522724 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:40:17.522731    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1e1a2be4-6d12-483e-907d-9aa9ea15c712-kube-api-access-876sl\" (UniqueName: \"kubernetes.io/projected/1e1a2be4-6d12-483e-907d-9aa9ea15c712-kube-api-access-876sl\") pod \"1e1a2be4-6d12-483e-907d-9aa9ea15c712\" (UID: \"1e1a2be4-6d12-483e-907d-9aa9ea15c712\") "
Oct 30 18:40:17.522899 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:40:17.522809    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1e1a2be4-6d12-483e-907d-9aa9ea15c712-metrics-certs\" (UniqueName: \"kubernetes.io/secret/1e1a2be4-6d12-483e-907d-9aa9ea15c712-metrics-certs\") pod \"1e1a2be4-6d12-483e-907d-9aa9ea15c712\" (UID: \"1e1a2be4-6d12-483e-907d-9aa9ea15c712\") "
Oct 30 18:40:17.525094 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:40:17.525057    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1e1a2be4-6d12-483e-907d-9aa9ea15c712-webhook-cert" pod "1e1a2be4-6d12-483e-907d-9aa9ea15c712" (UID: "1e1a2be4-6d12-483e-907d-9aa9ea15c712"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:40:17.525196 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:40:17.525116    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1e1a2be4-6d12-483e-907d-9aa9ea15c712-kube-api-access-876sl" pod "1e1a2be4-6d12-483e-907d-9aa9ea15c712" (UID: "1e1a2be4-6d12-483e-907d-9aa9ea15c712"). InnerVolumeSpecName "kube-api-access-876sl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:40:17.525196 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:40:17.525121    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1e1a2be4-6d12-483e-907d-9aa9ea15c712-metrics-certs" pod "1e1a2be4-6d12-483e-907d-9aa9ea15c712" (UID: "1e1a2be4-6d12-483e-907d-9aa9ea15c712"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:40:17.623939 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:40:17.623911    4888 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1e1a2be4-6d12-483e-907d-9aa9ea15c712-metrics-certs\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:40:17.624055 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:40:17.623942    4888 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1e1a2be4-6d12-483e-907d-9aa9ea15c712-webhook-cert\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:40:17.624055 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:40:17.623959    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-876sl\" (UniqueName: \"kubernetes.io/projected/1e1a2be4-6d12-483e-907d-9aa9ea15c712-kube-api-access-876sl\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:40:17.801673 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:40:17.801641    4888 generic.go:356] "Generic (PLEG): container finished" podID="1e1a2be4-6d12-483e-907d-9aa9ea15c712" containerID="fc19d53b0d5b808c5eb2320830f63e25149981aebd4d0e0ae84578f21573b480" exitCode=0
Oct 30 18:40:17.802087 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:40:17.801688    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5757c987cd-n6lrf"
Oct 30 18:40:17.802087 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:40:17.801728    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5757c987cd-n6lrf" event={"ID":"1e1a2be4-6d12-483e-907d-9aa9ea15c712","Type":"ContainerDied","Data":"fc19d53b0d5b808c5eb2320830f63e25149981aebd4d0e0ae84578f21573b480"}
Oct 30 18:40:17.802087 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:40:17.801805    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5757c987cd-n6lrf" event={"ID":"1e1a2be4-6d12-483e-907d-9aa9ea15c712","Type":"ContainerDied","Data":"a71cab0d36374060aea8caeeba3a2d4d9909a414fff10d3f16a293056470fa19"}
Oct 30 18:40:17.802087 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:40:17.801827    4888 scope.go:122] "RemoveContainer" containerID="fc19d53b0d5b808c5eb2320830f63e25149981aebd4d0e0ae84578f21573b480"
Oct 30 18:40:17.811185 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:40:17.811160    4888 scope.go:122] "RemoveContainer" containerID="fc19d53b0d5b808c5eb2320830f63e25149981aebd4d0e0ae84578f21573b480"
Oct 30 18:40:17.811557 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:40:17.811532    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fc19d53b0d5b808c5eb2320830f63e25149981aebd4d0e0ae84578f21573b480\": container with ID starting with fc19d53b0d5b808c5eb2320830f63e25149981aebd4d0e0ae84578f21573b480 not found: ID does not exist" containerID="fc19d53b0d5b808c5eb2320830f63e25149981aebd4d0e0ae84578f21573b480"
Oct 30 18:40:17.811614 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:40:17.811566    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fc19d53b0d5b808c5eb2320830f63e25149981aebd4d0e0ae84578f21573b480"} err="failed to get container status \"fc19d53b0d5b808c5eb2320830f63e25149981aebd4d0e0ae84578f21573b480\": rpc error: code = NotFound desc = could not find container \"fc19d53b0d5b808c5eb2320830f63e25149981aebd4d0e0ae84578f21573b480\": container with ID starting with fc19d53b0d5b808c5eb2320830f63e25149981aebd4d0e0ae84578f21573b480 not found: ID does not exist"
Oct 30 18:40:17.970278 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:40:17.970205    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5757c987cd-n6lrf"]
Oct 30 18:40:18.020868 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:40:18.020840    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-5757c987cd-n6lrf"]
Oct 30 18:40:19.889767 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:40:19.889736    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1e1a2be4-6d12-483e-907d-9aa9ea15c712" path="/var/lib/kubelet/pods/1e1a2be4-6d12-483e-907d-9aa9ea15c712/volumes"
Oct 30 18:40:20.388064 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:40:20.388032    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-776cc66d47-rrwt2"]
Oct 30 18:40:20.388475 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:40:20.388457    4888 cpu_manager.go:414] "containerMap: removing container" podUID="1e1a2be4-6d12-483e-907d-9aa9ea15c712" containerName="manager"
Oct 30 18:40:20.388475 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:40:20.388473    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1e1a2be4-6d12-483e-907d-9aa9ea15c712" containerName="manager"
Oct 30 18:40:20.388656 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:40:20.388541    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="1e1a2be4-6d12-483e-907d-9aa9ea15c712" containerName="manager"
Oct 30 18:40:20.392719 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:40:20.392696    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-776cc66d47-rrwt2"
Oct 30 18:40:20.449367 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:40:20.449328    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/79919fce-ca7a-47ce-bc62-490450a6ac2d-cert\") pod \"kserve-controller-manager-776cc66d47-rrwt2\" (UID: \"79919fce-ca7a-47ce-bc62-490450a6ac2d\") " pod="opendatahub/kserve-controller-manager-776cc66d47-rrwt2"
Oct 30 18:40:20.449506 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:40:20.449391    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8hlzg\" (UniqueName: \"kubernetes.io/projected/79919fce-ca7a-47ce-bc62-490450a6ac2d-kube-api-access-8hlzg\") pod \"kserve-controller-manager-776cc66d47-rrwt2\" (UID: \"79919fce-ca7a-47ce-bc62-490450a6ac2d\") " pod="opendatahub/kserve-controller-manager-776cc66d47-rrwt2"
Oct 30 18:40:20.455324 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:40:20.455302    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-776cc66d47-rrwt2"]
Oct 30 18:40:20.503150 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:40:20.503123    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Oct 30 18:40:20.503150 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:40:20.503129    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-mv27x\""
Oct 30 18:40:20.550621 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:40:20.550589    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/79919fce-ca7a-47ce-bc62-490450a6ac2d-cert\") pod \"kserve-controller-manager-776cc66d47-rrwt2\" (UID: \"79919fce-ca7a-47ce-bc62-490450a6ac2d\") " pod="opendatahub/kserve-controller-manager-776cc66d47-rrwt2"
Oct 30 18:40:20.550777 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:40:20.550657    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8hlzg\" (UniqueName: \"kubernetes.io/projected/79919fce-ca7a-47ce-bc62-490450a6ac2d-kube-api-access-8hlzg\") pod \"kserve-controller-manager-776cc66d47-rrwt2\" (UID: \"79919fce-ca7a-47ce-bc62-490450a6ac2d\") " pod="opendatahub/kserve-controller-manager-776cc66d47-rrwt2"
Oct 30 18:40:20.553105 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:40:20.553081    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/79919fce-ca7a-47ce-bc62-490450a6ac2d-cert\") pod \"kserve-controller-manager-776cc66d47-rrwt2\" (UID: \"79919fce-ca7a-47ce-bc62-490450a6ac2d\") " pod="opendatahub/kserve-controller-manager-776cc66d47-rrwt2"
Oct 30 18:40:20.631916 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:40:20.631890    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8hlzg\" (UniqueName: \"kubernetes.io/projected/79919fce-ca7a-47ce-bc62-490450a6ac2d-kube-api-access-8hlzg\") pod \"kserve-controller-manager-776cc66d47-rrwt2\" (UID: \"79919fce-ca7a-47ce-bc62-490450a6ac2d\") " pod="opendatahub/kserve-controller-manager-776cc66d47-rrwt2"
Oct 30 18:40:20.705499 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:40:20.705427    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-776cc66d47-rrwt2"
Oct 30 18:40:21.095726 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:40:21.095688    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-776cc66d47-rrwt2"]
Oct 30 18:40:21.097147 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:40:21.097113    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79919fce_ca7a_47ce_bc62_490450a6ac2d.slice/crio-15caf09f5fbba980355bff734ec158e1ced32de82703aad058b57e1d48e7f9b3 WatchSource:0}: Error finding container 15caf09f5fbba980355bff734ec158e1ced32de82703aad058b57e1d48e7f9b3: Status 404 returned error can't find the container with id 15caf09f5fbba980355bff734ec158e1ced32de82703aad058b57e1d48e7f9b3
Oct 30 18:40:21.824262 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:40:21.824227    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-776cc66d47-rrwt2" event={"ID":"79919fce-ca7a-47ce-bc62-490450a6ac2d","Type":"ContainerStarted","Data":"15caf09f5fbba980355bff734ec158e1ced32de82703aad058b57e1d48e7f9b3"}
Oct 30 18:40:24.843599 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:40:24.843566    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-776cc66d47-rrwt2" event={"ID":"79919fce-ca7a-47ce-bc62-490450a6ac2d","Type":"ContainerStarted","Data":"90fee25e3422f162b2046eb890ba766767df1ef0d50f8f6f6871bfe0418917be"}
Oct 30 18:40:24.843992 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:40:24.843694    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-776cc66d47-rrwt2"
Oct 30 18:40:24.963535 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:40:24.963486    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-776cc66d47-rrwt2" podStartSLOduration=1.958258251 podStartE2EDuration="4.963473793s" totalImagesPullingTime="3.005215542s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:40:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:40:24.962171336 +0000 UTC m=+389.209577361" watchObservedRunningTime="2026-10-30 18:40:24.963473793 +0000 UTC m=+389.210879838"
Oct 30 18:40:35.617788 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:40:35.617692    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-jbsw6" containerName="extractor"
Oct 30 18:40:45.472755 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:40:45.472724    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-rjmdz"]
Oct 30 18:40:54.090999 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:40:54.090957    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6f6944bcdd-tlt7v"]
Oct 30 18:40:54.141912 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:40:54.141883    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6f6944bcdd-tlt7v"
Oct 30 18:40:54.154069 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:40:54.154041    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fmjts\" (UniqueName: \"kubernetes.io/projected/71b0d5e7-a74e-409b-8d74-f95233623007-kube-api-access-fmjts\") pod \"authorino-6f6944bcdd-tlt7v\" (UID: \"71b0d5e7-a74e-409b-8d74-f95233623007\") " pod="kuadrant-system/authorino-6f6944bcdd-tlt7v"
Oct 30 18:40:54.154354 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:40:54.154335    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6f6944bcdd-tlt7v"]
Oct 30 18:40:54.202687 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:40:54.202663    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-fl2x7\""
Oct 30 18:40:54.255163 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:40:54.255128    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fmjts\" (UniqueName: \"kubernetes.io/projected/71b0d5e7-a74e-409b-8d74-f95233623007-kube-api-access-fmjts\") pod \"authorino-6f6944bcdd-tlt7v\" (UID: \"71b0d5e7-a74e-409b-8d74-f95233623007\") " pod="kuadrant-system/authorino-6f6944bcdd-tlt7v"
Oct 30 18:40:54.320364 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:40:54.320339    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fmjts\" (UniqueName: \"kubernetes.io/projected/71b0d5e7-a74e-409b-8d74-f95233623007-kube-api-access-fmjts\") pod \"authorino-6f6944bcdd-tlt7v\" (UID: \"71b0d5e7-a74e-409b-8d74-f95233623007\") " pod="kuadrant-system/authorino-6f6944bcdd-tlt7v"
Oct 30 18:40:54.510093 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:40:54.510026    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6f6944bcdd-tlt7v"]
Oct 30 18:40:54.510247 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:40:54.510234    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6f6944bcdd-tlt7v"
Oct 30 18:40:54.541074 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:40:54.541047    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-687b56f6ff-59vng"]
Oct 30 18:40:54.549600 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:40:54.549161    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-687b56f6ff-59vng"
Oct 30 18:40:54.558559 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:40:54.558529    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m7hbx\" (UniqueName: \"kubernetes.io/projected/8e7b7340-a8a8-4997-a56e-b0823b778e45-kube-api-access-m7hbx\") pod \"authorino-687b56f6ff-59vng\" (UID: \"8e7b7340-a8a8-4997-a56e-b0823b778e45\") " pod="kuadrant-system/authorino-687b56f6ff-59vng"
Oct 30 18:40:54.695706 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:40:54.608201    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-687b56f6ff-59vng"]
Oct 30 18:40:54.695706 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:40:54.659662    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m7hbx\" (UniqueName: \"kubernetes.io/projected/8e7b7340-a8a8-4997-a56e-b0823b778e45-kube-api-access-m7hbx\") pod \"authorino-687b56f6ff-59vng\" (UID: \"8e7b7340-a8a8-4997-a56e-b0823b778e45\") " pod="kuadrant-system/authorino-687b56f6ff-59vng"
Oct 30 18:40:54.721900 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:40:54.721878    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m7hbx\" (UniqueName: \"kubernetes.io/projected/8e7b7340-a8a8-4997-a56e-b0823b778e45-kube-api-access-m7hbx\") pod \"authorino-687b56f6ff-59vng\" (UID: \"8e7b7340-a8a8-4997-a56e-b0823b778e45\") " pod="kuadrant-system/authorino-687b56f6ff-59vng"
Oct 30 18:40:54.903101 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:40:54.903067    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-687b56f6ff-59vng"]
Oct 30 18:40:54.903340 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:40:54.903327    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-687b56f6ff-59vng"
Oct 30 18:40:54.924091 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:40:54.924052    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6f6944bcdd-tlt7v"]
Oct 30 18:40:54.924936 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:40:54.924910    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod71b0d5e7_a74e_409b_8d74_f95233623007.slice/crio-0113a62cc70c8d6f80b72349dfb6c9ddacabc8f99b18696cb6b6b909d6bf1674 WatchSource:0}: Error finding container 0113a62cc70c8d6f80b72349dfb6c9ddacabc8f99b18696cb6b6b909d6bf1674: Status 404 returned error can't find the container with id 0113a62cc70c8d6f80b72349dfb6c9ddacabc8f99b18696cb6b6b909d6bf1674
Oct 30 18:40:54.934442 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:40:54.934411    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-57c95767fc-6lkr8"]
Oct 30 18:40:54.937215 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:40:54.937197    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-57c95767fc-6lkr8"
Oct 30 18:40:54.963956 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:40:54.963927    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/91b5bf50-ee12-4846-92c2-7f30dfb0b4f2-tls-cert\") pod \"authorino-57c95767fc-6lkr8\" (UID: \"91b5bf50-ee12-4846-92c2-7f30dfb0b4f2\") " pod="kuadrant-system/authorino-57c95767fc-6lkr8"
Oct 30 18:40:54.964165 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:40:54.964131    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5pv24\" (UniqueName: \"kubernetes.io/projected/91b5bf50-ee12-4846-92c2-7f30dfb0b4f2-kube-api-access-5pv24\") pod \"authorino-57c95767fc-6lkr8\" (UID: \"91b5bf50-ee12-4846-92c2-7f30dfb0b4f2\") " pod="kuadrant-system/authorino-57c95767fc-6lkr8"
Oct 30 18:40:54.993369 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:40:54.993340    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Oct 30 18:40:54.996059 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:40:54.996032    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-57c95767fc-6lkr8"]
Oct 30 18:40:55.024482 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:40:55.024453    4888 generic.go:356] "Generic (PLEG): container finished" podID="bf0f21cf-8198-4fe4-8983-db083c82b34a" containerID="bf28cd527b037444b211a9ce7cbb22d6e612330c035e394c25bba819000865be" exitCode=0
Oct 30 18:40:55.024656 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:40:55.024535    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-mhzc5" event={"ID":"bf0f21cf-8198-4fe4-8983-db083c82b34a","Type":"ContainerDied","Data":"bf28cd527b037444b211a9ce7cbb22d6e612330c035e394c25bba819000865be"}
Oct 30 18:40:55.024987 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:40:55.024969    4888 scope.go:122] "RemoveContainer" containerID="bf28cd527b037444b211a9ce7cbb22d6e612330c035e394c25bba819000865be"
Oct 30 18:40:55.025877 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:40:55.025856    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6f6944bcdd-tlt7v" event={"ID":"71b0d5e7-a74e-409b-8d74-f95233623007","Type":"ContainerStarted","Data":"0113a62cc70c8d6f80b72349dfb6c9ddacabc8f99b18696cb6b6b909d6bf1674"}
Oct 30 18:40:55.026118 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:40:55.026094    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:40:55.026179 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:40:55.026167    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:40:55.026222 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:40:55.026203    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:40:55.064872 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:40:55.064833    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5pv24\" (UniqueName: \"kubernetes.io/projected/91b5bf50-ee12-4846-92c2-7f30dfb0b4f2-kube-api-access-5pv24\") pod \"authorino-57c95767fc-6lkr8\" (UID: \"91b5bf50-ee12-4846-92c2-7f30dfb0b4f2\") " pod="kuadrant-system/authorino-57c95767fc-6lkr8"
Oct 30 18:40:55.065074 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:40:55.064923    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/91b5bf50-ee12-4846-92c2-7f30dfb0b4f2-tls-cert\") pod \"authorino-57c95767fc-6lkr8\" (UID: \"91b5bf50-ee12-4846-92c2-7f30dfb0b4f2\") " pod="kuadrant-system/authorino-57c95767fc-6lkr8"
Oct 30 18:40:55.067659 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:40:55.067629    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/91b5bf50-ee12-4846-92c2-7f30dfb0b4f2-tls-cert\") pod \"authorino-57c95767fc-6lkr8\" (UID: \"91b5bf50-ee12-4846-92c2-7f30dfb0b4f2\") " pod="kuadrant-system/authorino-57c95767fc-6lkr8"
Oct 30 18:40:55.129984 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:40:55.129947    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5pv24\" (UniqueName: \"kubernetes.io/projected/91b5bf50-ee12-4846-92c2-7f30dfb0b4f2-kube-api-access-5pv24\") pod \"authorino-57c95767fc-6lkr8\" (UID: \"91b5bf50-ee12-4846-92c2-7f30dfb0b4f2\") " pod="kuadrant-system/authorino-57c95767fc-6lkr8"
Oct 30 18:40:55.249567 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:40:55.249489    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-57c95767fc-6lkr8"
Oct 30 18:40:55.288122 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:40:55.287570    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-687b56f6ff-59vng"]
Oct 30 18:40:55.288391 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:40:55.288332    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8e7b7340_a8a8_4997_a56e_b0823b778e45.slice/crio-c20ee24de7c493d78eae833e697cce387189de360141b6cc056183c4d5cf0812 WatchSource:0}: Error finding container c20ee24de7c493d78eae833e697cce387189de360141b6cc056183c4d5cf0812: Status 404 returned error can't find the container with id c20ee24de7c493d78eae833e697cce387189de360141b6cc056183c4d5cf0812
Oct 30 18:40:55.639438 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:40:55.639412    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-57c95767fc-6lkr8"]
Oct 30 18:40:55.640754 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:40:55.640727    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod91b5bf50_ee12_4846_92c2_7f30dfb0b4f2.slice/crio-4a34022844975c2a69af57fc1fd411317a41ce0baea5b2a533a694579c59b4ab WatchSource:0}: Error finding container 4a34022844975c2a69af57fc1fd411317a41ce0baea5b2a533a694579c59b4ab: Status 404 returned error can't find the container with id 4a34022844975c2a69af57fc1fd411317a41ce0baea5b2a533a694579c59b4ab
Oct 30 18:40:55.854175 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:40:55.854145    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-776cc66d47-rrwt2"
Oct 30 18:40:56.033245 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:40:56.033213    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-57c95767fc-6lkr8" event={"ID":"91b5bf50-ee12-4846-92c2-7f30dfb0b4f2","Type":"ContainerStarted","Data":"4a34022844975c2a69af57fc1fd411317a41ce0baea5b2a533a694579c59b4ab"}
Oct 30 18:40:56.034541 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:40:56.034517    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-687b56f6ff-59vng" event={"ID":"8e7b7340-a8a8-4997-a56e-b0823b778e45","Type":"ContainerStarted","Data":"c20ee24de7c493d78eae833e697cce387189de360141b6cc056183c4d5cf0812"}
Oct 30 18:40:56.036388 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:40:56.036362    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-mhzc5" event={"ID":"bf0f21cf-8198-4fe4-8983-db083c82b34a","Type":"ContainerStarted","Data":"4a9b2f452f9145f7aa7b2379b1ae2e9d191d8db4e55563588af3e448788acfbb"}
Oct 30 18:40:56.461819 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:40:56.461784    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-mhzc5"
Oct 30 18:40:56.462213 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:40:56.461959    4888 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-66db998887-mhzc5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.32:15021/healthz/ready\": dial tcp 10.132.0.32:15021: connect: connection refused" start-of-body=
Oct 30 18:40:56.462213 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:40:56.462008    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-mhzc5" podUID="bf0f21cf-8198-4fe4-8983-db083c82b34a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.32:15021/healthz/ready\": dial tcp 10.132.0.32:15021: connect: connection refused"
Oct 30 18:40:57.043440 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:40:57.043407    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-687b56f6ff-59vng" event={"ID":"8e7b7340-a8a8-4997-a56e-b0823b778e45","Type":"ContainerStarted","Data":"ed27e2566bed8b1ad34470cbaa8bbc85ca1a6ca70b590b8ccae1e3191edbbc56"}
Oct 30 18:40:57.043623 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:40:57.043415    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/authorino-687b56f6ff-59vng" podUID="8e7b7340-a8a8-4997-a56e-b0823b778e45" containerName="authorino" containerID="cri-o://ed27e2566bed8b1ad34470cbaa8bbc85ca1a6ca70b590b8ccae1e3191edbbc56" gracePeriod=30
Oct 30 18:40:57.172368 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:40:57.167881    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/authorino-687b56f6ff-59vng" podStartSLOduration=1.5373390059999998 podStartE2EDuration="3.167862357s" totalImagesPullingTime="1.630523351s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:40:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:40:57.166028173 +0000 UTC m=+421.413434198" watchObservedRunningTime="2026-10-30 18:40:57.167862357 +0000 UTC m=+421.415268383"
Oct 30 18:40:57.423241 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:40:57.423189    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-687b56f6ff-59vng"
Oct 30 18:40:57.462182 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:40:57.462155    4888 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-66db998887-mhzc5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.32:15021/healthz/ready\": dial tcp 10.132.0.32:15021: connect: connection refused" start-of-body=
Oct 30 18:40:57.462529 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:40:57.462201    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-mhzc5" podUID="bf0f21cf-8198-4fe4-8983-db083c82b34a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.32:15021/healthz/ready\": dial tcp 10.132.0.32:15021: connect: connection refused"
Oct 30 18:40:57.513843 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:40:57.513810    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8e7b7340-a8a8-4997-a56e-b0823b778e45-kube-api-access-m7hbx\" (UniqueName: \"kubernetes.io/projected/8e7b7340-a8a8-4997-a56e-b0823b778e45-kube-api-access-m7hbx\") pod \"8e7b7340-a8a8-4997-a56e-b0823b778e45\" (UID: \"8e7b7340-a8a8-4997-a56e-b0823b778e45\") "
Oct 30 18:40:57.516007 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:40:57.515977    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8e7b7340-a8a8-4997-a56e-b0823b778e45-kube-api-access-m7hbx" pod "8e7b7340-a8a8-4997-a56e-b0823b778e45" (UID: "8e7b7340-a8a8-4997-a56e-b0823b778e45"). InnerVolumeSpecName "kube-api-access-m7hbx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:40:57.615494 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:40:57.615461    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m7hbx\" (UniqueName: \"kubernetes.io/projected/8e7b7340-a8a8-4997-a56e-b0823b778e45-kube-api-access-m7hbx\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:40:58.051202 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:40:58.051165    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-57c95767fc-6lkr8" event={"ID":"91b5bf50-ee12-4846-92c2-7f30dfb0b4f2","Type":"ContainerStarted","Data":"c211a7b5c7e61ccf74f7c34e395c023db1b63e6a7827fc18c854582204d36b2c"}
Oct 30 18:40:58.052593 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:40:58.052568    4888 generic.go:356] "Generic (PLEG): container finished" podID="8e7b7340-a8a8-4997-a56e-b0823b778e45" containerID="ed27e2566bed8b1ad34470cbaa8bbc85ca1a6ca70b590b8ccae1e3191edbbc56" exitCode=0
Oct 30 18:40:58.052722 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:40:58.052630    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-687b56f6ff-59vng" event={"ID":"8e7b7340-a8a8-4997-a56e-b0823b778e45","Type":"ContainerDied","Data":"ed27e2566bed8b1ad34470cbaa8bbc85ca1a6ca70b590b8ccae1e3191edbbc56"}
Oct 30 18:40:58.052722 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:40:58.052642    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-687b56f6ff-59vng"
Oct 30 18:40:58.052722 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:40:58.052652    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-687b56f6ff-59vng" event={"ID":"8e7b7340-a8a8-4997-a56e-b0823b778e45","Type":"ContainerDied","Data":"c20ee24de7c493d78eae833e697cce387189de360141b6cc056183c4d5cf0812"}
Oct 30 18:40:58.052722 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:40:58.052669    4888 scope.go:122] "RemoveContainer" containerID="ed27e2566bed8b1ad34470cbaa8bbc85ca1a6ca70b590b8ccae1e3191edbbc56"
Oct 30 18:40:58.061740 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:40:58.061720    4888 scope.go:122] "RemoveContainer" containerID="ed27e2566bed8b1ad34470cbaa8bbc85ca1a6ca70b590b8ccae1e3191edbbc56"
Oct 30 18:40:58.062048 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:40:58.062028    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ed27e2566bed8b1ad34470cbaa8bbc85ca1a6ca70b590b8ccae1e3191edbbc56\": container with ID starting with ed27e2566bed8b1ad34470cbaa8bbc85ca1a6ca70b590b8ccae1e3191edbbc56 not found: ID does not exist" containerID="ed27e2566bed8b1ad34470cbaa8bbc85ca1a6ca70b590b8ccae1e3191edbbc56"
Oct 30 18:40:58.062117 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:40:58.062081    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ed27e2566bed8b1ad34470cbaa8bbc85ca1a6ca70b590b8ccae1e3191edbbc56"} err="failed to get container status \"ed27e2566bed8b1ad34470cbaa8bbc85ca1a6ca70b590b8ccae1e3191edbbc56\": rpc error: code = NotFound desc = could not find container \"ed27e2566bed8b1ad34470cbaa8bbc85ca1a6ca70b590b8ccae1e3191edbbc56\": container with ID starting with ed27e2566bed8b1ad34470cbaa8bbc85ca1a6ca70b590b8ccae1e3191edbbc56 not found: ID does not exist"
Oct 30 18:40:58.173362 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:40:58.170514    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/authorino-57c95767fc-6lkr8" podStartSLOduration=2.500667299 podStartE2EDuration="4.170484451s" totalImagesPullingTime="1.669817152s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:40:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:40:58.167661219 +0000 UTC m=+422.415067245" watchObservedRunningTime="2026-10-30 18:40:58.170484451 +0000 UTC m=+422.417890477"
Oct 30 18:40:58.295768 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:40:58.295723    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-687b56f6ff-59vng"]
Oct 30 18:40:58.297764 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:40:58.297734    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-687b56f6ff-59vng"]
Oct 30 18:40:58.462231 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:40:58.462139    4888 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-66db998887-mhzc5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.32:15021/healthz/ready\": dial tcp 10.132.0.32:15021: connect: connection refused" start-of-body=
Oct 30 18:40:58.462231 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:40:58.462209    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-mhzc5" podUID="bf0f21cf-8198-4fe4-8983-db083c82b34a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.32:15021/healthz/ready\": dial tcp 10.132.0.32:15021: connect: connection refused"
Oct 30 18:40:59.059884 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:40:59.059848    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6f6944bcdd-tlt7v" event={"ID":"71b0d5e7-a74e-409b-8d74-f95233623007","Type":"ContainerStarted","Data":"3bf0850ec392f310298c7001dc19a3dbbbd08aa65a762b2e54c3250843abb6b4"}
Oct 30 18:40:59.060069 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:40:59.059911    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/authorino-6f6944bcdd-tlt7v" podUID="71b0d5e7-a74e-409b-8d74-f95233623007" containerName="authorino" containerID="cri-o://3bf0850ec392f310298c7001dc19a3dbbbd08aa65a762b2e54c3250843abb6b4" gracePeriod=30
Oct 30 18:40:59.182538 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:40:59.176968    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/authorino-6f6944bcdd-tlt7v" podStartSLOduration=1.247401306 podStartE2EDuration="5.176947213s" totalImagesPullingTime="3.929545907s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:40:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:40:59.176697426 +0000 UTC m=+423.424103454" watchObservedRunningTime="2026-10-30 18:40:59.176947213 +0000 UTC m=+423.424353239"
Oct 30 18:40:59.347110 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:40:59.347087    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6f6944bcdd-tlt7v"
Oct 30 18:40:59.431820 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:40:59.431739    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/71b0d5e7-a74e-409b-8d74-f95233623007-kube-api-access-fmjts\" (UniqueName: \"kubernetes.io/projected/71b0d5e7-a74e-409b-8d74-f95233623007-kube-api-access-fmjts\") pod \"71b0d5e7-a74e-409b-8d74-f95233623007\" (UID: \"71b0d5e7-a74e-409b-8d74-f95233623007\") "
Oct 30 18:40:59.433936 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:40:59.433904    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/71b0d5e7-a74e-409b-8d74-f95233623007-kube-api-access-fmjts" pod "71b0d5e7-a74e-409b-8d74-f95233623007" (UID: "71b0d5e7-a74e-409b-8d74-f95233623007"). InnerVolumeSpecName "kube-api-access-fmjts". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:40:59.461809 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:40:59.461784    4888 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-66db998887-mhzc5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.32:15021/healthz/ready\": dial tcp 10.132.0.32:15021: connect: connection refused" start-of-body=
Oct 30 18:40:59.461904 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:40:59.461834    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-mhzc5" podUID="bf0f21cf-8198-4fe4-8983-db083c82b34a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.32:15021/healthz/ready\": dial tcp 10.132.0.32:15021: connect: connection refused"
Oct 30 18:40:59.532537 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:40:59.532508    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fmjts\" (UniqueName: \"kubernetes.io/projected/71b0d5e7-a74e-409b-8d74-f95233623007-kube-api-access-fmjts\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:40:59.896307 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:40:59.895682    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8e7b7340-a8a8-4997-a56e-b0823b778e45" path="/var/lib/kubelet/pods/8e7b7340-a8a8-4997-a56e-b0823b778e45/volumes"
Oct 30 18:41:00.066707 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:41:00.066673    4888 generic.go:356] "Generic (PLEG): container finished" podID="71b0d5e7-a74e-409b-8d74-f95233623007" containerID="3bf0850ec392f310298c7001dc19a3dbbbd08aa65a762b2e54c3250843abb6b4" exitCode=0
Oct 30 18:41:00.066856 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:41:00.066724    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6f6944bcdd-tlt7v"
Oct 30 18:41:00.066856 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:41:00.066758    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6f6944bcdd-tlt7v" event={"ID":"71b0d5e7-a74e-409b-8d74-f95233623007","Type":"ContainerDied","Data":"3bf0850ec392f310298c7001dc19a3dbbbd08aa65a762b2e54c3250843abb6b4"}
Oct 30 18:41:00.066856 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:41:00.066806    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6f6944bcdd-tlt7v" event={"ID":"71b0d5e7-a74e-409b-8d74-f95233623007","Type":"ContainerDied","Data":"0113a62cc70c8d6f80b72349dfb6c9ddacabc8f99b18696cb6b6b909d6bf1674"}
Oct 30 18:41:00.066856 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:41:00.066823    4888 scope.go:122] "RemoveContainer" containerID="3bf0850ec392f310298c7001dc19a3dbbbd08aa65a762b2e54c3250843abb6b4"
Oct 30 18:41:00.101842 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:41:00.101818    4888 scope.go:122] "RemoveContainer" containerID="3bf0850ec392f310298c7001dc19a3dbbbd08aa65a762b2e54c3250843abb6b4"
Oct 30 18:41:00.102461 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:41:00.102437    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3bf0850ec392f310298c7001dc19a3dbbbd08aa65a762b2e54c3250843abb6b4\": container with ID starting with 3bf0850ec392f310298c7001dc19a3dbbbd08aa65a762b2e54c3250843abb6b4 not found: ID does not exist" containerID="3bf0850ec392f310298c7001dc19a3dbbbd08aa65a762b2e54c3250843abb6b4"
Oct 30 18:41:00.102552 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:41:00.102471    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3bf0850ec392f310298c7001dc19a3dbbbd08aa65a762b2e54c3250843abb6b4"} err="failed to get container status \"3bf0850ec392f310298c7001dc19a3dbbbd08aa65a762b2e54c3250843abb6b4\": rpc error: code = NotFound desc = could not find container \"3bf0850ec392f310298c7001dc19a3dbbbd08aa65a762b2e54c3250843abb6b4\": container with ID starting with 3bf0850ec392f310298c7001dc19a3dbbbd08aa65a762b2e54c3250843abb6b4 not found: ID does not exist"
Oct 30 18:41:00.241271 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:41:00.241189    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6f6944bcdd-tlt7v"]
Oct 30 18:41:00.293470 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:41:00.293438    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6f6944bcdd-tlt7v"]
Oct 30 18:41:00.461611 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:41:00.461581    4888 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-66db998887-mhzc5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.32:15021/healthz/ready\": dial tcp 10.132.0.32:15021: connect: connection refused" start-of-body=
Oct 30 18:41:00.461780 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:41:00.461628    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-mhzc5" podUID="bf0f21cf-8198-4fe4-8983-db083c82b34a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.32:15021/healthz/ready\": dial tcp 10.132.0.32:15021: connect: connection refused"
Oct 30 18:41:01.465547 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:41:01.465523    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-mhzc5"
Oct 30 18:41:01.466003 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:41:01.465820    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-mhzc5"
Oct 30 18:41:01.466514 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:41:01.466497    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-mhzc5"
Oct 30 18:41:01.890420 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:41:01.890392    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="71b0d5e7-a74e-409b-8d74-f95233623007" path="/var/lib/kubelet/pods/71b0d5e7-a74e-409b-8d74-f95233623007/volumes"
Oct 30 18:41:05.617087 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:41:05.617049    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-jbsw6" containerName="extractor"
Oct 30 18:41:35.616995 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:41:35.616949    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-jbsw6" containerName="extractor"
Oct 30 18:42:05.617131 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:42:05.617059    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-jbsw6" containerName="extractor"
Oct 30 18:42:35.555071 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:42:35.555041    4888 generic.go:356] "Generic (PLEG): container finished" podID="149ec7c2-df24-4c6e-81ff-121bad25f86d" containerID="4ad76ac53237330f04d2a3e83f5c49b7a07fe18dd0a7a6302ab435d924555ddd" exitCode=1
Oct 30 18:42:35.555528 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:42:35.555119    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-86b6bf7f47-w2sgd" event={"ID":"149ec7c2-df24-4c6e-81ff-121bad25f86d","Type":"ContainerDied","Data":"4ad76ac53237330f04d2a3e83f5c49b7a07fe18dd0a7a6302ab435d924555ddd"}
Oct 30 18:42:35.555528 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:42:35.555442    4888 scope.go:122] "RemoveContainer" containerID="4ad76ac53237330f04d2a3e83f5c49b7a07fe18dd0a7a6302ab435d924555ddd"
Oct 30 18:42:35.616134 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:42:35.616102    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-jbsw6" containerName="extractor"
Oct 30 18:42:36.745680 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:42:36.745650    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-86b6bf7f47-w2sgd"
Oct 30 18:42:37.568015 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:42:37.567973    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-86b6bf7f47-w2sgd" event={"ID":"149ec7c2-df24-4c6e-81ff-121bad25f86d","Type":"ContainerStarted","Data":"39a1161806bad5166d227ca037289dfac220cb4a892cd77232d957723cb8a04a"}
Oct 30 18:42:37.568196 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:42:37.568078    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-86b6bf7f47-w2sgd"
Oct 30 18:42:48.575938 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:42:48.575904    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-86b6bf7f47-w2sgd"
Oct 30 18:42:59.841483 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:42:59.841442    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-rjmdz"]
Oct 30 18:43:05.616775 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:43:05.616735    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-jbsw6" containerName="extractor"
Oct 30 18:43:06.702910 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:06.702864    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-embedding-simulated-kserve-67bf594cd6-dz9tf"]
Oct 30 18:43:06.703462 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:06.703356    4888 cpu_manager.go:414] "containerMap: removing container" podUID="71b0d5e7-a74e-409b-8d74-f95233623007" containerName="authorino"
Oct 30 18:43:06.703462 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:06.703374    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="71b0d5e7-a74e-409b-8d74-f95233623007" containerName="authorino"
Oct 30 18:43:06.703462 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:06.703399    4888 cpu_manager.go:414] "containerMap: removing container" podUID="8e7b7340-a8a8-4997-a56e-b0823b778e45" containerName="authorino"
Oct 30 18:43:06.703462 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:06.703408    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8e7b7340-a8a8-4997-a56e-b0823b778e45" containerName="authorino"
Oct 30 18:43:06.703690 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:06.703503    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="71b0d5e7-a74e-409b-8d74-f95233623007" containerName="authorino"
Oct 30 18:43:06.703690 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:06.703516    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="8e7b7340-a8a8-4997-a56e-b0823b778e45" containerName="authorino"
Oct 30 18:43:06.708116 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:06.708083    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-dz9tf"
Oct 30 18:43:06.767573 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:06.767544    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-67bf594cd6-dz9tf"]
Oct 30 18:43:06.804022 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:06.803982    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x9v8v\" (UniqueName: \"kubernetes.io/projected/c7c621a4-fc1b-40e4-bb4e-1971246d0472-kube-api-access-x9v8v\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-dz9tf\" (UID: \"c7c621a4-fc1b-40e4-bb4e-1971246d0472\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-dz9tf"
Oct 30 18:43:06.804186 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:06.804101    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c7c621a4-fc1b-40e4-bb4e-1971246d0472-model-cache\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-dz9tf\" (UID: \"c7c621a4-fc1b-40e4-bb4e-1971246d0472\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-dz9tf"
Oct 30 18:43:06.804186 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:06.804152    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c7c621a4-fc1b-40e4-bb4e-1971246d0472-tls-certs\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-dz9tf\" (UID: \"c7c621a4-fc1b-40e4-bb4e-1971246d0472\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-dz9tf"
Oct 30 18:43:06.804186 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:06.804182    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c7c621a4-fc1b-40e4-bb4e-1971246d0472-dshm\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-dz9tf\" (UID: \"c7c621a4-fc1b-40e4-bb4e-1971246d0472\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-dz9tf"
Oct 30 18:43:06.804415 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:06.804202    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c7c621a4-fc1b-40e4-bb4e-1971246d0472-home\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-dz9tf\" (UID: \"c7c621a4-fc1b-40e4-bb4e-1971246d0472\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-dz9tf"
Oct 30 18:43:06.804488 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:06.804432    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c7c621a4-fc1b-40e4-bb4e-1971246d0472-tmp-dir\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-dz9tf\" (UID: \"c7c621a4-fc1b-40e4-bb4e-1971246d0472\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-dz9tf"
Oct 30 18:43:06.814442 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:06.814421    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Oct 30 18:43:06.814442 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:06.814441    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-s7879\""
Oct 30 18:43:06.814594 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:06.814491    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Oct 30 18:43:06.814681 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:06.814663    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-embedding-simulated-kserve-self-signed-certs\""
Oct 30 18:43:06.905519 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:06.905483    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x9v8v\" (UniqueName: \"kubernetes.io/projected/c7c621a4-fc1b-40e4-bb4e-1971246d0472-kube-api-access-x9v8v\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-dz9tf\" (UID: \"c7c621a4-fc1b-40e4-bb4e-1971246d0472\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-dz9tf"
Oct 30 18:43:06.905701 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:06.905542    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c7c621a4-fc1b-40e4-bb4e-1971246d0472-model-cache\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-dz9tf\" (UID: \"c7c621a4-fc1b-40e4-bb4e-1971246d0472\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-dz9tf"
Oct 30 18:43:06.905701 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:06.905580    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c7c621a4-fc1b-40e4-bb4e-1971246d0472-tls-certs\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-dz9tf\" (UID: \"c7c621a4-fc1b-40e4-bb4e-1971246d0472\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-dz9tf"
Oct 30 18:43:06.905701 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:06.905621    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c7c621a4-fc1b-40e4-bb4e-1971246d0472-dshm\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-dz9tf\" (UID: \"c7c621a4-fc1b-40e4-bb4e-1971246d0472\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-dz9tf"
Oct 30 18:43:06.905701 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:06.905646    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c7c621a4-fc1b-40e4-bb4e-1971246d0472-home\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-dz9tf\" (UID: \"c7c621a4-fc1b-40e4-bb4e-1971246d0472\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-dz9tf"
Oct 30 18:43:06.905701 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:06.905695    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c7c621a4-fc1b-40e4-bb4e-1971246d0472-tmp-dir\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-dz9tf\" (UID: \"c7c621a4-fc1b-40e4-bb4e-1971246d0472\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-dz9tf"
Oct 30 18:43:06.906104 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:06.906067    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c7c621a4-fc1b-40e4-bb4e-1971246d0472-model-cache\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-dz9tf\" (UID: \"c7c621a4-fc1b-40e4-bb4e-1971246d0472\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-dz9tf"
Oct 30 18:43:06.906104 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:06.906073    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c7c621a4-fc1b-40e4-bb4e-1971246d0472-tmp-dir\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-dz9tf\" (UID: \"c7c621a4-fc1b-40e4-bb4e-1971246d0472\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-dz9tf"
Oct 30 18:43:06.906259 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:06.906180    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c7c621a4-fc1b-40e4-bb4e-1971246d0472-home\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-dz9tf\" (UID: \"c7c621a4-fc1b-40e4-bb4e-1971246d0472\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-dz9tf"
Oct 30 18:43:06.908019 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:06.907996    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c7c621a4-fc1b-40e4-bb4e-1971246d0472-dshm\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-dz9tf\" (UID: \"c7c621a4-fc1b-40e4-bb4e-1971246d0472\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-dz9tf"
Oct 30 18:43:06.908298 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:06.908264    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c7c621a4-fc1b-40e4-bb4e-1971246d0472-tls-certs\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-dz9tf\" (UID: \"c7c621a4-fc1b-40e4-bb4e-1971246d0472\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-dz9tf"
Oct 30 18:43:06.968721 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:06.968654    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x9v8v\" (UniqueName: \"kubernetes.io/projected/c7c621a4-fc1b-40e4-bb4e-1971246d0472-kube-api-access-x9v8v\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-dz9tf\" (UID: \"c7c621a4-fc1b-40e4-bb4e-1971246d0472\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-dz9tf"
Oct 30 18:43:07.024421 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:07.024383    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-dz9tf"
Oct 30 18:43:07.421219 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:07.421193    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-67bf594cd6-dz9tf"]
Oct 30 18:43:07.422221 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:43:07.422195    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc7c621a4_fc1b_40e4_bb4e_1971246d0472.slice/crio-e7f5d310a91f302d13202f1e16e04211d9c0fff3a786cb4e1d97235f7901c178 WatchSource:0}: Error finding container e7f5d310a91f302d13202f1e16e04211d9c0fff3a786cb4e1d97235f7901c178: Status 404 returned error can't find the container with id e7f5d310a91f302d13202f1e16e04211d9c0fff3a786cb4e1d97235f7901c178
Oct 30 18:43:07.724988 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:07.724901    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-dz9tf" event={"ID":"c7c621a4-fc1b-40e4-bb4e-1971246d0472","Type":"ContainerStarted","Data":"e7f5d310a91f302d13202f1e16e04211d9c0fff3a786cb4e1d97235f7901c178"}
Oct 30 18:43:09.737871 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:09.737839    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-dz9tf" event={"ID":"c7c621a4-fc1b-40e4-bb4e-1971246d0472","Type":"ContainerStarted","Data":"b56528e3046d1612314742be7e2f8c94274fb589c3c869eba660963bd4efcb63"}
Oct 30 18:43:09.870008 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:09.869945    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-dz9tf" podStartSLOduration=2.319601511 podStartE2EDuration="3.869920232s" totalImagesPullingTime="1.550318721s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:43:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:43:09.866836303 +0000 UTC m=+554.114242330" watchObservedRunningTime="2026-10-30 18:43:09.869920232 +0000 UTC m=+554.117326264"
Oct 30 18:43:10.025202 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:10.025118    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-dz9tf"
Oct 30 18:43:11.628715 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:11.628683    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-69b588897d-cwzlx"]
Oct 30 18:43:11.632651 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:11.632628    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-cwzlx"
Oct 30 18:43:11.696998 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:11.696954    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Oct 30 18:43:11.717581 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:11.717548    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-69b588897d-cwzlx"]
Oct 30 18:43:11.747744 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:11.747701    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e9371581-24f3-4ce8-aae8-cc1a8614f4c0-home\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-cwzlx\" (UID: \"e9371581-24f3-4ce8-aae8-cc1a8614f4c0\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-cwzlx"
Oct 30 18:43:11.761492 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:11.761455    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e9371581-24f3-4ce8-aae8-cc1a8614f4c0-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-cwzlx\" (UID: \"e9371581-24f3-4ce8-aae8-cc1a8614f4c0\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-cwzlx"
Oct 30 18:43:11.761670 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:11.761626    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e9371581-24f3-4ce8-aae8-cc1a8614f4c0-dshm\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-cwzlx\" (UID: \"e9371581-24f3-4ce8-aae8-cc1a8614f4c0\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-cwzlx"
Oct 30 18:43:11.761670 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:11.761656    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e9371581-24f3-4ce8-aae8-cc1a8614f4c0-model-cache\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-cwzlx\" (UID: \"e9371581-24f3-4ce8-aae8-cc1a8614f4c0\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-cwzlx"
Oct 30 18:43:11.761772 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:11.761678    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-62rgj\" (UniqueName: \"kubernetes.io/projected/e9371581-24f3-4ce8-aae8-cc1a8614f4c0-kube-api-access-62rgj\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-cwzlx\" (UID: \"e9371581-24f3-4ce8-aae8-cc1a8614f4c0\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-cwzlx"
Oct 30 18:43:11.761772 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:11.761747    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e9371581-24f3-4ce8-aae8-cc1a8614f4c0-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-cwzlx\" (UID: \"e9371581-24f3-4ce8-aae8-cc1a8614f4c0\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-cwzlx"
Oct 30 18:43:11.792998 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:11.792951    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-rjmdz"]
Oct 30 18:43:11.863024 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:11.862980    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e9371581-24f3-4ce8-aae8-cc1a8614f4c0-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-cwzlx\" (UID: \"e9371581-24f3-4ce8-aae8-cc1a8614f4c0\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-cwzlx"
Oct 30 18:43:11.863217 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:11.863160    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e9371581-24f3-4ce8-aae8-cc1a8614f4c0-dshm\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-cwzlx\" (UID: \"e9371581-24f3-4ce8-aae8-cc1a8614f4c0\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-cwzlx"
Oct 30 18:43:11.863217 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:11.863188    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e9371581-24f3-4ce8-aae8-cc1a8614f4c0-model-cache\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-cwzlx\" (UID: \"e9371581-24f3-4ce8-aae8-cc1a8614f4c0\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-cwzlx"
Oct 30 18:43:11.863380 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:11.863230    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-62rgj\" (UniqueName: \"kubernetes.io/projected/e9371581-24f3-4ce8-aae8-cc1a8614f4c0-kube-api-access-62rgj\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-cwzlx\" (UID: \"e9371581-24f3-4ce8-aae8-cc1a8614f4c0\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-cwzlx"
Oct 30 18:43:11.863380 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:11.863262    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e9371581-24f3-4ce8-aae8-cc1a8614f4c0-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-cwzlx\" (UID: \"e9371581-24f3-4ce8-aae8-cc1a8614f4c0\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-cwzlx"
Oct 30 18:43:11.863380 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:11.863320    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e9371581-24f3-4ce8-aae8-cc1a8614f4c0-home\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-cwzlx\" (UID: \"e9371581-24f3-4ce8-aae8-cc1a8614f4c0\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-cwzlx"
Oct 30 18:43:11.864066 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:11.864019    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e9371581-24f3-4ce8-aae8-cc1a8614f4c0-model-cache\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-cwzlx\" (UID: \"e9371581-24f3-4ce8-aae8-cc1a8614f4c0\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-cwzlx"
Oct 30 18:43:11.864174 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:11.864072    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e9371581-24f3-4ce8-aae8-cc1a8614f4c0-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-cwzlx\" (UID: \"e9371581-24f3-4ce8-aae8-cc1a8614f4c0\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-cwzlx"
Oct 30 18:43:11.864174 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:11.864113    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e9371581-24f3-4ce8-aae8-cc1a8614f4c0-home\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-cwzlx\" (UID: \"e9371581-24f3-4ce8-aae8-cc1a8614f4c0\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-cwzlx"
Oct 30 18:43:11.866395 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:11.866374    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e9371581-24f3-4ce8-aae8-cc1a8614f4c0-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-cwzlx\" (UID: \"e9371581-24f3-4ce8-aae8-cc1a8614f4c0\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-cwzlx"
Oct 30 18:43:11.866556 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:11.866539    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e9371581-24f3-4ce8-aae8-cc1a8614f4c0-dshm\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-cwzlx\" (UID: \"e9371581-24f3-4ce8-aae8-cc1a8614f4c0\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-cwzlx"
Oct 30 18:43:11.929577 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:11.929502    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-62rgj\" (UniqueName: \"kubernetes.io/projected/e9371581-24f3-4ce8-aae8-cc1a8614f4c0-kube-api-access-62rgj\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-cwzlx\" (UID: \"e9371581-24f3-4ce8-aae8-cc1a8614f4c0\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-cwzlx"
Oct 30 18:43:11.948076 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:11.948045    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-cwzlx"
Oct 30 18:43:12.337061 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:12.337038    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-69b588897d-cwzlx"]
Oct 30 18:43:12.339239 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:43:12.339212    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9371581_24f3_4ce8_aae8_cc1a8614f4c0.slice/crio-1b79ac11737b9ede5ffb8aca2a776f54ebce3e09d07fa9432f58f356eef046a6 WatchSource:0}: Error finding container 1b79ac11737b9ede5ffb8aca2a776f54ebce3e09d07fa9432f58f356eef046a6: Status 404 returned error can't find the container with id 1b79ac11737b9ede5ffb8aca2a776f54ebce3e09d07fa9432f58f356eef046a6
Oct 30 18:43:12.768151 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:12.768037    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-cwzlx" event={"ID":"e9371581-24f3-4ce8-aae8-cc1a8614f4c0","Type":"ContainerStarted","Data":"1b79ac11737b9ede5ffb8aca2a776f54ebce3e09d07fa9432f58f356eef046a6"}
Oct 30 18:43:13.775622 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:13.775592    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-cwzlx" event={"ID":"e9371581-24f3-4ce8-aae8-cc1a8614f4c0","Type":"ContainerStarted","Data":"d3abbe4d566542ae1d313110412cda3d77e46653130c62759d4c11914534b123"}
Oct 30 18:43:13.897991 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:13.897946    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-cwzlx" podStartSLOduration=2.264806172 podStartE2EDuration="2.897932213s" totalImagesPullingTime="633.126041ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:43:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:43:13.895783979 +0000 UTC m=+558.143190004" watchObservedRunningTime="2026-10-30 18:43:13.897932213 +0000 UTC m=+558.145338238"
Oct 30 18:43:13.948441 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:13.948412    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-cwzlx"
Oct 30 18:43:16.439688 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:16.439648    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8pfh7p"]
Oct 30 18:43:16.443246 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:16.443224    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8pfh7p"
Oct 30 18:43:16.547613 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:16.547581    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Oct 30 18:43:16.549870 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:16.549848    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8pfh7p"]
Oct 30 18:43:16.605066 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:16.605031    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/01e4dc41-5ae7-4847-bf9c-59106df11788-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8pfh7p\" (UID: \"01e4dc41-5ae7-4847-bf9c-59106df11788\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8pfh7p"
Oct 30 18:43:16.605239 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:16.605158    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/01e4dc41-5ae7-4847-bf9c-59106df11788-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8pfh7p\" (UID: \"01e4dc41-5ae7-4847-bf9c-59106df11788\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8pfh7p"
Oct 30 18:43:16.605239 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:16.605216    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/01e4dc41-5ae7-4847-bf9c-59106df11788-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8pfh7p\" (UID: \"01e4dc41-5ae7-4847-bf9c-59106df11788\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8pfh7p"
Oct 30 18:43:16.605355 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:16.605253    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-42pr9\" (UniqueName: \"kubernetes.io/projected/01e4dc41-5ae7-4847-bf9c-59106df11788-kube-api-access-42pr9\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8pfh7p\" (UID: \"01e4dc41-5ae7-4847-bf9c-59106df11788\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8pfh7p"
Oct 30 18:43:16.605429 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:16.605410    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/01e4dc41-5ae7-4847-bf9c-59106df11788-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8pfh7p\" (UID: \"01e4dc41-5ae7-4847-bf9c-59106df11788\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8pfh7p"
Oct 30 18:43:16.605482 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:16.605457    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/01e4dc41-5ae7-4847-bf9c-59106df11788-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8pfh7p\" (UID: \"01e4dc41-5ae7-4847-bf9c-59106df11788\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8pfh7p"
Oct 30 18:43:16.706326 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:16.706218    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-42pr9\" (UniqueName: \"kubernetes.io/projected/01e4dc41-5ae7-4847-bf9c-59106df11788-kube-api-access-42pr9\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8pfh7p\" (UID: \"01e4dc41-5ae7-4847-bf9c-59106df11788\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8pfh7p"
Oct 30 18:43:16.706326 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:16.706293    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/01e4dc41-5ae7-4847-bf9c-59106df11788-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8pfh7p\" (UID: \"01e4dc41-5ae7-4847-bf9c-59106df11788\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8pfh7p"
Oct 30 18:43:16.706326 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:16.706317    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/01e4dc41-5ae7-4847-bf9c-59106df11788-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8pfh7p\" (UID: \"01e4dc41-5ae7-4847-bf9c-59106df11788\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8pfh7p"
Oct 30 18:43:16.706629 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:16.706345    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/01e4dc41-5ae7-4847-bf9c-59106df11788-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8pfh7p\" (UID: \"01e4dc41-5ae7-4847-bf9c-59106df11788\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8pfh7p"
Oct 30 18:43:16.706629 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:16.706398    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/01e4dc41-5ae7-4847-bf9c-59106df11788-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8pfh7p\" (UID: \"01e4dc41-5ae7-4847-bf9c-59106df11788\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8pfh7p"
Oct 30 18:43:16.706629 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:16.706422    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/01e4dc41-5ae7-4847-bf9c-59106df11788-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8pfh7p\" (UID: \"01e4dc41-5ae7-4847-bf9c-59106df11788\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8pfh7p"
Oct 30 18:43:16.706857 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:16.706835    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/01e4dc41-5ae7-4847-bf9c-59106df11788-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8pfh7p\" (UID: \"01e4dc41-5ae7-4847-bf9c-59106df11788\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8pfh7p"
Oct 30 18:43:16.706965 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:16.706938    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/01e4dc41-5ae7-4847-bf9c-59106df11788-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8pfh7p\" (UID: \"01e4dc41-5ae7-4847-bf9c-59106df11788\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8pfh7p"
Oct 30 18:43:16.707201 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:16.707179    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/01e4dc41-5ae7-4847-bf9c-59106df11788-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8pfh7p\" (UID: \"01e4dc41-5ae7-4847-bf9c-59106df11788\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8pfh7p"
Oct 30 18:43:16.708664 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:16.708643    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/01e4dc41-5ae7-4847-bf9c-59106df11788-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8pfh7p\" (UID: \"01e4dc41-5ae7-4847-bf9c-59106df11788\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8pfh7p"
Oct 30 18:43:16.709971 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:16.709954    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/01e4dc41-5ae7-4847-bf9c-59106df11788-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8pfh7p\" (UID: \"01e4dc41-5ae7-4847-bf9c-59106df11788\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8pfh7p"
Oct 30 18:43:16.769933 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:16.769896    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-42pr9\" (UniqueName: \"kubernetes.io/projected/01e4dc41-5ae7-4847-bf9c-59106df11788-kube-api-access-42pr9\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8pfh7p\" (UID: \"01e4dc41-5ae7-4847-bf9c-59106df11788\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8pfh7p"
Oct 30 18:43:17.025406 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:17.025325    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-dz9tf"
Oct 30 18:43:17.041067 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:17.041042    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-dz9tf"
Oct 30 18:43:17.057396 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:17.057367    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8pfh7p"
Oct 30 18:43:17.446229 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:17.446205    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8pfh7p"]
Oct 30 18:43:17.447149 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:43:17.447109    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod01e4dc41_5ae7_4847_bf9c_59106df11788.slice/crio-aedf4511005841ceb6ca985ca7766fb41b6cabe52e9acaaf887f1fa1aa55f385 WatchSource:0}: Error finding container aedf4511005841ceb6ca985ca7766fb41b6cabe52e9acaaf887f1fa1aa55f385: Status 404 returned error can't find the container with id aedf4511005841ceb6ca985ca7766fb41b6cabe52e9acaaf887f1fa1aa55f385
Oct 30 18:43:17.801584 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:17.801508    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8pfh7p" event={"ID":"01e4dc41-5ae7-4847-bf9c-59106df11788","Type":"ContainerStarted","Data":"aedf4511005841ceb6ca985ca7766fb41b6cabe52e9acaaf887f1fa1aa55f385"}
Oct 30 18:43:17.812832 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:17.812810    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-dz9tf"
Oct 30 18:43:18.809160 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:18.809119    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8pfh7p" event={"ID":"01e4dc41-5ae7-4847-bf9c-59106df11788","Type":"ContainerStarted","Data":"8ee8d277ebea71e37bbbe2bf4a829953dac01f5f486d9035cd6b5dffb70b8725"}
Oct 30 18:43:18.928430 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:18.928379    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8pfh7p" podStartSLOduration=2.333233038 podStartE2EDuration="2.928364969s" totalImagesPullingTime="595.131931ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:43:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:43:18.927188852 +0000 UTC m=+563.174594877" watchObservedRunningTime="2026-10-30 18:43:18.928364969 +0000 UTC m=+563.175770994"
Oct 30 18:43:19.057685 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:19.057652    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8pfh7p"
Oct 30 18:43:21.727923 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:21.727889    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-rjmdz"]
Oct 30 18:43:21.948912 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:21.948870    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-cwzlx"
Oct 30 18:43:21.961844 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:21.961814    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-cwzlx"
Oct 30 18:43:22.845669 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:22.845640    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-cwzlx"
Oct 30 18:43:27.057827 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:27.057789    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8pfh7p"
Oct 30 18:43:27.072078 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:27.072056    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8pfh7p"
Oct 30 18:43:27.786778 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:27.786742    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-rkm4t"]
Oct 30 18:43:27.793435 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:27.793417    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-rkm4t"
Oct 30 18:43:27.811906 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:27.811873    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1ab20f74-2c73-4d56-a604-aad4f8289450-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-rkm4t\" (UID: \"1ab20f74-2c73-4d56-a604-aad4f8289450\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-rkm4t"
Oct 30 18:43:27.812136 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:27.812091    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1ab20f74-2c73-4d56-a604-aad4f8289450-home\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-rkm4t\" (UID: \"1ab20f74-2c73-4d56-a604-aad4f8289450\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-rkm4t"
Oct 30 18:43:27.812255 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:27.812238    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hpgsc\" (UniqueName: \"kubernetes.io/projected/1ab20f74-2c73-4d56-a604-aad4f8289450-kube-api-access-hpgsc\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-rkm4t\" (UID: \"1ab20f74-2c73-4d56-a604-aad4f8289450\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-rkm4t"
Oct 30 18:43:27.815312 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:27.812414    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1ab20f74-2c73-4d56-a604-aad4f8289450-dshm\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-rkm4t\" (UID: \"1ab20f74-2c73-4d56-a604-aad4f8289450\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-rkm4t"
Oct 30 18:43:27.815312 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:27.812467    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1ab20f74-2c73-4d56-a604-aad4f8289450-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-rkm4t\" (UID: \"1ab20f74-2c73-4d56-a604-aad4f8289450\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-rkm4t"
Oct 30 18:43:27.815312 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:27.812560    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1ab20f74-2c73-4d56-a604-aad4f8289450-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-rkm4t\" (UID: \"1ab20f74-2c73-4d56-a604-aad4f8289450\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-rkm4t"
Oct 30 18:43:27.869836 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:27.869801    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-rkm4t"]
Oct 30 18:43:27.899643 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:27.899614    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Oct 30 18:43:27.911805 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:27.911775    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8pfh7p"
Oct 30 18:43:27.912997 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:27.912978    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1ab20f74-2c73-4d56-a604-aad4f8289450-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-rkm4t\" (UID: \"1ab20f74-2c73-4d56-a604-aad4f8289450\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-rkm4t"
Oct 30 18:43:27.913053 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:27.913036    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1ab20f74-2c73-4d56-a604-aad4f8289450-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-rkm4t\" (UID: \"1ab20f74-2c73-4d56-a604-aad4f8289450\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-rkm4t"
Oct 30 18:43:27.913118 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:27.913101    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1ab20f74-2c73-4d56-a604-aad4f8289450-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-rkm4t\" (UID: \"1ab20f74-2c73-4d56-a604-aad4f8289450\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-rkm4t"
Oct 30 18:43:27.913179 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:27.913137    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1ab20f74-2c73-4d56-a604-aad4f8289450-home\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-rkm4t\" (UID: \"1ab20f74-2c73-4d56-a604-aad4f8289450\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-rkm4t"
Oct 30 18:43:27.913179 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:27.913162    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hpgsc\" (UniqueName: \"kubernetes.io/projected/1ab20f74-2c73-4d56-a604-aad4f8289450-kube-api-access-hpgsc\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-rkm4t\" (UID: \"1ab20f74-2c73-4d56-a604-aad4f8289450\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-rkm4t"
Oct 30 18:43:27.913324 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:27.913188    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1ab20f74-2c73-4d56-a604-aad4f8289450-dshm\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-rkm4t\" (UID: \"1ab20f74-2c73-4d56-a604-aad4f8289450\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-rkm4t"
Oct 30 18:43:27.913514 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:27.913494    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1ab20f74-2c73-4d56-a604-aad4f8289450-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-rkm4t\" (UID: \"1ab20f74-2c73-4d56-a604-aad4f8289450\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-rkm4t"
Oct 30 18:43:27.913731 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:27.913703    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1ab20f74-2c73-4d56-a604-aad4f8289450-home\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-rkm4t\" (UID: \"1ab20f74-2c73-4d56-a604-aad4f8289450\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-rkm4t"
Oct 30 18:43:27.913809 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:27.913739    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1ab20f74-2c73-4d56-a604-aad4f8289450-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-rkm4t\" (UID: \"1ab20f74-2c73-4d56-a604-aad4f8289450\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-rkm4t"
Oct 30 18:43:27.915960 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:27.915930    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1ab20f74-2c73-4d56-a604-aad4f8289450-dshm\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-rkm4t\" (UID: \"1ab20f74-2c73-4d56-a604-aad4f8289450\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-rkm4t"
Oct 30 18:43:27.916076 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:27.916031    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1ab20f74-2c73-4d56-a604-aad4f8289450-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-rkm4t\" (UID: \"1ab20f74-2c73-4d56-a604-aad4f8289450\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-rkm4t"
Oct 30 18:43:27.978672 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:27.978636    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hpgsc\" (UniqueName: \"kubernetes.io/projected/1ab20f74-2c73-4d56-a604-aad4f8289450-kube-api-access-hpgsc\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-rkm4t\" (UID: \"1ab20f74-2c73-4d56-a604-aad4f8289450\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-rkm4t"
Oct 30 18:43:28.120348 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:28.120311    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-rkm4t"
Oct 30 18:43:28.502133 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:28.502110    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-rkm4t"]
Oct 30 18:43:28.504791 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:43:28.504760    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1ab20f74_2c73_4d56_a604_aad4f8289450.slice/crio-6c5dcfcd6799f9f6e31bff9aab9f7b7540f294117f2d153b0ca594ef0204fbcd WatchSource:0}: Error finding container 6c5dcfcd6799f9f6e31bff9aab9f7b7540f294117f2d153b0ca594ef0204fbcd: Status 404 returned error can't find the container with id 6c5dcfcd6799f9f6e31bff9aab9f7b7540f294117f2d153b0ca594ef0204fbcd
Oct 30 18:43:28.902766 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:28.902728    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-rkm4t" event={"ID":"1ab20f74-2c73-4d56-a604-aad4f8289450","Type":"ContainerStarted","Data":"6c5dcfcd6799f9f6e31bff9aab9f7b7540f294117f2d153b0ca594ef0204fbcd"}
Oct 30 18:43:29.935569 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:29.935529    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-rkm4t" event={"ID":"1ab20f74-2c73-4d56-a604-aad4f8289450","Type":"ContainerStarted","Data":"0a212c3266bdcddcd4916064c2404062d5f4e64ca3acb98cdfab5aeba044cc85"}
Oct 30 18:43:30.057296 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:30.057237    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-rkm4t" podStartSLOduration=2.448147747 podStartE2EDuration="3.057224731s" totalImagesPullingTime="609.076984ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:43:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:43:30.056461937 +0000 UTC m=+574.303867961" watchObservedRunningTime="2026-10-30 18:43:30.057224731 +0000 UTC m=+574.304630756"
Oct 30 18:43:30.121089 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:30.121050    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-rkm4t"
Oct 30 18:43:35.617623 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:43:35.617503    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-jbsw6" containerName="extractor"
Oct 30 18:43:36.216613 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:36.216574    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-rjmdz"]
Oct 30 18:43:38.120448 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:38.120410    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-rkm4t"
Oct 30 18:43:38.137890 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:38.137863    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-rkm4t"
Oct 30 18:43:38.995984 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:38.995958    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-rkm4t"
Oct 30 18:43:51.099385 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:51.099342    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-66db998887-wlrgd"]
Oct 30 18:43:51.102997 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:51.102978    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-wlrgd"
Oct 30 18:43:51.211640 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:51.211612    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-66db998887-wlrgd"]
Oct 30 18:43:51.217690 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:51.217662    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/36c8e780-ee5d-41e1-899f-99b9614c84ad-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-66db998887-wlrgd\" (UID: \"36c8e780-ee5d-41e1-899f-99b9614c84ad\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-wlrgd"
Oct 30 18:43:51.217795 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:51.217713    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/36c8e780-ee5d-41e1-899f-99b9614c84ad-workload-certs\") pod \"maas-default-gateway-openshift-default-66db998887-wlrgd\" (UID: \"36c8e780-ee5d-41e1-899f-99b9614c84ad\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-wlrgd"
Oct 30 18:43:51.217795 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:51.217734    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/36c8e780-ee5d-41e1-899f-99b9614c84ad-istio-podinfo\") pod \"maas-default-gateway-openshift-default-66db998887-wlrgd\" (UID: \"36c8e780-ee5d-41e1-899f-99b9614c84ad\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-wlrgd"
Oct 30 18:43:51.217905 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:51.217878    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/36c8e780-ee5d-41e1-899f-99b9614c84ad-istio-token\") pod \"maas-default-gateway-openshift-default-66db998887-wlrgd\" (UID: \"36c8e780-ee5d-41e1-899f-99b9614c84ad\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-wlrgd"
Oct 30 18:43:51.217952 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:51.217938    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/36c8e780-ee5d-41e1-899f-99b9614c84ad-istio-envoy\") pod \"maas-default-gateway-openshift-default-66db998887-wlrgd\" (UID: \"36c8e780-ee5d-41e1-899f-99b9614c84ad\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-wlrgd"
Oct 30 18:43:51.217986 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:51.217968    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2kmwl\" (UniqueName: \"kubernetes.io/projected/36c8e780-ee5d-41e1-899f-99b9614c84ad-kube-api-access-2kmwl\") pod \"maas-default-gateway-openshift-default-66db998887-wlrgd\" (UID: \"36c8e780-ee5d-41e1-899f-99b9614c84ad\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-wlrgd"
Oct 30 18:43:51.218072 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:51.218049    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/36c8e780-ee5d-41e1-899f-99b9614c84ad-workload-socket\") pod \"maas-default-gateway-openshift-default-66db998887-wlrgd\" (UID: \"36c8e780-ee5d-41e1-899f-99b9614c84ad\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-wlrgd"
Oct 30 18:43:51.218116 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:51.218100    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/36c8e780-ee5d-41e1-899f-99b9614c84ad-istio-data\") pod \"maas-default-gateway-openshift-default-66db998887-wlrgd\" (UID: \"36c8e780-ee5d-41e1-899f-99b9614c84ad\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-wlrgd"
Oct 30 18:43:51.218226 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:51.218207    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/36c8e780-ee5d-41e1-899f-99b9614c84ad-credential-socket\") pod \"maas-default-gateway-openshift-default-66db998887-wlrgd\" (UID: \"36c8e780-ee5d-41e1-899f-99b9614c84ad\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-wlrgd"
Oct 30 18:43:51.318854 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:51.318812    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/36c8e780-ee5d-41e1-899f-99b9614c84ad-istio-data\") pod \"maas-default-gateway-openshift-default-66db998887-wlrgd\" (UID: \"36c8e780-ee5d-41e1-899f-99b9614c84ad\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-wlrgd"
Oct 30 18:43:51.319056 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:51.318871    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/36c8e780-ee5d-41e1-899f-99b9614c84ad-credential-socket\") pod \"maas-default-gateway-openshift-default-66db998887-wlrgd\" (UID: \"36c8e780-ee5d-41e1-899f-99b9614c84ad\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-wlrgd"
Oct 30 18:43:51.319056 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:51.318905    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/36c8e780-ee5d-41e1-899f-99b9614c84ad-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-66db998887-wlrgd\" (UID: \"36c8e780-ee5d-41e1-899f-99b9614c84ad\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-wlrgd"
Oct 30 18:43:51.319056 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:51.318935    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/36c8e780-ee5d-41e1-899f-99b9614c84ad-workload-certs\") pod \"maas-default-gateway-openshift-default-66db998887-wlrgd\" (UID: \"36c8e780-ee5d-41e1-899f-99b9614c84ad\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-wlrgd"
Oct 30 18:43:51.319056 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:51.318950    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/36c8e780-ee5d-41e1-899f-99b9614c84ad-istio-podinfo\") pod \"maas-default-gateway-openshift-default-66db998887-wlrgd\" (UID: \"36c8e780-ee5d-41e1-899f-99b9614c84ad\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-wlrgd"
Oct 30 18:43:51.319056 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:51.318977    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/36c8e780-ee5d-41e1-899f-99b9614c84ad-istio-token\") pod \"maas-default-gateway-openshift-default-66db998887-wlrgd\" (UID: \"36c8e780-ee5d-41e1-899f-99b9614c84ad\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-wlrgd"
Oct 30 18:43:51.319056 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:51.319015    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/36c8e780-ee5d-41e1-899f-99b9614c84ad-istio-envoy\") pod \"maas-default-gateway-openshift-default-66db998887-wlrgd\" (UID: \"36c8e780-ee5d-41e1-899f-99b9614c84ad\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-wlrgd"
Oct 30 18:43:51.319056 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:51.319043    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2kmwl\" (UniqueName: \"kubernetes.io/projected/36c8e780-ee5d-41e1-899f-99b9614c84ad-kube-api-access-2kmwl\") pod \"maas-default-gateway-openshift-default-66db998887-wlrgd\" (UID: \"36c8e780-ee5d-41e1-899f-99b9614c84ad\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-wlrgd"
Oct 30 18:43:51.319480 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:51.319071    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/36c8e780-ee5d-41e1-899f-99b9614c84ad-workload-socket\") pod \"maas-default-gateway-openshift-default-66db998887-wlrgd\" (UID: \"36c8e780-ee5d-41e1-899f-99b9614c84ad\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-wlrgd"
Oct 30 18:43:51.319480 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:51.319361    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/36c8e780-ee5d-41e1-899f-99b9614c84ad-istio-data\") pod \"maas-default-gateway-openshift-default-66db998887-wlrgd\" (UID: \"36c8e780-ee5d-41e1-899f-99b9614c84ad\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-wlrgd"
Oct 30 18:43:51.319480 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:51.319447    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/36c8e780-ee5d-41e1-899f-99b9614c84ad-workload-certs\") pod \"maas-default-gateway-openshift-default-66db998887-wlrgd\" (UID: \"36c8e780-ee5d-41e1-899f-99b9614c84ad\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-wlrgd"
Oct 30 18:43:51.319480 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:51.319462    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/36c8e780-ee5d-41e1-899f-99b9614c84ad-workload-socket\") pod \"maas-default-gateway-openshift-default-66db998887-wlrgd\" (UID: \"36c8e780-ee5d-41e1-899f-99b9614c84ad\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-wlrgd"
Oct 30 18:43:51.319833 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:51.319810    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/36c8e780-ee5d-41e1-899f-99b9614c84ad-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-66db998887-wlrgd\" (UID: \"36c8e780-ee5d-41e1-899f-99b9614c84ad\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-wlrgd"
Oct 30 18:43:51.319872 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:51.319811    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/36c8e780-ee5d-41e1-899f-99b9614c84ad-credential-socket\") pod \"maas-default-gateway-openshift-default-66db998887-wlrgd\" (UID: \"36c8e780-ee5d-41e1-899f-99b9614c84ad\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-wlrgd"
Oct 30 18:43:51.321556 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:51.321537    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/36c8e780-ee5d-41e1-899f-99b9614c84ad-istio-envoy\") pod \"maas-default-gateway-openshift-default-66db998887-wlrgd\" (UID: \"36c8e780-ee5d-41e1-899f-99b9614c84ad\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-wlrgd"
Oct 30 18:43:51.321686 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:51.321668    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/36c8e780-ee5d-41e1-899f-99b9614c84ad-istio-podinfo\") pod \"maas-default-gateway-openshift-default-66db998887-wlrgd\" (UID: \"36c8e780-ee5d-41e1-899f-99b9614c84ad\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-wlrgd"
Oct 30 18:43:51.382324 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:51.382263    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/36c8e780-ee5d-41e1-899f-99b9614c84ad-istio-token\") pod \"maas-default-gateway-openshift-default-66db998887-wlrgd\" (UID: \"36c8e780-ee5d-41e1-899f-99b9614c84ad\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-wlrgd"
Oct 30 18:43:51.382957 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:51.382936    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2kmwl\" (UniqueName: \"kubernetes.io/projected/36c8e780-ee5d-41e1-899f-99b9614c84ad-kube-api-access-2kmwl\") pod \"maas-default-gateway-openshift-default-66db998887-wlrgd\" (UID: \"36c8e780-ee5d-41e1-899f-99b9614c84ad\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-wlrgd"
Oct 30 18:43:51.415367 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:51.415343    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-wlrgd"
Oct 30 18:43:51.842453 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:51.842427    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-66db998887-wlrgd"]
Oct 30 18:43:51.846799 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:43:51.846772    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36c8e780_ee5d_41e1_899f_99b9614c84ad.slice/crio-5d5fe36e4064275e67dde7c3a43737ec570a4f928d03712f5298b8b1fb847577 WatchSource:0}: Error finding container 5d5fe36e4064275e67dde7c3a43737ec570a4f928d03712f5298b8b1fb847577: Status 404 returned error can't find the container with id 5d5fe36e4064275e67dde7c3a43737ec570a4f928d03712f5298b8b1fb847577
Oct 30 18:43:51.848958 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:51.848931    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:43:51.849022 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:51.848994    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:43:51.849062 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:51.849023    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:43:52.055686 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:52.055651    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-wlrgd" event={"ID":"36c8e780-ee5d-41e1-899f-99b9614c84ad","Type":"ContainerStarted","Data":"dbad0b77cb998b7ee7ebd31f8462c53b27dd7ceae36ea003f51590f332358a1a"}
Oct 30 18:43:52.055859 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:52.055706    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-wlrgd" event={"ID":"36c8e780-ee5d-41e1-899f-99b9614c84ad","Type":"ContainerStarted","Data":"5d5fe36e4064275e67dde7c3a43737ec570a4f928d03712f5298b8b1fb847577"}
Oct 30 18:43:52.181981 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:52.181866    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-wlrgd" podStartSLOduration=1.181844631 podStartE2EDuration="1.181844631s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:43:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:43:52.178755575 +0000 UTC m=+596.426161605" watchObservedRunningTime="2026-10-30 18:43:52.181844631 +0000 UTC m=+596.429250657"
Oct 30 18:43:52.416456 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:52.416419    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-wlrgd"
Oct 30 18:43:52.418090 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:52.418065    4888 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-66db998887-wlrgd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body=
Oct 30 18:43:52.418204 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:52.418118    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-wlrgd" podUID="36c8e780-ee5d-41e1-899f-99b9614c84ad" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused"
Oct 30 18:43:53.415721 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:53.415687    4888 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-66db998887-wlrgd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body=
Oct 30 18:43:53.416149 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:53.415740    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-wlrgd" podUID="36c8e780-ee5d-41e1-899f-99b9614c84ad" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused"
Oct 30 18:43:54.415998 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:54.415966    4888 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-66db998887-wlrgd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body=
Oct 30 18:43:54.416484 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:54.416018    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-wlrgd" podUID="36c8e780-ee5d-41e1-899f-99b9614c84ad" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused"
Oct 30 18:43:55.416576 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:55.416541    4888 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-66db998887-wlrgd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body=
Oct 30 18:43:55.417024 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:55.416603    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-wlrgd" podUID="36c8e780-ee5d-41e1-899f-99b9614c84ad" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused"
Oct 30 18:43:55.945796 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:55.945767    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-7x4rb_382e8b31-a7fa-43f3-9f15-4c61627b0c04/console-operator/2.log"
Oct 30 18:43:55.946655 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:55.946638    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-7x4rb_382e8b31-a7fa-43f3-9f15-4c61627b0c04/console-operator/2.log"
Oct 30 18:43:55.953268 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:55.953248    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xfcdl_3f3fe764-2652-46c9-8e54-7882c0e6bcf1/ovn-acl-logging/0.log"
Oct 30 18:43:55.954179 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:55.954162    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xfcdl_3f3fe764-2652-46c9-8e54-7882c0e6bcf1/ovn-acl-logging/0.log"
Oct 30 18:43:56.416049 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:56.416020    4888 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-66db998887-wlrgd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body=
Oct 30 18:43:56.416244 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:56.416070    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-wlrgd" podUID="36c8e780-ee5d-41e1-899f-99b9614c84ad" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused"
Oct 30 18:43:57.416588 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:57.416549    4888 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-66db998887-wlrgd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body=
Oct 30 18:43:57.416996 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:57.416604    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-wlrgd" podUID="36c8e780-ee5d-41e1-899f-99b9614c84ad" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused"
Oct 30 18:43:58.415905 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:58.415869    4888 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-66db998887-wlrgd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body=
Oct 30 18:43:58.416070 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:58.415949    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-wlrgd" podUID="36c8e780-ee5d-41e1-899f-99b9614c84ad" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused"
Oct 30 18:43:59.906006 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:59.905973    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5f5864cd46-846ft"]
Oct 30 18:43:59.909852 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:59.909830    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5f5864cd46-846ft"
Oct 30 18:43:59.971357 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:43:59.971332    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5f5864cd46-846ft"]
Oct 30 18:44:00.020149 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:00.020117    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lf87d\" (UniqueName: \"kubernetes.io/projected/bfa5d691-05e1-48a0-9d53-52975ea89cf4-kube-api-access-lf87d\") pod \"authorino-5f5864cd46-846ft\" (UID: \"bfa5d691-05e1-48a0-9d53-52975ea89cf4\") " pod="kuadrant-system/authorino-5f5864cd46-846ft"
Oct 30 18:44:00.020338 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:00.020163    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/bfa5d691-05e1-48a0-9d53-52975ea89cf4-tls-cert\") pod \"authorino-5f5864cd46-846ft\" (UID: \"bfa5d691-05e1-48a0-9d53-52975ea89cf4\") " pod="kuadrant-system/authorino-5f5864cd46-846ft"
Oct 30 18:44:00.068063 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:00.068031    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-wlrgd"
Oct 30 18:44:00.068528 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:00.068491    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-wlrgd"
Oct 30 18:44:00.069421 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:00.069398    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-wlrgd"
Oct 30 18:44:00.120908 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:00.120878    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lf87d\" (UniqueName: \"kubernetes.io/projected/bfa5d691-05e1-48a0-9d53-52975ea89cf4-kube-api-access-lf87d\") pod \"authorino-5f5864cd46-846ft\" (UID: \"bfa5d691-05e1-48a0-9d53-52975ea89cf4\") " pod="kuadrant-system/authorino-5f5864cd46-846ft"
Oct 30 18:44:00.121040 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:00.120918    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/bfa5d691-05e1-48a0-9d53-52975ea89cf4-tls-cert\") pod \"authorino-5f5864cd46-846ft\" (UID: \"bfa5d691-05e1-48a0-9d53-52975ea89cf4\") " pod="kuadrant-system/authorino-5f5864cd46-846ft"
Oct 30 18:44:00.123709 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:00.123689    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/bfa5d691-05e1-48a0-9d53-52975ea89cf4-tls-cert\") pod \"authorino-5f5864cd46-846ft\" (UID: \"bfa5d691-05e1-48a0-9d53-52975ea89cf4\") " pod="kuadrant-system/authorino-5f5864cd46-846ft"
Oct 30 18:44:00.183718 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:00.183654    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lf87d\" (UniqueName: \"kubernetes.io/projected/bfa5d691-05e1-48a0-9d53-52975ea89cf4-kube-api-access-lf87d\") pod \"authorino-5f5864cd46-846ft\" (UID: \"bfa5d691-05e1-48a0-9d53-52975ea89cf4\") " pod="kuadrant-system/authorino-5f5864cd46-846ft"
Oct 30 18:44:00.222116 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:00.222087    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5f5864cd46-846ft"
Oct 30 18:44:00.674201 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:00.674169    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5f5864cd46-846ft"]
Oct 30 18:44:00.674585 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:44:00.674559    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbfa5d691_05e1_48a0_9d53_52975ea89cf4.slice/crio-f3607c6e11ffdbe5a6b04bf1b815057ba9be8dcbc8d46a7d0d88fb7f155e2a6d WatchSource:0}: Error finding container f3607c6e11ffdbe5a6b04bf1b815057ba9be8dcbc8d46a7d0d88fb7f155e2a6d: Status 404 returned error can't find the container with id f3607c6e11ffdbe5a6b04bf1b815057ba9be8dcbc8d46a7d0d88fb7f155e2a6d
Oct 30 18:44:01.127883 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:01.127844    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5f5864cd46-846ft" event={"ID":"bfa5d691-05e1-48a0-9d53-52975ea89cf4","Type":"ContainerStarted","Data":"f3607c6e11ffdbe5a6b04bf1b815057ba9be8dcbc8d46a7d0d88fb7f155e2a6d"}
Oct 30 18:44:03.142092 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:03.142039    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5f5864cd46-846ft" event={"ID":"bfa5d691-05e1-48a0-9d53-52975ea89cf4","Type":"ContainerStarted","Data":"71bf1dd9579df1887b7c89ecff1aef8aab1fb3ff41fda060656877eaefa4c337"}
Oct 30 18:44:03.264326 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:03.264234    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/authorino-5f5864cd46-846ft" podStartSLOduration=2.727076459 podStartE2EDuration="4.264212631s" totalImagesPullingTime="1.537136172s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:43:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:44:03.260732494 +0000 UTC m=+607.508138542" watchObservedRunningTime="2026-10-30 18:44:03.264212631 +0000 UTC m=+607.511618714"
Oct 30 18:44:03.317229 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:03.317191    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-57c95767fc-6lkr8"]
Oct 30 18:44:03.319004 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:03.318950    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/authorino-57c95767fc-6lkr8" podUID="91b5bf50-ee12-4846-92c2-7f30dfb0b4f2" containerName="authorino" containerID="cri-o://c211a7b5c7e61ccf74f7c34e395c023db1b63e6a7827fc18c854582204d36b2c" gracePeriod=30
Oct 30 18:44:03.618488 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:03.618462    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-57c95767fc-6lkr8"
Oct 30 18:44:03.756961 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:03.756876    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/91b5bf50-ee12-4846-92c2-7f30dfb0b4f2-tls-cert\" (UniqueName: \"kubernetes.io/secret/91b5bf50-ee12-4846-92c2-7f30dfb0b4f2-tls-cert\") pod \"91b5bf50-ee12-4846-92c2-7f30dfb0b4f2\" (UID: \"91b5bf50-ee12-4846-92c2-7f30dfb0b4f2\") "
Oct 30 18:44:03.756961 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:03.756928    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/91b5bf50-ee12-4846-92c2-7f30dfb0b4f2-kube-api-access-5pv24\" (UniqueName: \"kubernetes.io/projected/91b5bf50-ee12-4846-92c2-7f30dfb0b4f2-kube-api-access-5pv24\") pod \"91b5bf50-ee12-4846-92c2-7f30dfb0b4f2\" (UID: \"91b5bf50-ee12-4846-92c2-7f30dfb0b4f2\") "
Oct 30 18:44:03.759139 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:03.759110    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/91b5bf50-ee12-4846-92c2-7f30dfb0b4f2-kube-api-access-5pv24" pod "91b5bf50-ee12-4846-92c2-7f30dfb0b4f2" (UID: "91b5bf50-ee12-4846-92c2-7f30dfb0b4f2"). InnerVolumeSpecName "kube-api-access-5pv24". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:44:03.770089 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:03.770062    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/91b5bf50-ee12-4846-92c2-7f30dfb0b4f2-tls-cert" pod "91b5bf50-ee12-4846-92c2-7f30dfb0b4f2" (UID: "91b5bf50-ee12-4846-92c2-7f30dfb0b4f2"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:44:03.858632 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:03.858604    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5pv24\" (UniqueName: \"kubernetes.io/projected/91b5bf50-ee12-4846-92c2-7f30dfb0b4f2-kube-api-access-5pv24\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:44:03.858632 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:03.858630    4888 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/91b5bf50-ee12-4846-92c2-7f30dfb0b4f2-tls-cert\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:44:04.150537 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:04.150496    4888 generic.go:356] "Generic (PLEG): container finished" podID="91b5bf50-ee12-4846-92c2-7f30dfb0b4f2" containerID="c211a7b5c7e61ccf74f7c34e395c023db1b63e6a7827fc18c854582204d36b2c" exitCode=0
Oct 30 18:44:04.151033 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:04.150552    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-57c95767fc-6lkr8"
Oct 30 18:44:04.151033 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:04.150590    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-57c95767fc-6lkr8" event={"ID":"91b5bf50-ee12-4846-92c2-7f30dfb0b4f2","Type":"ContainerDied","Data":"c211a7b5c7e61ccf74f7c34e395c023db1b63e6a7827fc18c854582204d36b2c"}
Oct 30 18:44:04.151033 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:04.150632    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-57c95767fc-6lkr8" event={"ID":"91b5bf50-ee12-4846-92c2-7f30dfb0b4f2","Type":"ContainerDied","Data":"4a34022844975c2a69af57fc1fd411317a41ce0baea5b2a533a694579c59b4ab"}
Oct 30 18:44:04.151033 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:04.150654    4888 scope.go:122] "RemoveContainer" containerID="c211a7b5c7e61ccf74f7c34e395c023db1b63e6a7827fc18c854582204d36b2c"
Oct 30 18:44:04.160186 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:04.160165    4888 scope.go:122] "RemoveContainer" containerID="c211a7b5c7e61ccf74f7c34e395c023db1b63e6a7827fc18c854582204d36b2c"
Oct 30 18:44:04.160464 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:44:04.160442    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c211a7b5c7e61ccf74f7c34e395c023db1b63e6a7827fc18c854582204d36b2c\": container with ID starting with c211a7b5c7e61ccf74f7c34e395c023db1b63e6a7827fc18c854582204d36b2c not found: ID does not exist" containerID="c211a7b5c7e61ccf74f7c34e395c023db1b63e6a7827fc18c854582204d36b2c"
Oct 30 18:44:04.160551 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:04.160477    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c211a7b5c7e61ccf74f7c34e395c023db1b63e6a7827fc18c854582204d36b2c"} err="failed to get container status \"c211a7b5c7e61ccf74f7c34e395c023db1b63e6a7827fc18c854582204d36b2c\": rpc error: code = NotFound desc = could not find container \"c211a7b5c7e61ccf74f7c34e395c023db1b63e6a7827fc18c854582204d36b2c\": container with ID starting with c211a7b5c7e61ccf74f7c34e395c023db1b63e6a7827fc18c854582204d36b2c not found: ID does not exist"
Oct 30 18:44:04.343999 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:04.343968    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-57c95767fc-6lkr8"]
Oct 30 18:44:04.353008 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:04.352984    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-57c95767fc-6lkr8"]
Oct 30 18:44:05.619212 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:44:05.619175    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-jbsw6" containerName="extractor"
Oct 30 18:44:05.891014 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:05.890938    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="91b5bf50-ee12-4846-92c2-7f30dfb0b4f2" path="/var/lib/kubelet/pods/91b5bf50-ee12-4846-92c2-7f30dfb0b4f2/volumes"
Oct 30 18:44:06.226835 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:06.226702    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-66db998887-6v8vj"]
Oct 30 18:44:06.227249 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:06.227229    4888 cpu_manager.go:414] "containerMap: removing container" podUID="91b5bf50-ee12-4846-92c2-7f30dfb0b4f2" containerName="authorino"
Oct 30 18:44:06.227387 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:06.227251    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="91b5bf50-ee12-4846-92c2-7f30dfb0b4f2" containerName="authorino"
Oct 30 18:44:06.227454 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:06.227388    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="91b5bf50-ee12-4846-92c2-7f30dfb0b4f2" containerName="authorino"
Oct 30 18:44:06.231919 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:06.231893    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-6v8vj"
Oct 30 18:44:06.386942 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:06.386908    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5fc32c21-43ea-4089-b2c0-3d416459607d-credential-socket\") pod \"maas-default-gateway-openshift-default-66db998887-6v8vj\" (UID: \"5fc32c21-43ea-4089-b2c0-3d416459607d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-6v8vj"
Oct 30 18:44:06.386942 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:06.386951    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5fc32c21-43ea-4089-b2c0-3d416459607d-istio-token\") pod \"maas-default-gateway-openshift-default-66db998887-6v8vj\" (UID: \"5fc32c21-43ea-4089-b2c0-3d416459607d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-6v8vj"
Oct 30 18:44:06.387160 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:06.387047    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5fc32c21-43ea-4089-b2c0-3d416459607d-workload-certs\") pod \"maas-default-gateway-openshift-default-66db998887-6v8vj\" (UID: \"5fc32c21-43ea-4089-b2c0-3d416459607d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-6v8vj"
Oct 30 18:44:06.387160 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:06.387104    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5fc32c21-43ea-4089-b2c0-3d416459607d-istio-data\") pod \"maas-default-gateway-openshift-default-66db998887-6v8vj\" (UID: \"5fc32c21-43ea-4089-b2c0-3d416459607d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-6v8vj"
Oct 30 18:44:06.387227 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:06.387153    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5fc32c21-43ea-4089-b2c0-3d416459607d-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-66db998887-6v8vj\" (UID: \"5fc32c21-43ea-4089-b2c0-3d416459607d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-6v8vj"
Oct 30 18:44:06.387227 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:06.387201    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r677k\" (UniqueName: \"kubernetes.io/projected/5fc32c21-43ea-4089-b2c0-3d416459607d-kube-api-access-r677k\") pod \"maas-default-gateway-openshift-default-66db998887-6v8vj\" (UID: \"5fc32c21-43ea-4089-b2c0-3d416459607d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-6v8vj"
Oct 30 18:44:06.387316 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:06.387256    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5fc32c21-43ea-4089-b2c0-3d416459607d-istio-podinfo\") pod \"maas-default-gateway-openshift-default-66db998887-6v8vj\" (UID: \"5fc32c21-43ea-4089-b2c0-3d416459607d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-6v8vj"
Oct 30 18:44:06.387354 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:06.387336    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5fc32c21-43ea-4089-b2c0-3d416459607d-istio-envoy\") pod \"maas-default-gateway-openshift-default-66db998887-6v8vj\" (UID: \"5fc32c21-43ea-4089-b2c0-3d416459607d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-6v8vj"
Oct 30 18:44:06.387385 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:06.387370    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5fc32c21-43ea-4089-b2c0-3d416459607d-workload-socket\") pod \"maas-default-gateway-openshift-default-66db998887-6v8vj\" (UID: \"5fc32c21-43ea-4089-b2c0-3d416459607d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-6v8vj"
Oct 30 18:44:06.450161 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:06.450125    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-66db998887-6v8vj"]
Oct 30 18:44:06.488432 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:06.488341    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5fc32c21-43ea-4089-b2c0-3d416459607d-istio-podinfo\") pod \"maas-default-gateway-openshift-default-66db998887-6v8vj\" (UID: \"5fc32c21-43ea-4089-b2c0-3d416459607d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-6v8vj"
Oct 30 18:44:06.488432 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:06.488397    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5fc32c21-43ea-4089-b2c0-3d416459607d-istio-envoy\") pod \"maas-default-gateway-openshift-default-66db998887-6v8vj\" (UID: \"5fc32c21-43ea-4089-b2c0-3d416459607d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-6v8vj"
Oct 30 18:44:06.488432 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:06.488419    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5fc32c21-43ea-4089-b2c0-3d416459607d-workload-socket\") pod \"maas-default-gateway-openshift-default-66db998887-6v8vj\" (UID: \"5fc32c21-43ea-4089-b2c0-3d416459607d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-6v8vj"
Oct 30 18:44:06.488702 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:06.488454    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5fc32c21-43ea-4089-b2c0-3d416459607d-credential-socket\") pod \"maas-default-gateway-openshift-default-66db998887-6v8vj\" (UID: \"5fc32c21-43ea-4089-b2c0-3d416459607d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-6v8vj"
Oct 30 18:44:06.488702 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:06.488474    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5fc32c21-43ea-4089-b2c0-3d416459607d-istio-token\") pod \"maas-default-gateway-openshift-default-66db998887-6v8vj\" (UID: \"5fc32c21-43ea-4089-b2c0-3d416459607d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-6v8vj"
Oct 30 18:44:06.488702 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:06.488521    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5fc32c21-43ea-4089-b2c0-3d416459607d-workload-certs\") pod \"maas-default-gateway-openshift-default-66db998887-6v8vj\" (UID: \"5fc32c21-43ea-4089-b2c0-3d416459607d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-6v8vj"
Oct 30 18:44:06.488702 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:06.488579    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5fc32c21-43ea-4089-b2c0-3d416459607d-istio-data\") pod \"maas-default-gateway-openshift-default-66db998887-6v8vj\" (UID: \"5fc32c21-43ea-4089-b2c0-3d416459607d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-6v8vj"
Oct 30 18:44:06.488702 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:06.488610    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5fc32c21-43ea-4089-b2c0-3d416459607d-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-66db998887-6v8vj\" (UID: \"5fc32c21-43ea-4089-b2c0-3d416459607d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-6v8vj"
Oct 30 18:44:06.488702 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:06.488638    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r677k\" (UniqueName: \"kubernetes.io/projected/5fc32c21-43ea-4089-b2c0-3d416459607d-kube-api-access-r677k\") pod \"maas-default-gateway-openshift-default-66db998887-6v8vj\" (UID: \"5fc32c21-43ea-4089-b2c0-3d416459607d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-6v8vj"
Oct 30 18:44:06.489057 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:06.488879    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5fc32c21-43ea-4089-b2c0-3d416459607d-credential-socket\") pod \"maas-default-gateway-openshift-default-66db998887-6v8vj\" (UID: \"5fc32c21-43ea-4089-b2c0-3d416459607d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-6v8vj"
Oct 30 18:44:06.489057 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:06.488941    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5fc32c21-43ea-4089-b2c0-3d416459607d-workload-socket\") pod \"maas-default-gateway-openshift-default-66db998887-6v8vj\" (UID: \"5fc32c21-43ea-4089-b2c0-3d416459607d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-6v8vj"
Oct 30 18:44:06.489162 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:06.489118    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5fc32c21-43ea-4089-b2c0-3d416459607d-workload-certs\") pod \"maas-default-gateway-openshift-default-66db998887-6v8vj\" (UID: \"5fc32c21-43ea-4089-b2c0-3d416459607d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-6v8vj"
Oct 30 18:44:06.489201 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:06.489176    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5fc32c21-43ea-4089-b2c0-3d416459607d-istio-data\") pod \"maas-default-gateway-openshift-default-66db998887-6v8vj\" (UID: \"5fc32c21-43ea-4089-b2c0-3d416459607d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-6v8vj"
Oct 30 18:44:06.489503 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:06.489435    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5fc32c21-43ea-4089-b2c0-3d416459607d-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-66db998887-6v8vj\" (UID: \"5fc32c21-43ea-4089-b2c0-3d416459607d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-6v8vj"
Oct 30 18:44:06.491188 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:06.491160    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5fc32c21-43ea-4089-b2c0-3d416459607d-istio-envoy\") pod \"maas-default-gateway-openshift-default-66db998887-6v8vj\" (UID: \"5fc32c21-43ea-4089-b2c0-3d416459607d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-6v8vj"
Oct 30 18:44:06.491428 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:06.491411    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5fc32c21-43ea-4089-b2c0-3d416459607d-istio-podinfo\") pod \"maas-default-gateway-openshift-default-66db998887-6v8vj\" (UID: \"5fc32c21-43ea-4089-b2c0-3d416459607d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-6v8vj"
Oct 30 18:44:06.565306 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:06.565007    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r677k\" (UniqueName: \"kubernetes.io/projected/5fc32c21-43ea-4089-b2c0-3d416459607d-kube-api-access-r677k\") pod \"maas-default-gateway-openshift-default-66db998887-6v8vj\" (UID: \"5fc32c21-43ea-4089-b2c0-3d416459607d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-6v8vj"
Oct 30 18:44:06.574310 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:06.573796    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5fc32c21-43ea-4089-b2c0-3d416459607d-istio-token\") pod \"maas-default-gateway-openshift-default-66db998887-6v8vj\" (UID: \"5fc32c21-43ea-4089-b2c0-3d416459607d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-6v8vj"
Oct 30 18:44:06.845475 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:06.845442    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-6v8vj"
Oct 30 18:44:07.280450 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:07.280422    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-66db998887-6v8vj"]
Oct 30 18:44:07.281606 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:44:07.281577    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5fc32c21_43ea_4089_b2c0_3d416459607d.slice/crio-82cdee296f8680720308aa5228fa26a6bf7d468ac56d1fa87c7a2cd18441e73c WatchSource:0}: Error finding container 82cdee296f8680720308aa5228fa26a6bf7d468ac56d1fa87c7a2cd18441e73c: Status 404 returned error can't find the container with id 82cdee296f8680720308aa5228fa26a6bf7d468ac56d1fa87c7a2cd18441e73c
Oct 30 18:44:07.283782 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:07.283751    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:44:07.283866 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:07.283815    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:44:07.283866 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:07.283856    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:44:08.200551 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:08.200508    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-6v8vj" event={"ID":"5fc32c21-43ea-4089-b2c0-3d416459607d","Type":"ContainerStarted","Data":"dd7a57dd1923dc8963fb179933cccd20d3c6460def98dcf10c724f2c7f89fbb0"}
Oct 30 18:44:08.201069 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:08.200563    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-6v8vj" event={"ID":"5fc32c21-43ea-4089-b2c0-3d416459607d","Type":"ContainerStarted","Data":"82cdee296f8680720308aa5228fa26a6bf7d468ac56d1fa87c7a2cd18441e73c"}
Oct 30 18:44:08.392355 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:08.391938    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-6v8vj" podStartSLOduration=2.39191906 podStartE2EDuration="2.39191906s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:44:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:44:08.390859547 +0000 UTC m=+612.638265597" watchObservedRunningTime="2026-10-30 18:44:08.39191906 +0000 UTC m=+612.639325087"
Oct 30 18:44:08.846046 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:08.846013    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-6v8vj"
Oct 30 18:44:08.847663 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:08.847638    4888 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-66db998887-6v8vj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body=
Oct 30 18:44:08.847769 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:08.847703    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-6v8vj" podUID="5fc32c21-43ea-4089-b2c0-3d416459607d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused"
Oct 30 18:44:09.846156 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:09.846118    4888 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-66db998887-6v8vj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body=
Oct 30 18:44:09.846754 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:09.846186    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-6v8vj" podUID="5fc32c21-43ea-4089-b2c0-3d416459607d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused"
Oct 30 18:44:10.846233 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:10.846197    4888 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-66db998887-6v8vj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body=
Oct 30 18:44:10.846866 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:10.846832    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-6v8vj" podUID="5fc32c21-43ea-4089-b2c0-3d416459607d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused"
Oct 30 18:44:11.846742 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:11.846697    4888 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-66db998887-6v8vj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body=
Oct 30 18:44:11.847149 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:11.846762    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-6v8vj" podUID="5fc32c21-43ea-4089-b2c0-3d416459607d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused"
Oct 30 18:44:12.846759 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:12.846719    4888 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-66db998887-6v8vj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body=
Oct 30 18:44:12.847232 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:12.846781    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-6v8vj" podUID="5fc32c21-43ea-4089-b2c0-3d416459607d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused"
Oct 30 18:44:13.846160 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:13.846129    4888 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-66db998887-6v8vj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body=
Oct 30 18:44:13.846374 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:13.846175    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-6v8vj" podUID="5fc32c21-43ea-4089-b2c0-3d416459607d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused"
Oct 30 18:44:15.547164 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:15.547136    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-6v8vj"
Oct 30 18:44:15.547657 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:15.547508    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-6v8vj"
Oct 30 18:44:15.548464 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:15.548432    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-6v8vj"
Oct 30 18:44:34.127683 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:34.127653    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-59144a1b-openshift-default-78d555d5bc-lk4t7"]
Oct 30 18:44:34.132611 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:34.132591    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-59144a1b-openshift-default-78d555d5bc-lk4t7"
Oct 30 18:44:34.186775 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:34.186745    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-59144a1b-openshift-default-dockercfg-78s7d\""
Oct 30 18:44:34.239611 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:34.239576    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-59144a1b-openshift-default-78d555d5bc-lk4t7"]
Oct 30 18:44:34.246043 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:34.246021    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cf71a4b7-6006-44ba-945c-bb8749af44ad-istiod-ca-cert\") pod \"e2e-ait-59144a1b-openshift-default-78d555d5bc-lk4t7\" (UID: \"cf71a4b7-6006-44ba-945c-bb8749af44ad\") " pod="openshift-ingress/e2e-ait-59144a1b-openshift-default-78d555d5bc-lk4t7"
Oct 30 18:44:34.246167 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:34.246051    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cf71a4b7-6006-44ba-945c-bb8749af44ad-istio-podinfo\") pod \"e2e-ait-59144a1b-openshift-default-78d555d5bc-lk4t7\" (UID: \"cf71a4b7-6006-44ba-945c-bb8749af44ad\") " pod="openshift-ingress/e2e-ait-59144a1b-openshift-default-78d555d5bc-lk4t7"
Oct 30 18:44:34.246167 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:34.246075    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf71a4b7-6006-44ba-945c-bb8749af44ad-credential-socket\") pod \"e2e-ait-59144a1b-openshift-default-78d555d5bc-lk4t7\" (UID: \"cf71a4b7-6006-44ba-945c-bb8749af44ad\") " pod="openshift-ingress/e2e-ait-59144a1b-openshift-default-78d555d5bc-lk4t7"
Oct 30 18:44:34.246238 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:34.246212    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cf71a4b7-6006-44ba-945c-bb8749af44ad-istio-token\") pod \"e2e-ait-59144a1b-openshift-default-78d555d5bc-lk4t7\" (UID: \"cf71a4b7-6006-44ba-945c-bb8749af44ad\") " pod="openshift-ingress/e2e-ait-59144a1b-openshift-default-78d555d5bc-lk4t7"
Oct 30 18:44:34.246295 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:34.246263    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cf71a4b7-6006-44ba-945c-bb8749af44ad-workload-certs\") pod \"e2e-ait-59144a1b-openshift-default-78d555d5bc-lk4t7\" (UID: \"cf71a4b7-6006-44ba-945c-bb8749af44ad\") " pod="openshift-ingress/e2e-ait-59144a1b-openshift-default-78d555d5bc-lk4t7"
Oct 30 18:44:34.246388 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:34.246373    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf71a4b7-6006-44ba-945c-bb8749af44ad-workload-socket\") pod \"e2e-ait-59144a1b-openshift-default-78d555d5bc-lk4t7\" (UID: \"cf71a4b7-6006-44ba-945c-bb8749af44ad\") " pod="openshift-ingress/e2e-ait-59144a1b-openshift-default-78d555d5bc-lk4t7"
Oct 30 18:44:34.246423 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:34.246397    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cf71a4b7-6006-44ba-945c-bb8749af44ad-istio-envoy\") pod \"e2e-ait-59144a1b-openshift-default-78d555d5bc-lk4t7\" (UID: \"cf71a4b7-6006-44ba-945c-bb8749af44ad\") " pod="openshift-ingress/e2e-ait-59144a1b-openshift-default-78d555d5bc-lk4t7"
Oct 30 18:44:34.246456 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:34.246446    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cf71a4b7-6006-44ba-945c-bb8749af44ad-istio-data\") pod \"e2e-ait-59144a1b-openshift-default-78d555d5bc-lk4t7\" (UID: \"cf71a4b7-6006-44ba-945c-bb8749af44ad\") " pod="openshift-ingress/e2e-ait-59144a1b-openshift-default-78d555d5bc-lk4t7"
Oct 30 18:44:34.246488 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:34.246477    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nsv9h\" (UniqueName: \"kubernetes.io/projected/cf71a4b7-6006-44ba-945c-bb8749af44ad-kube-api-access-nsv9h\") pod \"e2e-ait-59144a1b-openshift-default-78d555d5bc-lk4t7\" (UID: \"cf71a4b7-6006-44ba-945c-bb8749af44ad\") " pod="openshift-ingress/e2e-ait-59144a1b-openshift-default-78d555d5bc-lk4t7"
Oct 30 18:44:34.347199 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:34.347165    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cf71a4b7-6006-44ba-945c-bb8749af44ad-istiod-ca-cert\") pod \"e2e-ait-59144a1b-openshift-default-78d555d5bc-lk4t7\" (UID: \"cf71a4b7-6006-44ba-945c-bb8749af44ad\") " pod="openshift-ingress/e2e-ait-59144a1b-openshift-default-78d555d5bc-lk4t7"
Oct 30 18:44:34.347199 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:34.347200    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cf71a4b7-6006-44ba-945c-bb8749af44ad-istio-podinfo\") pod \"e2e-ait-59144a1b-openshift-default-78d555d5bc-lk4t7\" (UID: \"cf71a4b7-6006-44ba-945c-bb8749af44ad\") " pod="openshift-ingress/e2e-ait-59144a1b-openshift-default-78d555d5bc-lk4t7"
Oct 30 18:44:34.347495 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:34.347220    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf71a4b7-6006-44ba-945c-bb8749af44ad-credential-socket\") pod \"e2e-ait-59144a1b-openshift-default-78d555d5bc-lk4t7\" (UID: \"cf71a4b7-6006-44ba-945c-bb8749af44ad\") " pod="openshift-ingress/e2e-ait-59144a1b-openshift-default-78d555d5bc-lk4t7"
Oct 30 18:44:34.347495 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:34.347353    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cf71a4b7-6006-44ba-945c-bb8749af44ad-istio-token\") pod \"e2e-ait-59144a1b-openshift-default-78d555d5bc-lk4t7\" (UID: \"cf71a4b7-6006-44ba-945c-bb8749af44ad\") " pod="openshift-ingress/e2e-ait-59144a1b-openshift-default-78d555d5bc-lk4t7"
Oct 30 18:44:34.347495 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:34.347439    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cf71a4b7-6006-44ba-945c-bb8749af44ad-workload-certs\") pod \"e2e-ait-59144a1b-openshift-default-78d555d5bc-lk4t7\" (UID: \"cf71a4b7-6006-44ba-945c-bb8749af44ad\") " pod="openshift-ingress/e2e-ait-59144a1b-openshift-default-78d555d5bc-lk4t7"
Oct 30 18:44:34.347495 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:34.347472    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf71a4b7-6006-44ba-945c-bb8749af44ad-workload-socket\") pod \"e2e-ait-59144a1b-openshift-default-78d555d5bc-lk4t7\" (UID: \"cf71a4b7-6006-44ba-945c-bb8749af44ad\") " pod="openshift-ingress/e2e-ait-59144a1b-openshift-default-78d555d5bc-lk4t7"
Oct 30 18:44:34.347495 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:34.347495    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cf71a4b7-6006-44ba-945c-bb8749af44ad-istio-envoy\") pod \"e2e-ait-59144a1b-openshift-default-78d555d5bc-lk4t7\" (UID: \"cf71a4b7-6006-44ba-945c-bb8749af44ad\") " pod="openshift-ingress/e2e-ait-59144a1b-openshift-default-78d555d5bc-lk4t7"
Oct 30 18:44:34.347745 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:34.347528    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cf71a4b7-6006-44ba-945c-bb8749af44ad-istio-data\") pod \"e2e-ait-59144a1b-openshift-default-78d555d5bc-lk4t7\" (UID: \"cf71a4b7-6006-44ba-945c-bb8749af44ad\") " pod="openshift-ingress/e2e-ait-59144a1b-openshift-default-78d555d5bc-lk4t7"
Oct 30 18:44:34.347745 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:34.347584    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nsv9h\" (UniqueName: \"kubernetes.io/projected/cf71a4b7-6006-44ba-945c-bb8749af44ad-kube-api-access-nsv9h\") pod \"e2e-ait-59144a1b-openshift-default-78d555d5bc-lk4t7\" (UID: \"cf71a4b7-6006-44ba-945c-bb8749af44ad\") " pod="openshift-ingress/e2e-ait-59144a1b-openshift-default-78d555d5bc-lk4t7"
Oct 30 18:44:34.347745 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:34.347674    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf71a4b7-6006-44ba-945c-bb8749af44ad-credential-socket\") pod \"e2e-ait-59144a1b-openshift-default-78d555d5bc-lk4t7\" (UID: \"cf71a4b7-6006-44ba-945c-bb8749af44ad\") " pod="openshift-ingress/e2e-ait-59144a1b-openshift-default-78d555d5bc-lk4t7"
Oct 30 18:44:34.347950 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:34.347918    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf71a4b7-6006-44ba-945c-bb8749af44ad-workload-socket\") pod \"e2e-ait-59144a1b-openshift-default-78d555d5bc-lk4t7\" (UID: \"cf71a4b7-6006-44ba-945c-bb8749af44ad\") " pod="openshift-ingress/e2e-ait-59144a1b-openshift-default-78d555d5bc-lk4t7"
Oct 30 18:44:34.348010 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:34.347953    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cf71a4b7-6006-44ba-945c-bb8749af44ad-istio-data\") pod \"e2e-ait-59144a1b-openshift-default-78d555d5bc-lk4t7\" (UID: \"cf71a4b7-6006-44ba-945c-bb8749af44ad\") " pod="openshift-ingress/e2e-ait-59144a1b-openshift-default-78d555d5bc-lk4t7"
Oct 30 18:44:34.348046 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:34.348013    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cf71a4b7-6006-44ba-945c-bb8749af44ad-workload-certs\") pod \"e2e-ait-59144a1b-openshift-default-78d555d5bc-lk4t7\" (UID: \"cf71a4b7-6006-44ba-945c-bb8749af44ad\") " pod="openshift-ingress/e2e-ait-59144a1b-openshift-default-78d555d5bc-lk4t7"
Oct 30 18:44:34.348079 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:34.348061    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cf71a4b7-6006-44ba-945c-bb8749af44ad-istiod-ca-cert\") pod \"e2e-ait-59144a1b-openshift-default-78d555d5bc-lk4t7\" (UID: \"cf71a4b7-6006-44ba-945c-bb8749af44ad\") " pod="openshift-ingress/e2e-ait-59144a1b-openshift-default-78d555d5bc-lk4t7"
Oct 30 18:44:34.349826 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:34.349809    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cf71a4b7-6006-44ba-945c-bb8749af44ad-istio-envoy\") pod \"e2e-ait-59144a1b-openshift-default-78d555d5bc-lk4t7\" (UID: \"cf71a4b7-6006-44ba-945c-bb8749af44ad\") " pod="openshift-ingress/e2e-ait-59144a1b-openshift-default-78d555d5bc-lk4t7"
Oct 30 18:44:34.350037 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:34.350019    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cf71a4b7-6006-44ba-945c-bb8749af44ad-istio-podinfo\") pod \"e2e-ait-59144a1b-openshift-default-78d555d5bc-lk4t7\" (UID: \"cf71a4b7-6006-44ba-945c-bb8749af44ad\") " pod="openshift-ingress/e2e-ait-59144a1b-openshift-default-78d555d5bc-lk4t7"
Oct 30 18:44:34.409450 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:34.409393    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cf71a4b7-6006-44ba-945c-bb8749af44ad-istio-token\") pod \"e2e-ait-59144a1b-openshift-default-78d555d5bc-lk4t7\" (UID: \"cf71a4b7-6006-44ba-945c-bb8749af44ad\") " pod="openshift-ingress/e2e-ait-59144a1b-openshift-default-78d555d5bc-lk4t7"
Oct 30 18:44:34.414071 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:34.414052    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nsv9h\" (UniqueName: \"kubernetes.io/projected/cf71a4b7-6006-44ba-945c-bb8749af44ad-kube-api-access-nsv9h\") pod \"e2e-ait-59144a1b-openshift-default-78d555d5bc-lk4t7\" (UID: \"cf71a4b7-6006-44ba-945c-bb8749af44ad\") " pod="openshift-ingress/e2e-ait-59144a1b-openshift-default-78d555d5bc-lk4t7"
Oct 30 18:44:34.452967 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:34.452944    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-59144a1b-openshift-default-78d555d5bc-lk4t7"
Oct 30 18:44:34.891438 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:44:34.891407    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf71a4b7_6006_44ba_945c_bb8749af44ad.slice/crio-f20bace6c42b409cc2a263e3dae90c01fb28e0af5ddc8b7a33fea16205f9c2a1 WatchSource:0}: Error finding container f20bace6c42b409cc2a263e3dae90c01fb28e0af5ddc8b7a33fea16205f9c2a1: Status 404 returned error can't find the container with id f20bace6c42b409cc2a263e3dae90c01fb28e0af5ddc8b7a33fea16205f9c2a1
Oct 30 18:44:34.891800 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:34.891780    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-59144a1b-openshift-default-78d555d5bc-lk4t7"]
Oct 30 18:44:34.893449 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:34.893421    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:44:34.893571 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:34.893477    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:44:34.893571 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:34.893511    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:44:35.363910 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:35.363881    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-59144a1b-openshift-default-78d555d5bc-lk4t7" event={"ID":"cf71a4b7-6006-44ba-945c-bb8749af44ad","Type":"ContainerStarted","Data":"ee735a6244ff50f6b0d5fc8a91a7a09353311d638bff44afe7c9f112d9276a24"}
Oct 30 18:44:35.364314 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:35.363928    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-59144a1b-openshift-default-78d555d5bc-lk4t7" event={"ID":"cf71a4b7-6006-44ba-945c-bb8749af44ad","Type":"ContainerStarted","Data":"f20bace6c42b409cc2a263e3dae90c01fb28e0af5ddc8b7a33fea16205f9c2a1"}
Oct 30 18:44:35.453629 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:35.453596    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-59144a1b-openshift-default-78d555d5bc-lk4t7"
Oct 30 18:44:35.458856 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:35.458832    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-59144a1b-openshift-default-78d555d5bc-lk4t7"
Oct 30 18:44:35.618484 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:44:35.618392    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-jbsw6" containerName="extractor"
Oct 30 18:44:35.637058 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:35.637007    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-59144a1b-openshift-default-78d555d5bc-lk4t7" podStartSLOduration=1.636989985 podStartE2EDuration="1.636989985s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:44:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:44:35.510624531 +0000 UTC m=+639.758030565" watchObservedRunningTime="2026-10-30 18:44:35.636989985 +0000 UTC m=+639.884396012"
Oct 30 18:44:36.369267 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:36.369235    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-59144a1b-openshift-default-78d555d5bc-lk4t7"
Oct 30 18:44:36.370412 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:36.370391    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-59144a1b-openshift-default-78d555d5bc-lk4t7"
Oct 30 18:44:36.426646 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:36.426621    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-def78c01-openshift-default-59959c464f-6qpnd"]
Oct 30 18:44:36.430776 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:36.430757    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-def78c01-openshift-default-59959c464f-6qpnd"
Oct 30 18:44:36.486587 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:36.486558    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-def78c01-openshift-default-dockercfg-lmdxx\""
Oct 30 18:44:36.496490 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:36.496458    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-def78c01-openshift-default-59959c464f-6qpnd"]
Oct 30 18:44:36.571978 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:36.571433    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hvkjm\" (UniqueName: \"kubernetes.io/projected/87986146-ca55-47e6-8f17-9df4ed880eeb-kube-api-access-hvkjm\") pod \"e2e-ait-def78c01-openshift-default-59959c464f-6qpnd\" (UID: \"87986146-ca55-47e6-8f17-9df4ed880eeb\") " pod="openshift-ingress/e2e-ait-def78c01-openshift-default-59959c464f-6qpnd"
Oct 30 18:44:36.571978 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:36.571525    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/87986146-ca55-47e6-8f17-9df4ed880eeb-istio-envoy\") pod \"e2e-ait-def78c01-openshift-default-59959c464f-6qpnd\" (UID: \"87986146-ca55-47e6-8f17-9df4ed880eeb\") " pod="openshift-ingress/e2e-ait-def78c01-openshift-default-59959c464f-6qpnd"
Oct 30 18:44:36.571978 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:36.571556    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/87986146-ca55-47e6-8f17-9df4ed880eeb-istio-token\") pod \"e2e-ait-def78c01-openshift-default-59959c464f-6qpnd\" (UID: \"87986146-ca55-47e6-8f17-9df4ed880eeb\") " pod="openshift-ingress/e2e-ait-def78c01-openshift-default-59959c464f-6qpnd"
Oct 30 18:44:36.571978 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:36.571583    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/87986146-ca55-47e6-8f17-9df4ed880eeb-istio-podinfo\") pod \"e2e-ait-def78c01-openshift-default-59959c464f-6qpnd\" (UID: \"87986146-ca55-47e6-8f17-9df4ed880eeb\") " pod="openshift-ingress/e2e-ait-def78c01-openshift-default-59959c464f-6qpnd"
Oct 30 18:44:36.571978 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:36.571677    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/87986146-ca55-47e6-8f17-9df4ed880eeb-istio-data\") pod \"e2e-ait-def78c01-openshift-default-59959c464f-6qpnd\" (UID: \"87986146-ca55-47e6-8f17-9df4ed880eeb\") " pod="openshift-ingress/e2e-ait-def78c01-openshift-default-59959c464f-6qpnd"
Oct 30 18:44:36.571978 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:36.571705    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/87986146-ca55-47e6-8f17-9df4ed880eeb-istiod-ca-cert\") pod \"e2e-ait-def78c01-openshift-default-59959c464f-6qpnd\" (UID: \"87986146-ca55-47e6-8f17-9df4ed880eeb\") " pod="openshift-ingress/e2e-ait-def78c01-openshift-default-59959c464f-6qpnd"
Oct 30 18:44:36.571978 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:36.571736    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/87986146-ca55-47e6-8f17-9df4ed880eeb-workload-certs\") pod \"e2e-ait-def78c01-openshift-default-59959c464f-6qpnd\" (UID: \"87986146-ca55-47e6-8f17-9df4ed880eeb\") " pod="openshift-ingress/e2e-ait-def78c01-openshift-default-59959c464f-6qpnd"
Oct 30 18:44:36.571978 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:36.571766    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/87986146-ca55-47e6-8f17-9df4ed880eeb-workload-socket\") pod \"e2e-ait-def78c01-openshift-default-59959c464f-6qpnd\" (UID: \"87986146-ca55-47e6-8f17-9df4ed880eeb\") " pod="openshift-ingress/e2e-ait-def78c01-openshift-default-59959c464f-6qpnd"
Oct 30 18:44:36.571978 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:36.571806    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/87986146-ca55-47e6-8f17-9df4ed880eeb-credential-socket\") pod \"e2e-ait-def78c01-openshift-default-59959c464f-6qpnd\" (UID: \"87986146-ca55-47e6-8f17-9df4ed880eeb\") " pod="openshift-ingress/e2e-ait-def78c01-openshift-default-59959c464f-6qpnd"
Oct 30 18:44:36.672772 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:36.672682    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/87986146-ca55-47e6-8f17-9df4ed880eeb-istio-envoy\") pod \"e2e-ait-def78c01-openshift-default-59959c464f-6qpnd\" (UID: \"87986146-ca55-47e6-8f17-9df4ed880eeb\") " pod="openshift-ingress/e2e-ait-def78c01-openshift-default-59959c464f-6qpnd"
Oct 30 18:44:36.672772 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:36.672718    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/87986146-ca55-47e6-8f17-9df4ed880eeb-istio-token\") pod \"e2e-ait-def78c01-openshift-default-59959c464f-6qpnd\" (UID: \"87986146-ca55-47e6-8f17-9df4ed880eeb\") " pod="openshift-ingress/e2e-ait-def78c01-openshift-default-59959c464f-6qpnd"
Oct 30 18:44:36.672772 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:36.672742    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/87986146-ca55-47e6-8f17-9df4ed880eeb-istio-podinfo\") pod \"e2e-ait-def78c01-openshift-default-59959c464f-6qpnd\" (UID: \"87986146-ca55-47e6-8f17-9df4ed880eeb\") " pod="openshift-ingress/e2e-ait-def78c01-openshift-default-59959c464f-6qpnd"
Oct 30 18:44:36.673058 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:36.672780    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/87986146-ca55-47e6-8f17-9df4ed880eeb-istio-data\") pod \"e2e-ait-def78c01-openshift-default-59959c464f-6qpnd\" (UID: \"87986146-ca55-47e6-8f17-9df4ed880eeb\") " pod="openshift-ingress/e2e-ait-def78c01-openshift-default-59959c464f-6qpnd"
Oct 30 18:44:36.673058 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:36.672811    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/87986146-ca55-47e6-8f17-9df4ed880eeb-istiod-ca-cert\") pod \"e2e-ait-def78c01-openshift-default-59959c464f-6qpnd\" (UID: \"87986146-ca55-47e6-8f17-9df4ed880eeb\") " pod="openshift-ingress/e2e-ait-def78c01-openshift-default-59959c464f-6qpnd"
Oct 30 18:44:36.673058 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:36.672846    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/87986146-ca55-47e6-8f17-9df4ed880eeb-workload-certs\") pod \"e2e-ait-def78c01-openshift-default-59959c464f-6qpnd\" (UID: \"87986146-ca55-47e6-8f17-9df4ed880eeb\") " pod="openshift-ingress/e2e-ait-def78c01-openshift-default-59959c464f-6qpnd"
Oct 30 18:44:36.673058 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:36.672891    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/87986146-ca55-47e6-8f17-9df4ed880eeb-workload-socket\") pod \"e2e-ait-def78c01-openshift-default-59959c464f-6qpnd\" (UID: \"87986146-ca55-47e6-8f17-9df4ed880eeb\") " pod="openshift-ingress/e2e-ait-def78c01-openshift-default-59959c464f-6qpnd"
Oct 30 18:44:36.673058 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:36.672931    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/87986146-ca55-47e6-8f17-9df4ed880eeb-credential-socket\") pod \"e2e-ait-def78c01-openshift-default-59959c464f-6qpnd\" (UID: \"87986146-ca55-47e6-8f17-9df4ed880eeb\") " pod="openshift-ingress/e2e-ait-def78c01-openshift-default-59959c464f-6qpnd"
Oct 30 18:44:36.673058 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:36.672982    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hvkjm\" (UniqueName: \"kubernetes.io/projected/87986146-ca55-47e6-8f17-9df4ed880eeb-kube-api-access-hvkjm\") pod \"e2e-ait-def78c01-openshift-default-59959c464f-6qpnd\" (UID: \"87986146-ca55-47e6-8f17-9df4ed880eeb\") " pod="openshift-ingress/e2e-ait-def78c01-openshift-default-59959c464f-6qpnd"
Oct 30 18:44:36.673521 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:36.673494    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/87986146-ca55-47e6-8f17-9df4ed880eeb-istio-data\") pod \"e2e-ait-def78c01-openshift-default-59959c464f-6qpnd\" (UID: \"87986146-ca55-47e6-8f17-9df4ed880eeb\") " pod="openshift-ingress/e2e-ait-def78c01-openshift-default-59959c464f-6qpnd"
Oct 30 18:44:36.673882 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:36.673828    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/87986146-ca55-47e6-8f17-9df4ed880eeb-workload-socket\") pod \"e2e-ait-def78c01-openshift-default-59959c464f-6qpnd\" (UID: \"87986146-ca55-47e6-8f17-9df4ed880eeb\") " pod="openshift-ingress/e2e-ait-def78c01-openshift-default-59959c464f-6qpnd"
Oct 30 18:44:36.673992 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:36.673938    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/87986146-ca55-47e6-8f17-9df4ed880eeb-credential-socket\") pod \"e2e-ait-def78c01-openshift-default-59959c464f-6qpnd\" (UID: \"87986146-ca55-47e6-8f17-9df4ed880eeb\") " pod="openshift-ingress/e2e-ait-def78c01-openshift-default-59959c464f-6qpnd"
Oct 30 18:44:36.674605 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:36.674131    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/87986146-ca55-47e6-8f17-9df4ed880eeb-workload-certs\") pod \"e2e-ait-def78c01-openshift-default-59959c464f-6qpnd\" (UID: \"87986146-ca55-47e6-8f17-9df4ed880eeb\") " pod="openshift-ingress/e2e-ait-def78c01-openshift-default-59959c464f-6qpnd"
Oct 30 18:44:36.674605 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:36.674490    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/87986146-ca55-47e6-8f17-9df4ed880eeb-istiod-ca-cert\") pod \"e2e-ait-def78c01-openshift-default-59959c464f-6qpnd\" (UID: \"87986146-ca55-47e6-8f17-9df4ed880eeb\") " pod="openshift-ingress/e2e-ait-def78c01-openshift-default-59959c464f-6qpnd"
Oct 30 18:44:36.682003 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:36.681963    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/87986146-ca55-47e6-8f17-9df4ed880eeb-istio-envoy\") pod \"e2e-ait-def78c01-openshift-default-59959c464f-6qpnd\" (UID: \"87986146-ca55-47e6-8f17-9df4ed880eeb\") " pod="openshift-ingress/e2e-ait-def78c01-openshift-default-59959c464f-6qpnd"
Oct 30 18:44:36.691619 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:36.690948    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/87986146-ca55-47e6-8f17-9df4ed880eeb-istio-podinfo\") pod \"e2e-ait-def78c01-openshift-default-59959c464f-6qpnd\" (UID: \"87986146-ca55-47e6-8f17-9df4ed880eeb\") " pod="openshift-ingress/e2e-ait-def78c01-openshift-default-59959c464f-6qpnd"
Oct 30 18:44:36.735218 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:36.735189    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hvkjm\" (UniqueName: \"kubernetes.io/projected/87986146-ca55-47e6-8f17-9df4ed880eeb-kube-api-access-hvkjm\") pod \"e2e-ait-def78c01-openshift-default-59959c464f-6qpnd\" (UID: \"87986146-ca55-47e6-8f17-9df4ed880eeb\") " pod="openshift-ingress/e2e-ait-def78c01-openshift-default-59959c464f-6qpnd"
Oct 30 18:44:36.735218 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:36.735204    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/87986146-ca55-47e6-8f17-9df4ed880eeb-istio-token\") pod \"e2e-ait-def78c01-openshift-default-59959c464f-6qpnd\" (UID: \"87986146-ca55-47e6-8f17-9df4ed880eeb\") " pod="openshift-ingress/e2e-ait-def78c01-openshift-default-59959c464f-6qpnd"
Oct 30 18:44:36.754096 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:36.754076    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-def78c01-openshift-default-59959c464f-6qpnd"
Oct 30 18:44:37.188726 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:37.188702    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-def78c01-openshift-default-59959c464f-6qpnd"]
Oct 30 18:44:37.189481 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:44:37.189459    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87986146_ca55_47e6_8f17_9df4ed880eeb.slice/crio-257bcad6cc559f8c9c88a3bacdf3d6784a4e79a311e376849ed471078e3525de WatchSource:0}: Error finding container 257bcad6cc559f8c9c88a3bacdf3d6784a4e79a311e376849ed471078e3525de: Status 404 returned error can't find the container with id 257bcad6cc559f8c9c88a3bacdf3d6784a4e79a311e376849ed471078e3525de
Oct 30 18:44:37.191655 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:37.191623    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:44:37.191755 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:37.191684    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:44:37.191755 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:37.191715    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:44:37.376925 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:37.376894    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-def78c01-openshift-default-59959c464f-6qpnd" event={"ID":"87986146-ca55-47e6-8f17-9df4ed880eeb","Type":"ContainerStarted","Data":"30a2f64a02963e4beb48c6fdb8f56c71d76426dd791eb38175e528b4f2c91c2e"}
Oct 30 18:44:37.377352 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:37.376955    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-def78c01-openshift-default-59959c464f-6qpnd" event={"ID":"87986146-ca55-47e6-8f17-9df4ed880eeb","Type":"ContainerStarted","Data":"257bcad6cc559f8c9c88a3bacdf3d6784a4e79a311e376849ed471078e3525de"}
Oct 30 18:44:37.507372 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:37.507262    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-def78c01-openshift-default-59959c464f-6qpnd" podStartSLOduration=1.50724643 podStartE2EDuration="1.50724643s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:44:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:44:37.505364048 +0000 UTC m=+641.752770073" watchObservedRunningTime="2026-10-30 18:44:37.50724643 +0000 UTC m=+641.754652527"
Oct 30 18:44:37.754850 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:37.754819    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-def78c01-openshift-default-59959c464f-6qpnd"
Oct 30 18:44:38.759844 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:38.759815    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-def78c01-openshift-default-59959c464f-6qpnd"
Oct 30 18:44:39.391655 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:39.391623    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-def78c01-openshift-default-59959c464f-6qpnd"
Oct 30 18:44:39.392700 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:39.392678    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-def78c01-openshift-default-59959c464f-6qpnd"
Oct 30 18:44:41.278693 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:41.278663    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-529e04e2-d954bc5c5-m5jvj"]
Oct 30 18:44:41.282740 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:41.282720    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-529e04e2-d954bc5c5-m5jvj"
Oct 30 18:44:41.341111 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:41.341079    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-529e04e2-d954bc5c5-m5jvj"]
Oct 30 18:44:41.389716 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:41.389675    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-529e04e2\""
Oct 30 18:44:41.389902 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:41.389731    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-529e04e2-dockercfg-wstvs\""
Oct 30 18:44:41.422822 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:41.422785    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-64rdg\" (UniqueName: \"kubernetes.io/projected/37620fd5-3a09-4ccb-8708-c51b3cae9520-kube-api-access-64rdg\") pod \"payload-pre-processing-e2e-ait-529e04e2-d954bc5c5-m5jvj\" (UID: \"37620fd5-3a09-4ccb-8708-c51b3cae9520\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-529e04e2-d954bc5c5-m5jvj"
Oct 30 18:44:41.423014 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:41.422956    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/37620fd5-3a09-4ccb-8708-c51b3cae9520-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-529e04e2-d954bc5c5-m5jvj\" (UID: \"37620fd5-3a09-4ccb-8708-c51b3cae9520\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-529e04e2-d954bc5c5-m5jvj"
Oct 30 18:44:41.524353 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:41.524316    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/37620fd5-3a09-4ccb-8708-c51b3cae9520-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-529e04e2-d954bc5c5-m5jvj\" (UID: \"37620fd5-3a09-4ccb-8708-c51b3cae9520\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-529e04e2-d954bc5c5-m5jvj"
Oct 30 18:44:41.524560 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:41.524367    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-64rdg\" (UniqueName: \"kubernetes.io/projected/37620fd5-3a09-4ccb-8708-c51b3cae9520-kube-api-access-64rdg\") pod \"payload-pre-processing-e2e-ait-529e04e2-d954bc5c5-m5jvj\" (UID: \"37620fd5-3a09-4ccb-8708-c51b3cae9520\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-529e04e2-d954bc5c5-m5jvj"
Oct 30 18:44:41.525100 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:41.525081    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/37620fd5-3a09-4ccb-8708-c51b3cae9520-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-529e04e2-d954bc5c5-m5jvj\" (UID: \"37620fd5-3a09-4ccb-8708-c51b3cae9520\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-529e04e2-d954bc5c5-m5jvj"
Oct 30 18:44:41.587132 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:41.587093    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-64rdg\" (UniqueName: \"kubernetes.io/projected/37620fd5-3a09-4ccb-8708-c51b3cae9520-kube-api-access-64rdg\") pod \"payload-pre-processing-e2e-ait-529e04e2-d954bc5c5-m5jvj\" (UID: \"37620fd5-3a09-4ccb-8708-c51b3cae9520\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-529e04e2-d954bc5c5-m5jvj"
Oct 30 18:44:41.595817 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:41.595794    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-529e04e2-d954bc5c5-m5jvj"
Oct 30 18:44:41.975840 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:41.975808    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-529e04e2-d954bc5c5-m5jvj"]
Oct 30 18:44:41.977218 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:44:41.977194    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37620fd5_3a09_4ccb_8708_c51b3cae9520.slice/crio-54f11e85b206afa1534955dd392ec0e10eae1ecb5e2a72325095f81725f37577 WatchSource:0}: Error finding container 54f11e85b206afa1534955dd392ec0e10eae1ecb5e2a72325095f81725f37577: Status 404 returned error can't find the container with id 54f11e85b206afa1534955dd392ec0e10eae1ecb5e2a72325095f81725f37577
Oct 30 18:44:41.979695 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:41.979678    4888 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 18:44:42.415221 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:42.415182    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-529e04e2-d954bc5c5-m5jvj" event={"ID":"37620fd5-3a09-4ccb-8708-c51b3cae9520","Type":"ContainerStarted","Data":"54f11e85b206afa1534955dd392ec0e10eae1ecb5e2a72325095f81725f37577"}
Oct 30 18:44:45.436365 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:45.436335    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-529e04e2-d954bc5c5-m5jvj" event={"ID":"37620fd5-3a09-4ccb-8708-c51b3cae9520","Type":"ContainerStarted","Data":"c1f29e07d553f3519dc34f5299907005a90c635ef4b0640b8bb62d4bce2b9e59"}
Oct 30 18:44:45.436747 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:45.436444    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-529e04e2-d954bc5c5-m5jvj"
Oct 30 18:44:45.560084 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:45.559993    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-529e04e2-d954bc5c5-m5jvj" podStartSLOduration=1.351644648 podStartE2EDuration="4.559982007s" totalImagesPullingTime="3.208337359s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:44:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:44:45.558965886 +0000 UTC m=+649.806371920" watchObservedRunningTime="2026-10-30 18:44:45.559982007 +0000 UTC m=+649.807388065"
Oct 30 18:44:46.283486 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:46.283455    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-59144a1b-7d9455dd67-dmj9h"]
Oct 30 18:44:46.287327 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:46.287309    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-59144a1b-7d9455dd67-dmj9h"
Oct 30 18:44:46.342011 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:46.341983    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-59144a1b\""
Oct 30 18:44:46.344972 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:46.344948    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-59144a1b-7d9455dd67-dmj9h"]
Oct 30 18:44:46.386960 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:46.372048    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8g5nx\" (UniqueName: \"kubernetes.io/projected/230b68f1-1eeb-4db0-a160-9248f8314240-kube-api-access-8g5nx\") pod \"payload-pre-processing-e2e-ait-59144a1b-7d9455dd67-dmj9h\" (UID: \"230b68f1-1eeb-4db0-a160-9248f8314240\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-59144a1b-7d9455dd67-dmj9h"
Oct 30 18:44:46.386960 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:46.372116    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/230b68f1-1eeb-4db0-a160-9248f8314240-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-59144a1b-7d9455dd67-dmj9h\" (UID: \"230b68f1-1eeb-4db0-a160-9248f8314240\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-59144a1b-7d9455dd67-dmj9h"
Oct 30 18:44:46.392781 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:46.392741    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-59144a1b-dockercfg-kdxgt\""
Oct 30 18:44:46.440607 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:46.440580    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-59144a1b-569dd9dbd8-45qd4"]
Oct 30 18:44:46.444247 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:46.444227    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-59144a1b-569dd9dbd8-45qd4"
Oct 30 18:44:46.473795 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:46.473769    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8g5nx\" (UniqueName: \"kubernetes.io/projected/230b68f1-1eeb-4db0-a160-9248f8314240-kube-api-access-8g5nx\") pod \"payload-pre-processing-e2e-ait-59144a1b-7d9455dd67-dmj9h\" (UID: \"230b68f1-1eeb-4db0-a160-9248f8314240\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-59144a1b-7d9455dd67-dmj9h"
Oct 30 18:44:46.473919 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:46.473814    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/230b68f1-1eeb-4db0-a160-9248f8314240-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-59144a1b-7d9455dd67-dmj9h\" (UID: \"230b68f1-1eeb-4db0-a160-9248f8314240\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-59144a1b-7d9455dd67-dmj9h"
Oct 30 18:44:46.474663 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:46.474636    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/230b68f1-1eeb-4db0-a160-9248f8314240-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-59144a1b-7d9455dd67-dmj9h\" (UID: \"230b68f1-1eeb-4db0-a160-9248f8314240\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-59144a1b-7d9455dd67-dmj9h"
Oct 30 18:44:46.520386 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:46.520363    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-59144a1b-569dd9dbd8-45qd4"]
Oct 30 18:44:46.546169 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:46.546094    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8g5nx\" (UniqueName: \"kubernetes.io/projected/230b68f1-1eeb-4db0-a160-9248f8314240-kube-api-access-8g5nx\") pod \"payload-pre-processing-e2e-ait-59144a1b-7d9455dd67-dmj9h\" (UID: \"230b68f1-1eeb-4db0-a160-9248f8314240\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-59144a1b-7d9455dd67-dmj9h"
Oct 30 18:44:46.574758 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:46.574727    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e1cf8b12-1a2a-4935-8c3a-97bfe3281cb6-plugins-config-volume\") pod \"payload-processing-e2e-ait-59144a1b-569dd9dbd8-45qd4\" (UID: \"e1cf8b12-1a2a-4935-8c3a-97bfe3281cb6\") " pod="openshift-ingress/payload-processing-e2e-ait-59144a1b-569dd9dbd8-45qd4"
Oct 30 18:44:46.574914 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:46.574777    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qxwjw\" (UniqueName: \"kubernetes.io/projected/e1cf8b12-1a2a-4935-8c3a-97bfe3281cb6-kube-api-access-qxwjw\") pod \"payload-processing-e2e-ait-59144a1b-569dd9dbd8-45qd4\" (UID: \"e1cf8b12-1a2a-4935-8c3a-97bfe3281cb6\") " pod="openshift-ingress/payload-processing-e2e-ait-59144a1b-569dd9dbd8-45qd4"
Oct 30 18:44:46.600620 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:46.600597    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-59144a1b-7d9455dd67-dmj9h"
Oct 30 18:44:46.676136 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:46.676104    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e1cf8b12-1a2a-4935-8c3a-97bfe3281cb6-plugins-config-volume\") pod \"payload-processing-e2e-ait-59144a1b-569dd9dbd8-45qd4\" (UID: \"e1cf8b12-1a2a-4935-8c3a-97bfe3281cb6\") " pod="openshift-ingress/payload-processing-e2e-ait-59144a1b-569dd9dbd8-45qd4"
Oct 30 18:44:46.676261 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:46.676175    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qxwjw\" (UniqueName: \"kubernetes.io/projected/e1cf8b12-1a2a-4935-8c3a-97bfe3281cb6-kube-api-access-qxwjw\") pod \"payload-processing-e2e-ait-59144a1b-569dd9dbd8-45qd4\" (UID: \"e1cf8b12-1a2a-4935-8c3a-97bfe3281cb6\") " pod="openshift-ingress/payload-processing-e2e-ait-59144a1b-569dd9dbd8-45qd4"
Oct 30 18:44:46.676944 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:46.676926    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e1cf8b12-1a2a-4935-8c3a-97bfe3281cb6-plugins-config-volume\") pod \"payload-processing-e2e-ait-59144a1b-569dd9dbd8-45qd4\" (UID: \"e1cf8b12-1a2a-4935-8c3a-97bfe3281cb6\") " pod="openshift-ingress/payload-processing-e2e-ait-59144a1b-569dd9dbd8-45qd4"
Oct 30 18:44:46.739162 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:46.739138    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qxwjw\" (UniqueName: \"kubernetes.io/projected/e1cf8b12-1a2a-4935-8c3a-97bfe3281cb6-kube-api-access-qxwjw\") pod \"payload-processing-e2e-ait-59144a1b-569dd9dbd8-45qd4\" (UID: \"e1cf8b12-1a2a-4935-8c3a-97bfe3281cb6\") " pod="openshift-ingress/payload-processing-e2e-ait-59144a1b-569dd9dbd8-45qd4"
Oct 30 18:44:46.758101 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:46.758081    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-59144a1b-569dd9dbd8-45qd4"
Oct 30 18:44:46.978536 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:44:46.978504    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod230b68f1_1eeb_4db0_a160_9248f8314240.slice/crio-aa78764a6d7f32b0644dd828ca4434b27897d166ebbac085f45cbcbdc8837f20 WatchSource:0}: Error finding container aa78764a6d7f32b0644dd828ca4434b27897d166ebbac085f45cbcbdc8837f20: Status 404 returned error can't find the container with id aa78764a6d7f32b0644dd828ca4434b27897d166ebbac085f45cbcbdc8837f20
Oct 30 18:44:46.978756 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:46.978735    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-59144a1b-7d9455dd67-dmj9h"]
Oct 30 18:44:47.141315 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:47.140205    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-59144a1b-569dd9dbd8-45qd4"]
Oct 30 18:44:47.141315 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:44:47.140671    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1cf8b12_1a2a_4935_8c3a_97bfe3281cb6.slice/crio-3efb010ad23340a9ee7ac6ac0ccdfe5671171547d32337593e38f25c42f59a6c WatchSource:0}: Error finding container 3efb010ad23340a9ee7ac6ac0ccdfe5671171547d32337593e38f25c42f59a6c: Status 404 returned error can't find the container with id 3efb010ad23340a9ee7ac6ac0ccdfe5671171547d32337593e38f25c42f59a6c
Oct 30 18:44:47.449573 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:47.449482    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-59144a1b-7d9455dd67-dmj9h" event={"ID":"230b68f1-1eeb-4db0-a160-9248f8314240","Type":"ContainerStarted","Data":"11d8ab27b4ddc1570bc79f437c4c9b54b453f5a7599f917bb9df3731fc8f4006"}
Oct 30 18:44:47.449573 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:47.449545    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-59144a1b-7d9455dd67-dmj9h" event={"ID":"230b68f1-1eeb-4db0-a160-9248f8314240","Type":"ContainerStarted","Data":"aa78764a6d7f32b0644dd828ca4434b27897d166ebbac085f45cbcbdc8837f20"}
Oct 30 18:44:47.450082 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:47.449691    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-59144a1b-7d9455dd67-dmj9h"
Oct 30 18:44:47.451260 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:47.451236    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-59144a1b-569dd9dbd8-45qd4" event={"ID":"e1cf8b12-1a2a-4935-8c3a-97bfe3281cb6","Type":"ContainerStarted","Data":"09e4971cdd8b96300a148b7b0276e7be874f6966cd3826671c7cb9424a60120a"}
Oct 30 18:44:47.451424 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:47.451271    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-59144a1b-569dd9dbd8-45qd4" event={"ID":"e1cf8b12-1a2a-4935-8c3a-97bfe3281cb6","Type":"ContainerStarted","Data":"3efb010ad23340a9ee7ac6ac0ccdfe5671171547d32337593e38f25c42f59a6c"}
Oct 30 18:44:47.451424 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:47.451400    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-59144a1b-569dd9dbd8-45qd4"
Oct 30 18:44:47.577782 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:47.577736    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-59144a1b-7d9455dd67-dmj9h" podStartSLOduration=1.5777231600000001 podStartE2EDuration="1.57772316s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:44:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:44:47.576305521 +0000 UTC m=+651.823711542" watchObservedRunningTime="2026-10-30 18:44:47.57772316 +0000 UTC m=+651.825129186"
Oct 30 18:44:47.695889 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:47.695722    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-59144a1b-569dd9dbd8-45qd4" podStartSLOduration=1.695701708 podStartE2EDuration="1.695701708s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:44:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:44:47.694363363 +0000 UTC m=+651.941769435" watchObservedRunningTime="2026-10-30 18:44:47.695701708 +0000 UTC m=+651.943107735"
Oct 30 18:44:48.818099 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:48.818067    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-def78c01-855988f75-dln2g"]
Oct 30 18:44:48.822032 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:48.822017    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-def78c01-855988f75-dln2g"
Oct 30 18:44:48.878109 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:48.878083    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-def78c01\""
Oct 30 18:44:48.931477 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:48.931452    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-def78c01-dockercfg-7c6gv\""
Oct 30 18:44:48.931860 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:48.931842    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-def78c01-855988f75-dln2g"]
Oct 30 18:44:48.972460 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:48.972430    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-def78c01-6994fdb9d-kxfn6"]
Oct 30 18:44:48.976115 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:48.976100    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-def78c01-6994fdb9d-kxfn6"
Oct 30 18:44:48.999715 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:48.999689    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9c207780-577e-483c-b70c-ff0e44e2dc0f-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-def78c01-855988f75-dln2g\" (UID: \"9c207780-577e-483c-b70c-ff0e44e2dc0f\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-def78c01-855988f75-dln2g"
Oct 30 18:44:48.999881 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:48.999746    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5jpfb\" (UniqueName: \"kubernetes.io/projected/9c207780-577e-483c-b70c-ff0e44e2dc0f-kube-api-access-5jpfb\") pod \"payload-pre-processing-e2e-ait-def78c01-855988f75-dln2g\" (UID: \"9c207780-577e-483c-b70c-ff0e44e2dc0f\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-def78c01-855988f75-dln2g"
Oct 30 18:44:49.045329 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:49.044922    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-def78c01-6994fdb9d-kxfn6"]
Oct 30 18:44:49.101183 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:49.101149    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/82903e3e-982c-4708-a10c-8328e324f7b6-plugins-config-volume\") pod \"payload-processing-e2e-ait-def78c01-6994fdb9d-kxfn6\" (UID: \"82903e3e-982c-4708-a10c-8328e324f7b6\") " pod="openshift-ingress/payload-processing-e2e-ait-def78c01-6994fdb9d-kxfn6"
Oct 30 18:44:49.101347 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:49.101220    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9c207780-577e-483c-b70c-ff0e44e2dc0f-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-def78c01-855988f75-dln2g\" (UID: \"9c207780-577e-483c-b70c-ff0e44e2dc0f\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-def78c01-855988f75-dln2g"
Oct 30 18:44:49.101347 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:49.101270    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5jpfb\" (UniqueName: \"kubernetes.io/projected/9c207780-577e-483c-b70c-ff0e44e2dc0f-kube-api-access-5jpfb\") pod \"payload-pre-processing-e2e-ait-def78c01-855988f75-dln2g\" (UID: \"9c207780-577e-483c-b70c-ff0e44e2dc0f\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-def78c01-855988f75-dln2g"
Oct 30 18:44:49.101438 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:49.101351    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hh6zt\" (UniqueName: \"kubernetes.io/projected/82903e3e-982c-4708-a10c-8328e324f7b6-kube-api-access-hh6zt\") pod \"payload-processing-e2e-ait-def78c01-6994fdb9d-kxfn6\" (UID: \"82903e3e-982c-4708-a10c-8328e324f7b6\") " pod="openshift-ingress/payload-processing-e2e-ait-def78c01-6994fdb9d-kxfn6"
Oct 30 18:44:49.101971 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:49.101954    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9c207780-577e-483c-b70c-ff0e44e2dc0f-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-def78c01-855988f75-dln2g\" (UID: \"9c207780-577e-483c-b70c-ff0e44e2dc0f\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-def78c01-855988f75-dln2g"
Oct 30 18:44:49.164116 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:49.164088    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5jpfb\" (UniqueName: \"kubernetes.io/projected/9c207780-577e-483c-b70c-ff0e44e2dc0f-kube-api-access-5jpfb\") pod \"payload-pre-processing-e2e-ait-def78c01-855988f75-dln2g\" (UID: \"9c207780-577e-483c-b70c-ff0e44e2dc0f\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-def78c01-855988f75-dln2g"
Oct 30 18:44:49.202301 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:49.202256    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hh6zt\" (UniqueName: \"kubernetes.io/projected/82903e3e-982c-4708-a10c-8328e324f7b6-kube-api-access-hh6zt\") pod \"payload-processing-e2e-ait-def78c01-6994fdb9d-kxfn6\" (UID: \"82903e3e-982c-4708-a10c-8328e324f7b6\") " pod="openshift-ingress/payload-processing-e2e-ait-def78c01-6994fdb9d-kxfn6"
Oct 30 18:44:49.202484 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:49.202366    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/82903e3e-982c-4708-a10c-8328e324f7b6-plugins-config-volume\") pod \"payload-processing-e2e-ait-def78c01-6994fdb9d-kxfn6\" (UID: \"82903e3e-982c-4708-a10c-8328e324f7b6\") " pod="openshift-ingress/payload-processing-e2e-ait-def78c01-6994fdb9d-kxfn6"
Oct 30 18:44:49.203174 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:49.203149    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/82903e3e-982c-4708-a10c-8328e324f7b6-plugins-config-volume\") pod \"payload-processing-e2e-ait-def78c01-6994fdb9d-kxfn6\" (UID: \"82903e3e-982c-4708-a10c-8328e324f7b6\") " pod="openshift-ingress/payload-processing-e2e-ait-def78c01-6994fdb9d-kxfn6"
Oct 30 18:44:49.264757 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:49.264735    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hh6zt\" (UniqueName: \"kubernetes.io/projected/82903e3e-982c-4708-a10c-8328e324f7b6-kube-api-access-hh6zt\") pod \"payload-processing-e2e-ait-def78c01-6994fdb9d-kxfn6\" (UID: \"82903e3e-982c-4708-a10c-8328e324f7b6\") " pod="openshift-ingress/payload-processing-e2e-ait-def78c01-6994fdb9d-kxfn6"
Oct 30 18:44:49.291554 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:49.291527    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-def78c01-6994fdb9d-kxfn6"
Oct 30 18:44:49.437466 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:49.437383    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-def78c01-855988f75-dln2g"
Oct 30 18:44:49.676180 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:49.676153    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-def78c01-6994fdb9d-kxfn6"]
Oct 30 18:44:49.677130 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:44:49.677098    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82903e3e_982c_4708_a10c_8328e324f7b6.slice/crio-660e966cd64769e9edffd1d991c00c1530c5ac79b8f168d4b6378b57f8ca2b5f WatchSource:0}: Error finding container 660e966cd64769e9edffd1d991c00c1530c5ac79b8f168d4b6378b57f8ca2b5f: Status 404 returned error can't find the container with id 660e966cd64769e9edffd1d991c00c1530c5ac79b8f168d4b6378b57f8ca2b5f
Oct 30 18:44:49.824955 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:49.824929    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-def78c01-855988f75-dln2g"]
Oct 30 18:44:49.825692 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:44:49.825670    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c207780_577e_483c_b70c_ff0e44e2dc0f.slice/crio-870752801e93969a0ee929e70e71238dc7623ae0ab0bdb9fd0239eb704b616df WatchSource:0}: Error finding container 870752801e93969a0ee929e70e71238dc7623ae0ab0bdb9fd0239eb704b616df: Status 404 returned error can't find the container with id 870752801e93969a0ee929e70e71238dc7623ae0ab0bdb9fd0239eb704b616df
Oct 30 18:44:50.471215 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:50.471181    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-def78c01-855988f75-dln2g" event={"ID":"9c207780-577e-483c-b70c-ff0e44e2dc0f","Type":"ContainerStarted","Data":"3a280c94f8e2a5bfd41cb3e4fc14931ae808e19e5248e13ef2ea354ab2bb0e3f"}
Oct 30 18:44:50.471402 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:50.471232    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-def78c01-855988f75-dln2g" event={"ID":"9c207780-577e-483c-b70c-ff0e44e2dc0f","Type":"ContainerStarted","Data":"870752801e93969a0ee929e70e71238dc7623ae0ab0bdb9fd0239eb704b616df"}
Oct 30 18:44:50.471402 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:50.471263    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-def78c01-855988f75-dln2g"
Oct 30 18:44:50.472975 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:50.472949    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-def78c01-6994fdb9d-kxfn6" event={"ID":"82903e3e-982c-4708-a10c-8328e324f7b6","Type":"ContainerStarted","Data":"7a37934ea4e9b3d755990e8d6455612ba077aca80631c205bd5f7a17dac7785a"}
Oct 30 18:44:50.473098 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:50.473002    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-def78c01-6994fdb9d-kxfn6" event={"ID":"82903e3e-982c-4708-a10c-8328e324f7b6","Type":"ContainerStarted","Data":"660e966cd64769e9edffd1d991c00c1530c5ac79b8f168d4b6378b57f8ca2b5f"}
Oct 30 18:44:50.473098 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:50.473032    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-def78c01-6994fdb9d-kxfn6"
Oct 30 18:44:50.592532 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:50.592485    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-def78c01-855988f75-dln2g" podStartSLOduration=2.592471914 podStartE2EDuration="2.592471914s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:44:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:44:50.589613688 +0000 UTC m=+654.837019712" watchObservedRunningTime="2026-10-30 18:44:50.592471914 +0000 UTC m=+654.839877944"
Oct 30 18:44:51.475350 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:51.475275    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-def78c01-6994fdb9d-kxfn6" podStartSLOduration=3.4752623209999998 podStartE2EDuration="3.475262321s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:44:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:44:50.713785231 +0000 UTC m=+654.961191257" watchObservedRunningTime="2026-10-30 18:44:51.475262321 +0000 UTC m=+655.722668407"
Oct 30 18:44:51.478119 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:51.478099    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-16a49490-779c98d64c-m8q2t"]
Oct 30 18:44:51.481931 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:51.481914    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-16a49490-779c98d64c-m8q2t"
Oct 30 18:44:51.537296 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:51.537248    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-16a49490\""
Oct 30 18:44:51.537538 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:51.537274    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-16a49490-dockercfg-7qmkg\""
Oct 30 18:44:51.551378 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:51.551351    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-16a49490-779c98d64c-m8q2t"]
Oct 30 18:44:51.591139 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:51.591109    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-16a49490-5d9d5546d8-dgjqr"]
Oct 30 18:44:51.594765 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:51.594751    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-16a49490-5d9d5546d8-dgjqr"
Oct 30 18:44:51.629251 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:51.629214    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c7f9892e-42f6-4674-8a84-c1a4d2049807-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-16a49490-779c98d64c-m8q2t\" (UID: \"c7f9892e-42f6-4674-8a84-c1a4d2049807\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-16a49490-779c98d64c-m8q2t"
Oct 30 18:44:51.629765 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:51.629737    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w8h4n\" (UniqueName: \"kubernetes.io/projected/c7f9892e-42f6-4674-8a84-c1a4d2049807-kube-api-access-w8h4n\") pod \"payload-pre-processing-e2e-migrate-16a49490-779c98d64c-m8q2t\" (UID: \"c7f9892e-42f6-4674-8a84-c1a4d2049807\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-16a49490-779c98d64c-m8q2t"
Oct 30 18:44:51.654603 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:51.654567    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-16a49490-5d9d5546d8-dgjqr"]
Oct 30 18:44:51.731804 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:51.731736    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fcd5e44c-96f2-44e7-baea-b55634db50e6-plugins-config-volume\") pod \"payload-processing-e2e-migrate-16a49490-5d9d5546d8-dgjqr\" (UID: \"fcd5e44c-96f2-44e7-baea-b55634db50e6\") " pod="openshift-ingress/payload-processing-e2e-migrate-16a49490-5d9d5546d8-dgjqr"
Oct 30 18:44:51.731939 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:51.731809    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c7f9892e-42f6-4674-8a84-c1a4d2049807-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-16a49490-779c98d64c-m8q2t\" (UID: \"c7f9892e-42f6-4674-8a84-c1a4d2049807\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-16a49490-779c98d64c-m8q2t"
Oct 30 18:44:51.731979 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:51.731942    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pp6pb\" (UniqueName: \"kubernetes.io/projected/fcd5e44c-96f2-44e7-baea-b55634db50e6-kube-api-access-pp6pb\") pod \"payload-processing-e2e-migrate-16a49490-5d9d5546d8-dgjqr\" (UID: \"fcd5e44c-96f2-44e7-baea-b55634db50e6\") " pod="openshift-ingress/payload-processing-e2e-migrate-16a49490-5d9d5546d8-dgjqr"
Oct 30 18:44:51.732073 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:51.732054    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w8h4n\" (UniqueName: \"kubernetes.io/projected/c7f9892e-42f6-4674-8a84-c1a4d2049807-kube-api-access-w8h4n\") pod \"payload-pre-processing-e2e-migrate-16a49490-779c98d64c-m8q2t\" (UID: \"c7f9892e-42f6-4674-8a84-c1a4d2049807\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-16a49490-779c98d64c-m8q2t"
Oct 30 18:44:51.732614 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:51.732598    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c7f9892e-42f6-4674-8a84-c1a4d2049807-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-16a49490-779c98d64c-m8q2t\" (UID: \"c7f9892e-42f6-4674-8a84-c1a4d2049807\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-16a49490-779c98d64c-m8q2t"
Oct 30 18:44:51.794490 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:51.794465    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w8h4n\" (UniqueName: \"kubernetes.io/projected/c7f9892e-42f6-4674-8a84-c1a4d2049807-kube-api-access-w8h4n\") pod \"payload-pre-processing-e2e-migrate-16a49490-779c98d64c-m8q2t\" (UID: \"c7f9892e-42f6-4674-8a84-c1a4d2049807\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-16a49490-779c98d64c-m8q2t"
Oct 30 18:44:51.796218 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:51.796174    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-16a49490-779c98d64c-m8q2t"
Oct 30 18:44:51.832806 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:51.832777    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pp6pb\" (UniqueName: \"kubernetes.io/projected/fcd5e44c-96f2-44e7-baea-b55634db50e6-kube-api-access-pp6pb\") pod \"payload-processing-e2e-migrate-16a49490-5d9d5546d8-dgjqr\" (UID: \"fcd5e44c-96f2-44e7-baea-b55634db50e6\") " pod="openshift-ingress/payload-processing-e2e-migrate-16a49490-5d9d5546d8-dgjqr"
Oct 30 18:44:51.832917 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:51.832880    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fcd5e44c-96f2-44e7-baea-b55634db50e6-plugins-config-volume\") pod \"payload-processing-e2e-migrate-16a49490-5d9d5546d8-dgjqr\" (UID: \"fcd5e44c-96f2-44e7-baea-b55634db50e6\") " pod="openshift-ingress/payload-processing-e2e-migrate-16a49490-5d9d5546d8-dgjqr"
Oct 30 18:44:51.833619 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:51.833593    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fcd5e44c-96f2-44e7-baea-b55634db50e6-plugins-config-volume\") pod \"payload-processing-e2e-migrate-16a49490-5d9d5546d8-dgjqr\" (UID: \"fcd5e44c-96f2-44e7-baea-b55634db50e6\") " pod="openshift-ingress/payload-processing-e2e-migrate-16a49490-5d9d5546d8-dgjqr"
Oct 30 18:44:51.895442 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:51.895413    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pp6pb\" (UniqueName: \"kubernetes.io/projected/fcd5e44c-96f2-44e7-baea-b55634db50e6-kube-api-access-pp6pb\") pod \"payload-processing-e2e-migrate-16a49490-5d9d5546d8-dgjqr\" (UID: \"fcd5e44c-96f2-44e7-baea-b55634db50e6\") " pod="openshift-ingress/payload-processing-e2e-migrate-16a49490-5d9d5546d8-dgjqr"
Oct 30 18:44:51.907329 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:51.907303    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-16a49490-5d9d5546d8-dgjqr"
Oct 30 18:44:52.185529 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:52.185498    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-16a49490-779c98d64c-m8q2t"]
Oct 30 18:44:52.185796 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:44:52.185776    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc7f9892e_42f6_4674_8a84_c1a4d2049807.slice/crio-7ecd54f42a1a9c7895b22031b99a98ce1af9572c2bf999fccd57c2286bed85b6 WatchSource:0}: Error finding container 7ecd54f42a1a9c7895b22031b99a98ce1af9572c2bf999fccd57c2286bed85b6: Status 404 returned error can't find the container with id 7ecd54f42a1a9c7895b22031b99a98ce1af9572c2bf999fccd57c2286bed85b6
Oct 30 18:44:52.313741 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:52.313717    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-16a49490-5d9d5546d8-dgjqr"]
Oct 30 18:44:52.314597 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:44:52.314577    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfcd5e44c_96f2_44e7_baea_b55634db50e6.slice/crio-e75544568f3ba9f3a0b1a398d2de5dc68022bc26f06b470851f7e599a656ba45 WatchSource:0}: Error finding container e75544568f3ba9f3a0b1a398d2de5dc68022bc26f06b470851f7e599a656ba45: Status 404 returned error can't find the container with id e75544568f3ba9f3a0b1a398d2de5dc68022bc26f06b470851f7e599a656ba45
Oct 30 18:44:52.486354 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:52.486320    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-16a49490-5d9d5546d8-dgjqr" event={"ID":"fcd5e44c-96f2-44e7-baea-b55634db50e6","Type":"ContainerStarted","Data":"89336fe57d09afcfbf39f71718770360ab096c7a241b1955d0be1227dd1aeb01"}
Oct 30 18:44:52.486777 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:52.486380    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-16a49490-5d9d5546d8-dgjqr" event={"ID":"fcd5e44c-96f2-44e7-baea-b55634db50e6","Type":"ContainerStarted","Data":"e75544568f3ba9f3a0b1a398d2de5dc68022bc26f06b470851f7e599a656ba45"}
Oct 30 18:44:52.486777 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:52.486408    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-migrate-16a49490-5d9d5546d8-dgjqr"
Oct 30 18:44:52.488054 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:52.488033    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-16a49490-779c98d64c-m8q2t" event={"ID":"c7f9892e-42f6-4674-8a84-c1a4d2049807","Type":"ContainerStarted","Data":"dd2dd56b97aa944cbc505e79eda49819465492504f4f3242ffd3bd091956db06"}
Oct 30 18:44:52.488143 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:52.488069    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-16a49490-779c98d64c-m8q2t" event={"ID":"c7f9892e-42f6-4674-8a84-c1a4d2049807","Type":"ContainerStarted","Data":"7ecd54f42a1a9c7895b22031b99a98ce1af9572c2bf999fccd57c2286bed85b6"}
Oct 30 18:44:52.488177 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:52.488155    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-16a49490-779c98d64c-m8q2t"
Oct 30 18:44:52.612898 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:52.612856    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-migrate-16a49490-5d9d5546d8-dgjqr" podStartSLOduration=1.612843778 podStartE2EDuration="1.612843778s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:44:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:44:52.611309357 +0000 UTC m=+656.858715383" watchObservedRunningTime="2026-10-30 18:44:52.612843778 +0000 UTC m=+656.860249803"
Oct 30 18:44:52.734595 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:52.734545    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-migrate-16a49490-779c98d64c-m8q2t" podStartSLOduration=1.734530358 podStartE2EDuration="1.734530358s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:44:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:44:52.734162824 +0000 UTC m=+656.981568849" watchObservedRunningTime="2026-10-30 18:44:52.734530358 +0000 UTC m=+656.981936388"
Oct 30 18:44:55.611994 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:55.611959    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-rjmdz"]
Oct 30 18:44:56.446002 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:56.445975    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-529e04e2-d954bc5c5-m5jvj"
Oct 30 18:44:58.460135 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:58.460106    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-59144a1b-569dd9dbd8-45qd4"
Oct 30 18:44:58.460559 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:58.460167    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-59144a1b-7d9455dd67-dmj9h"
Oct 30 18:44:58.831223 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:58.831186    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-620f2843-699959648c-2vshr"]
Oct 30 18:44:58.836546 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:58.836523    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-620f2843-699959648c-2vshr"
Oct 30 18:44:58.890516 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:58.890480    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-620f2843-dockercfg-dmbjv\""
Oct 30 18:44:58.890753 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:58.890619    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-620f2843\""
Oct 30 18:44:58.892676 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:58.892648    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-620f2843-699959648c-2vshr"]
Oct 30 18:44:59.023738 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:59.023703    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3fa35762-2452-4ca7-8b74-25ebda4d8d83-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-620f2843-699959648c-2vshr\" (UID: \"3fa35762-2452-4ca7-8b74-25ebda4d8d83\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-620f2843-699959648c-2vshr"
Oct 30 18:44:59.023895 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:59.023768    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kf6kj\" (UniqueName: \"kubernetes.io/projected/3fa35762-2452-4ca7-8b74-25ebda4d8d83-kube-api-access-kf6kj\") pod \"payload-pre-processing-e2e-derive-620f2843-699959648c-2vshr\" (UID: \"3fa35762-2452-4ca7-8b74-25ebda4d8d83\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-620f2843-699959648c-2vshr"
Oct 30 18:44:59.124448 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:59.124358    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3fa35762-2452-4ca7-8b74-25ebda4d8d83-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-620f2843-699959648c-2vshr\" (UID: \"3fa35762-2452-4ca7-8b74-25ebda4d8d83\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-620f2843-699959648c-2vshr"
Oct 30 18:44:59.124448 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:59.124413    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kf6kj\" (UniqueName: \"kubernetes.io/projected/3fa35762-2452-4ca7-8b74-25ebda4d8d83-kube-api-access-kf6kj\") pod \"payload-pre-processing-e2e-derive-620f2843-699959648c-2vshr\" (UID: \"3fa35762-2452-4ca7-8b74-25ebda4d8d83\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-620f2843-699959648c-2vshr"
Oct 30 18:44:59.125275 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:59.125254    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3fa35762-2452-4ca7-8b74-25ebda4d8d83-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-620f2843-699959648c-2vshr\" (UID: \"3fa35762-2452-4ca7-8b74-25ebda4d8d83\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-620f2843-699959648c-2vshr"
Oct 30 18:44:59.188266 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:59.188234    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kf6kj\" (UniqueName: \"kubernetes.io/projected/3fa35762-2452-4ca7-8b74-25ebda4d8d83-kube-api-access-kf6kj\") pod \"payload-pre-processing-e2e-derive-620f2843-699959648c-2vshr\" (UID: \"3fa35762-2452-4ca7-8b74-25ebda4d8d83\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-620f2843-699959648c-2vshr"
Oct 30 18:44:59.450809 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:59.450737    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-620f2843-699959648c-2vshr"
Oct 30 18:44:59.839712 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:44:59.839689    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-620f2843-699959648c-2vshr"]
Oct 30 18:44:59.841862 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:44:59.841830    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3fa35762_2452_4ca7_8b74_25ebda4d8d83.slice/crio-5aa7f7c6972ac07fa28339f1ce87958df8757ce60638097b92fe80e0bec31e5c WatchSource:0}: Error finding container 5aa7f7c6972ac07fa28339f1ce87958df8757ce60638097b92fe80e0bec31e5c: Status 404 returned error can't find the container with id 5aa7f7c6972ac07fa28339f1ce87958df8757ce60638097b92fe80e0bec31e5c
Oct 30 18:45:00.170091 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:00.170017    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-59144a1b-29889765-qxkk7"]
Oct 30 18:45:00.174007 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:00.173982    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-59144a1b-29889765-qxkk7"
Oct 30 18:45:00.177508 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:00.177486    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889765-qknhc"]
Oct 30 18:45:00.181218 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:00.181199    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-529e04e2-29889765-vhkgl"]
Oct 30 18:45:00.181394 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:00.181375    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889765-qknhc"
Oct 30 18:45:00.185103 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:00.185081    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-529e04e2-29889765-vhkgl"
Oct 30 18:45:00.227877 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:00.227840    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-2qg7n\""
Oct 30 18:45:00.231441 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:00.231417    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-59144a1b-29889765-qxkk7"]
Oct 30 18:45:00.231441 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:00.231439    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 30 18:45:00.231584 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:00.231456    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 30 18:45:00.240684 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:00.240660    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/121702e5-40be-4cc8-94a7-897d1ad4c471-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-ait-529e04e2-29889765-vhkgl\" (UID: \"121702e5-40be-4cc8-94a7-897d1ad4c471\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-529e04e2-29889765-vhkgl"
Oct 30 18:45:00.240795 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:00.240723    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sxzwr\" (UniqueName: \"kubernetes.io/projected/4aa64e3a-64c9-4cb1-89a7-19dfd4aa6706-kube-api-access-sxzwr\") pod \"maas-api-key-cleanup-e2e-ait-59144a1b-29889765-qxkk7\" (UID: \"4aa64e3a-64c9-4cb1-89a7-19dfd4aa6706\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-59144a1b-29889765-qxkk7"
Oct 30 18:45:00.240832 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:00.240798    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mp2nf\" (UniqueName: \"kubernetes.io/projected/121702e5-40be-4cc8-94a7-897d1ad4c471-kube-api-access-mp2nf\") pod \"maas-api-key-cleanup-e2e-ait-529e04e2-29889765-vhkgl\" (UID: \"121702e5-40be-4cc8-94a7-897d1ad4c471\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-529e04e2-29889765-vhkgl"
Oct 30 18:45:00.240864 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:00.240836    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4aa64e3a-64c9-4cb1-89a7-19dfd4aa6706-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-ait-59144a1b-29889765-qxkk7\" (UID: \"4aa64e3a-64c9-4cb1-89a7-19dfd4aa6706\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-59144a1b-29889765-qxkk7"
Oct 30 18:45:00.240978 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:00.240955    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nmdwg\" (UniqueName: \"kubernetes.io/projected/21d9b47a-e5e0-4bd7-9a9d-384e1be64ad9-kube-api-access-nmdwg\") pod \"maas-api-key-cleanup-29889765-qknhc\" (UID: \"21d9b47a-e5e0-4bd7-9a9d-384e1be64ad9\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889765-qknhc"
Oct 30 18:45:00.241016 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:00.241002    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/21d9b47a-e5e0-4bd7-9a9d-384e1be64ad9-maas-api-service-ca\") pod \"maas-api-key-cleanup-29889765-qknhc\" (UID: \"21d9b47a-e5e0-4bd7-9a9d-384e1be64ad9\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889765-qknhc"
Oct 30 18:45:00.275386 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:00.275352    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889765-qknhc"]
Oct 30 18:45:00.277520 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:00.277493    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-529e04e2-29889765-vhkgl"]
Oct 30 18:45:00.341866 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:00.341837    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/21d9b47a-e5e0-4bd7-9a9d-384e1be64ad9-maas-api-service-ca\") pod \"maas-api-key-cleanup-29889765-qknhc\" (UID: \"21d9b47a-e5e0-4bd7-9a9d-384e1be64ad9\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889765-qknhc"
Oct 30 18:45:00.342029 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:00.341883    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/121702e5-40be-4cc8-94a7-897d1ad4c471-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-ait-529e04e2-29889765-vhkgl\" (UID: \"121702e5-40be-4cc8-94a7-897d1ad4c471\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-529e04e2-29889765-vhkgl"
Oct 30 18:45:00.342029 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:00.341933    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sxzwr\" (UniqueName: \"kubernetes.io/projected/4aa64e3a-64c9-4cb1-89a7-19dfd4aa6706-kube-api-access-sxzwr\") pod \"maas-api-key-cleanup-e2e-ait-59144a1b-29889765-qxkk7\" (UID: \"4aa64e3a-64c9-4cb1-89a7-19dfd4aa6706\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-59144a1b-29889765-qxkk7"
Oct 30 18:45:00.342029 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:00.341972    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mp2nf\" (UniqueName: \"kubernetes.io/projected/121702e5-40be-4cc8-94a7-897d1ad4c471-kube-api-access-mp2nf\") pod \"maas-api-key-cleanup-e2e-ait-529e04e2-29889765-vhkgl\" (UID: \"121702e5-40be-4cc8-94a7-897d1ad4c471\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-529e04e2-29889765-vhkgl"
Oct 30 18:45:00.342029 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:00.342008    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4aa64e3a-64c9-4cb1-89a7-19dfd4aa6706-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-ait-59144a1b-29889765-qxkk7\" (UID: \"4aa64e3a-64c9-4cb1-89a7-19dfd4aa6706\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-59144a1b-29889765-qxkk7"
Oct 30 18:45:00.342265 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:00.342181    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nmdwg\" (UniqueName: \"kubernetes.io/projected/21d9b47a-e5e0-4bd7-9a9d-384e1be64ad9-kube-api-access-nmdwg\") pod \"maas-api-key-cleanup-29889765-qknhc\" (UID: \"21d9b47a-e5e0-4bd7-9a9d-384e1be64ad9\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889765-qknhc"
Oct 30 18:45:00.342659 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:00.342634    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/21d9b47a-e5e0-4bd7-9a9d-384e1be64ad9-maas-api-service-ca\") pod \"maas-api-key-cleanup-29889765-qknhc\" (UID: \"21d9b47a-e5e0-4bd7-9a9d-384e1be64ad9\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889765-qknhc"
Oct 30 18:45:00.342757 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:00.342666    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/121702e5-40be-4cc8-94a7-897d1ad4c471-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-ait-529e04e2-29889765-vhkgl\" (UID: \"121702e5-40be-4cc8-94a7-897d1ad4c471\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-529e04e2-29889765-vhkgl"
Oct 30 18:45:00.342757 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:00.342672    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4aa64e3a-64c9-4cb1-89a7-19dfd4aa6706-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-ait-59144a1b-29889765-qxkk7\" (UID: \"4aa64e3a-64c9-4cb1-89a7-19dfd4aa6706\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-59144a1b-29889765-qxkk7"
Oct 30 18:45:00.405261 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:00.405235    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mp2nf\" (UniqueName: \"kubernetes.io/projected/121702e5-40be-4cc8-94a7-897d1ad4c471-kube-api-access-mp2nf\") pod \"maas-api-key-cleanup-e2e-ait-529e04e2-29889765-vhkgl\" (UID: \"121702e5-40be-4cc8-94a7-897d1ad4c471\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-529e04e2-29889765-vhkgl"
Oct 30 18:45:00.405450 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:00.405429    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nmdwg\" (UniqueName: \"kubernetes.io/projected/21d9b47a-e5e0-4bd7-9a9d-384e1be64ad9-kube-api-access-nmdwg\") pod \"maas-api-key-cleanup-29889765-qknhc\" (UID: \"21d9b47a-e5e0-4bd7-9a9d-384e1be64ad9\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889765-qknhc"
Oct 30 18:45:00.405502 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:00.405429    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sxzwr\" (UniqueName: \"kubernetes.io/projected/4aa64e3a-64c9-4cb1-89a7-19dfd4aa6706-kube-api-access-sxzwr\") pod \"maas-api-key-cleanup-e2e-ait-59144a1b-29889765-qxkk7\" (UID: \"4aa64e3a-64c9-4cb1-89a7-19dfd4aa6706\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-59144a1b-29889765-qxkk7"
Oct 30 18:45:00.487972 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:00.487904    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-59144a1b-29889765-qxkk7"
Oct 30 18:45:00.498240 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:00.498215    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889765-qknhc"
Oct 30 18:45:00.504258 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:00.504237    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-529e04e2-29889765-vhkgl"
Oct 30 18:45:00.574883 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:00.574851    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-620f2843-699959648c-2vshr" event={"ID":"3fa35762-2452-4ca7-8b74-25ebda4d8d83","Type":"ContainerStarted","Data":"a92254a2d29ae91e09490996ff6d5728bf531588ce5f518c5b24814eeba8ce80"}
Oct 30 18:45:00.575025 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:00.574905    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-620f2843-699959648c-2vshr" event={"ID":"3fa35762-2452-4ca7-8b74-25ebda4d8d83","Type":"ContainerStarted","Data":"5aa7f7c6972ac07fa28339f1ce87958df8757ce60638097b92fe80e0bec31e5c"}
Oct 30 18:45:00.575462 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:00.575381    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-620f2843-699959648c-2vshr"
Oct 30 18:45:00.700922 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:00.700864    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-derive-620f2843-699959648c-2vshr" podStartSLOduration=2.700849405 podStartE2EDuration="2.700849405s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:44:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:45:00.699228778 +0000 UTC m=+664.946634801" watchObservedRunningTime="2026-10-30 18:45:00.700849405 +0000 UTC m=+664.948255429"
Oct 30 18:45:00.875536 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:00.875507    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-59144a1b-29889765-qxkk7"]
Oct 30 18:45:00.876123 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:45:00.876096    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4aa64e3a_64c9_4cb1_89a7_19dfd4aa6706.slice/crio-e437a70cf965669fed73fee3679e0cbcbdd7f9849f8b150266d6529c8e8a97dc WatchSource:0}: Error finding container e437a70cf965669fed73fee3679e0cbcbdd7f9849f8b150266d6529c8e8a97dc: Status 404 returned error can't find the container with id e437a70cf965669fed73fee3679e0cbcbdd7f9849f8b150266d6529c8e8a97dc
Oct 30 18:45:00.924666 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:00.924634    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-529e04e2-29889765-vhkgl"]
Oct 30 18:45:00.925742 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:45:00.925709    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod121702e5_40be_4cc8_94a7_897d1ad4c471.slice/crio-c184bcc397425cf9cde4e28d3e468f9b46aa2bbf963318196bc302317c9b09d4 WatchSource:0}: Error finding container c184bcc397425cf9cde4e28d3e468f9b46aa2bbf963318196bc302317c9b09d4: Status 404 returned error can't find the container with id c184bcc397425cf9cde4e28d3e468f9b46aa2bbf963318196bc302317c9b09d4
Oct 30 18:45:00.926678 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:45:00.926639    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod21d9b47a_e5e0_4bd7_9a9d_384e1be64ad9.slice/crio-ec2a496b125aa0d378833e51513d9ecc0fb1afb7c7fb92f5a1550ae82ee24c3e WatchSource:0}: Error finding container ec2a496b125aa0d378833e51513d9ecc0fb1afb7c7fb92f5a1550ae82ee24c3e: Status 404 returned error can't find the container with id ec2a496b125aa0d378833e51513d9ecc0fb1afb7c7fb92f5a1550ae82ee24c3e
Oct 30 18:45:00.927776 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:00.927716    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889765-qknhc"]
Oct 30 18:45:01.482809 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:01.482784    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-def78c01-855988f75-dln2g"
Oct 30 18:45:01.483017 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:01.482847    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-def78c01-6994fdb9d-kxfn6"
Oct 30 18:45:01.582590 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:01.582555    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-59144a1b-29889765-qxkk7" event={"ID":"4aa64e3a-64c9-4cb1-89a7-19dfd4aa6706","Type":"ContainerStarted","Data":"e437a70cf965669fed73fee3679e0cbcbdd7f9849f8b150266d6529c8e8a97dc"}
Oct 30 18:45:01.583803 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:01.583774    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-529e04e2-29889765-vhkgl" event={"ID":"121702e5-40be-4cc8-94a7-897d1ad4c471","Type":"ContainerStarted","Data":"c184bcc397425cf9cde4e28d3e468f9b46aa2bbf963318196bc302317c9b09d4"}
Oct 30 18:45:01.584922 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:01.584901    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889765-qknhc" event={"ID":"21d9b47a-e5e0-4bd7-9a9d-384e1be64ad9","Type":"ContainerStarted","Data":"ec2a496b125aa0d378833e51513d9ecc0fb1afb7c7fb92f5a1550ae82ee24c3e"}
Oct 30 18:45:03.498176 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:03.498149    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-16a49490-779c98d64c-m8q2t"
Oct 30 18:45:03.499840 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:03.499816    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-migrate-16a49490-5d9d5546d8-dgjqr"
Oct 30 18:45:05.628134 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:45:05.628094    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-jbsw6" containerName="extractor"
Oct 30 18:45:05.900952 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:05.900926    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-rjmdz"]
Oct 30 18:45:06.628984 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:06.628959    4888 generic.go:356] "Generic (PLEG): container finished" podID="121702e5-40be-4cc8-94a7-897d1ad4c471" containerID="05194c45cde4456b1c8505cb8ce4b3b83740e5834e8a226e87cdafbf63c94573" exitCode=7
Oct 30 18:45:06.629436 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:06.629049    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-529e04e2-29889765-vhkgl" event={"ID":"121702e5-40be-4cc8-94a7-897d1ad4c471","Type":"ContainerDied","Data":"05194c45cde4456b1c8505cb8ce4b3b83740e5834e8a226e87cdafbf63c94573"}
Oct 30 18:45:06.629436 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:06.629270    4888 scope.go:122] "RemoveContainer" containerID="05194c45cde4456b1c8505cb8ce4b3b83740e5834e8a226e87cdafbf63c94573"
Oct 30 18:45:06.630805 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:06.630770    4888 generic.go:356] "Generic (PLEG): container finished" podID="21d9b47a-e5e0-4bd7-9a9d-384e1be64ad9" containerID="ad6709986319546d99293a80f92ca5e8cd050b7154d0f20394c7f03a743f028d" exitCode=60
Oct 30 18:45:06.630918 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:06.630855    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889765-qknhc" event={"ID":"21d9b47a-e5e0-4bd7-9a9d-384e1be64ad9","Type":"ContainerDied","Data":"ad6709986319546d99293a80f92ca5e8cd050b7154d0f20394c7f03a743f028d"}
Oct 30 18:45:06.631212 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:06.631189    4888 scope.go:122] "RemoveContainer" containerID="ad6709986319546d99293a80f92ca5e8cd050b7154d0f20394c7f03a743f028d"
Oct 30 18:45:06.632650 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:06.632628    4888 generic.go:356] "Generic (PLEG): container finished" podID="4aa64e3a-64c9-4cb1-89a7-19dfd4aa6706" containerID="b598b729d399e65778fc516a5b59089edb110d5792b5f387762edcc317e713ad" exitCode=60
Oct 30 18:45:06.632748 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:06.632682    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-59144a1b-29889765-qxkk7" event={"ID":"4aa64e3a-64c9-4cb1-89a7-19dfd4aa6706","Type":"ContainerDied","Data":"b598b729d399e65778fc516a5b59089edb110d5792b5f387762edcc317e713ad"}
Oct 30 18:45:06.632887 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:06.632873    4888 scope.go:122] "RemoveContainer" containerID="b598b729d399e65778fc516a5b59089edb110d5792b5f387762edcc317e713ad"
Oct 30 18:45:07.640489 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:07.640465    4888 generic.go:356] "Generic (PLEG): container finished" podID="4aa64e3a-64c9-4cb1-89a7-19dfd4aa6706" containerID="214773e35a1ed77e88e8198eebee68cf9ee6d18e94c0d004604a13fb2743c102" exitCode=60
Oct 30 18:45:07.640878 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:07.640554    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-59144a1b-29889765-qxkk7" event={"ID":"4aa64e3a-64c9-4cb1-89a7-19dfd4aa6706","Type":"ContainerDied","Data":"214773e35a1ed77e88e8198eebee68cf9ee6d18e94c0d004604a13fb2743c102"}
Oct 30 18:45:07.640878 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:07.640613    4888 scope.go:122] "RemoveContainer" containerID="b598b729d399e65778fc516a5b59089edb110d5792b5f387762edcc317e713ad"
Oct 30 18:45:07.640878 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:07.640847    4888 scope.go:122] "RemoveContainer" containerID="214773e35a1ed77e88e8198eebee68cf9ee6d18e94c0d004604a13fb2743c102"
Oct 30 18:45:07.641113 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:45:07.641094    4888 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-ait-59144a1b-29889765-qxkk7_odh-ai-gateway-infra(4aa64e3a-64c9-4cb1-89a7-19dfd4aa6706)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-59144a1b-29889765-qxkk7" podUID="4aa64e3a-64c9-4cb1-89a7-19dfd4aa6706"
Oct 30 18:45:07.642651 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:07.642632    4888 generic.go:356] "Generic (PLEG): container finished" podID="121702e5-40be-4cc8-94a7-897d1ad4c471" containerID="7329a4ce384658edae5d1de06c01706e2f108d67fb2e2bfb2f7e72ea53005185" exitCode=7
Oct 30 18:45:07.642747 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:07.642736    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-529e04e2-29889765-vhkgl" event={"ID":"121702e5-40be-4cc8-94a7-897d1ad4c471","Type":"ContainerDied","Data":"7329a4ce384658edae5d1de06c01706e2f108d67fb2e2bfb2f7e72ea53005185"}
Oct 30 18:45:07.642922 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:07.642909    4888 scope.go:122] "RemoveContainer" containerID="7329a4ce384658edae5d1de06c01706e2f108d67fb2e2bfb2f7e72ea53005185"
Oct 30 18:45:07.643151 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:45:07.643133    4888 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-ait-529e04e2-29889765-vhkgl_odh-ai-gateway-infra(121702e5-40be-4cc8-94a7-897d1ad4c471)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-529e04e2-29889765-vhkgl" podUID="121702e5-40be-4cc8-94a7-897d1ad4c471"
Oct 30 18:45:07.644536 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:07.644517    4888 generic.go:356] "Generic (PLEG): container finished" podID="21d9b47a-e5e0-4bd7-9a9d-384e1be64ad9" containerID="d69e51f6ec91c14327a876f3fa2b83c39b034b4a619b271a24d6ea4ec2197f39" exitCode=60
Oct 30 18:45:07.644632 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:07.644582    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889765-qknhc" event={"ID":"21d9b47a-e5e0-4bd7-9a9d-384e1be64ad9","Type":"ContainerDied","Data":"d69e51f6ec91c14327a876f3fa2b83c39b034b4a619b271a24d6ea4ec2197f39"}
Oct 30 18:45:07.644861 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:07.644782    4888 scope.go:122] "RemoveContainer" containerID="d69e51f6ec91c14327a876f3fa2b83c39b034b4a619b271a24d6ea4ec2197f39"
Oct 30 18:45:07.644987 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:45:07.644970    4888 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-29889765-qknhc_odh-ai-gateway-infra(21d9b47a-e5e0-4bd7-9a9d-384e1be64ad9)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889765-qknhc" podUID="21d9b47a-e5e0-4bd7-9a9d-384e1be64ad9"
Oct 30 18:45:07.705572 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:07.705543    4888 scope.go:122] "RemoveContainer" containerID="05194c45cde4456b1c8505cb8ce4b3b83740e5834e8a226e87cdafbf63c94573"
Oct 30 18:45:07.809137 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:07.809111    4888 scope.go:122] "RemoveContainer" containerID="ad6709986319546d99293a80f92ca5e8cd050b7154d0f20394c7f03a743f028d"
Oct 30 18:45:08.654311 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:08.654269    4888 scope.go:122] "RemoveContainer" containerID="214773e35a1ed77e88e8198eebee68cf9ee6d18e94c0d004604a13fb2743c102"
Oct 30 18:45:08.654812 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:45:08.654598    4888 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-ait-59144a1b-29889765-qxkk7_odh-ai-gateway-infra(4aa64e3a-64c9-4cb1-89a7-19dfd4aa6706)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-59144a1b-29889765-qxkk7" podUID="4aa64e3a-64c9-4cb1-89a7-19dfd4aa6706"
Oct 30 18:45:08.655937 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:08.655915    4888 scope.go:122] "RemoveContainer" containerID="7329a4ce384658edae5d1de06c01706e2f108d67fb2e2bfb2f7e72ea53005185"
Oct 30 18:45:08.656169 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:45:08.656148    4888 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-ait-529e04e2-29889765-vhkgl_odh-ai-gateway-infra(121702e5-40be-4cc8-94a7-897d1ad4c471)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-529e04e2-29889765-vhkgl" podUID="121702e5-40be-4cc8-94a7-897d1ad4c471"
Oct 30 18:45:08.657334 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:08.657315    4888 scope.go:122] "RemoveContainer" containerID="d69e51f6ec91c14327a876f3fa2b83c39b034b4a619b271a24d6ea4ec2197f39"
Oct 30 18:45:08.657571 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:45:08.657550    4888 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-29889765-qknhc_odh-ai-gateway-infra(21d9b47a-e5e0-4bd7-9a9d-384e1be64ad9)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889765-qknhc" podUID="21d9b47a-e5e0-4bd7-9a9d-384e1be64ad9"
Oct 30 18:45:09.253807 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:09.253773    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-620f2843-fd7f7f68f-vt65m"]
Oct 30 18:45:09.257739 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:09.257723    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-620f2843-fd7f7f68f-vt65m"
Oct 30 18:45:09.314947 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:09.314681    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-gxfk7\""
Oct 30 18:45:09.314947 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:09.314814    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-620f2843-serving-cert\""
Oct 30 18:45:09.314947 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:09.314900    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Oct 30 18:45:09.318985 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:09.318961    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-620f2843-fd7f7f68f-vt65m"]
Oct 30 18:45:09.341148 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:09.341121    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/790c0c38-1b9e-4fbc-b765-a7d280ac665c-maas-api-tls\") pod \"maas-api-e2e-derive-620f2843-fd7f7f68f-vt65m\" (UID: \"790c0c38-1b9e-4fbc-b765-a7d280ac665c\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-620f2843-fd7f7f68f-vt65m"
Oct 30 18:45:09.341307 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:09.341190    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-whp5r\" (UniqueName: \"kubernetes.io/projected/790c0c38-1b9e-4fbc-b765-a7d280ac665c-kube-api-access-whp5r\") pod \"maas-api-e2e-derive-620f2843-fd7f7f68f-vt65m\" (UID: \"790c0c38-1b9e-4fbc-b765-a7d280ac665c\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-620f2843-fd7f7f68f-vt65m"
Oct 30 18:45:09.341374 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:09.341350    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/790c0c38-1b9e-4fbc-b765-a7d280ac665c-openshift-service-ca\") pod \"maas-api-e2e-derive-620f2843-fd7f7f68f-vt65m\" (UID: \"790c0c38-1b9e-4fbc-b765-a7d280ac665c\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-620f2843-fd7f7f68f-vt65m"
Oct 30 18:45:09.341510 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:09.341490    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/790c0c38-1b9e-4fbc-b765-a7d280ac665c-metrics-certs\") pod \"maas-api-e2e-derive-620f2843-fd7f7f68f-vt65m\" (UID: \"790c0c38-1b9e-4fbc-b765-a7d280ac665c\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-620f2843-fd7f7f68f-vt65m"
Oct 30 18:45:09.442626 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:09.442595    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/790c0c38-1b9e-4fbc-b765-a7d280ac665c-metrics-certs\") pod \"maas-api-e2e-derive-620f2843-fd7f7f68f-vt65m\" (UID: \"790c0c38-1b9e-4fbc-b765-a7d280ac665c\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-620f2843-fd7f7f68f-vt65m"
Oct 30 18:45:09.442812 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:09.442676    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/790c0c38-1b9e-4fbc-b765-a7d280ac665c-maas-api-tls\") pod \"maas-api-e2e-derive-620f2843-fd7f7f68f-vt65m\" (UID: \"790c0c38-1b9e-4fbc-b765-a7d280ac665c\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-620f2843-fd7f7f68f-vt65m"
Oct 30 18:45:09.442812 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:09.442730    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-whp5r\" (UniqueName: \"kubernetes.io/projected/790c0c38-1b9e-4fbc-b765-a7d280ac665c-kube-api-access-whp5r\") pod \"maas-api-e2e-derive-620f2843-fd7f7f68f-vt65m\" (UID: \"790c0c38-1b9e-4fbc-b765-a7d280ac665c\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-620f2843-fd7f7f68f-vt65m"
Oct 30 18:45:09.442812 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:09.442774    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/790c0c38-1b9e-4fbc-b765-a7d280ac665c-openshift-service-ca\") pod \"maas-api-e2e-derive-620f2843-fd7f7f68f-vt65m\" (UID: \"790c0c38-1b9e-4fbc-b765-a7d280ac665c\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-620f2843-fd7f7f68f-vt65m"
Oct 30 18:45:09.443200 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:45:09.443176    4888 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-620f2843-serving-cert: secret "maas-api-e2e-derive-620f2843-serving-cert" not found
Oct 30 18:45:09.443378 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:45:09.443246    4888 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/790c0c38-1b9e-4fbc-b765-a7d280ac665c-maas-api-tls podName:790c0c38-1b9e-4fbc-b765-a7d280ac665c nodeName:}" failed. No retries permitted until 2026-10-30 18:45:09.943226761 +0000 UTC m=+674.190632783 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/790c0c38-1b9e-4fbc-b765-a7d280ac665c-maas-api-tls") pod "maas-api-e2e-derive-620f2843-fd7f7f68f-vt65m" (UID: "790c0c38-1b9e-4fbc-b765-a7d280ac665c") : secret "maas-api-e2e-derive-620f2843-serving-cert" not found
Oct 30 18:45:09.443496 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:09.443471    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/790c0c38-1b9e-4fbc-b765-a7d280ac665c-openshift-service-ca\") pod \"maas-api-e2e-derive-620f2843-fd7f7f68f-vt65m\" (UID: \"790c0c38-1b9e-4fbc-b765-a7d280ac665c\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-620f2843-fd7f7f68f-vt65m"
Oct 30 18:45:09.445792 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:09.445649    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/790c0c38-1b9e-4fbc-b765-a7d280ac665c-metrics-certs\") pod \"maas-api-e2e-derive-620f2843-fd7f7f68f-vt65m\" (UID: \"790c0c38-1b9e-4fbc-b765-a7d280ac665c\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-620f2843-fd7f7f68f-vt65m"
Oct 30 18:45:09.510618 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:09.510553    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-whp5r\" (UniqueName: \"kubernetes.io/projected/790c0c38-1b9e-4fbc-b765-a7d280ac665c-kube-api-access-whp5r\") pod \"maas-api-e2e-derive-620f2843-fd7f7f68f-vt65m\" (UID: \"790c0c38-1b9e-4fbc-b765-a7d280ac665c\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-620f2843-fd7f7f68f-vt65m"
Oct 30 18:45:09.947239 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:09.947210    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/790c0c38-1b9e-4fbc-b765-a7d280ac665c-maas-api-tls\") pod \"maas-api-e2e-derive-620f2843-fd7f7f68f-vt65m\" (UID: \"790c0c38-1b9e-4fbc-b765-a7d280ac665c\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-620f2843-fd7f7f68f-vt65m"
Oct 30 18:45:09.949749 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:09.949730    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/790c0c38-1b9e-4fbc-b765-a7d280ac665c-maas-api-tls\") pod \"maas-api-e2e-derive-620f2843-fd7f7f68f-vt65m\" (UID: \"790c0c38-1b9e-4fbc-b765-a7d280ac665c\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-620f2843-fd7f7f68f-vt65m"
Oct 30 18:45:10.184500 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:10.184464    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-620f2843-fd7f7f68f-vt65m"
Oct 30 18:45:10.597316 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:10.594897    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-620f2843-fd7f7f68f-vt65m"]
Oct 30 18:45:10.597316 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:45:10.597181    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod790c0c38_1b9e_4fbc_b765_a7d280ac665c.slice/crio-d6c4c1a7886150f9066aef93078c6692fb540c5ac4b779e8e0cfcc90164a7e3e WatchSource:0}: Error finding container d6c4c1a7886150f9066aef93078c6692fb540c5ac4b779e8e0cfcc90164a7e3e: Status 404 returned error can't find the container with id d6c4c1a7886150f9066aef93078c6692fb540c5ac4b779e8e0cfcc90164a7e3e
Oct 30 18:45:10.692388 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:10.691678    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-620f2843-fd7f7f68f-vt65m" event={"ID":"790c0c38-1b9e-4fbc-b765-a7d280ac665c","Type":"ContainerStarted","Data":"d6c4c1a7886150f9066aef93078c6692fb540c5ac4b779e8e0cfcc90164a7e3e"}
Oct 30 18:45:11.593143 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:11.593114    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-620f2843-699959648c-2vshr"
Oct 30 18:45:13.717506 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:13.717465    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-620f2843-fd7f7f68f-vt65m" event={"ID":"790c0c38-1b9e-4fbc-b765-a7d280ac665c","Type":"ContainerStarted","Data":"4f145df9c33b492e96215dc4eee656f62eb551a3713a5f9aefdd2774a2767123"}
Oct 30 18:45:13.717977 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:13.717677    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-620f2843-fd7f7f68f-vt65m"
Oct 30 18:45:16.000169 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:16.000097    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-620f2843-fd7f7f68f-vt65m" podStartSLOduration=4.6944698670000005 podStartE2EDuration="7.000077854s" totalImagesPullingTime="2.305607987s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:45:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:45:13.841573318 +0000 UTC m=+678.088979343" watchObservedRunningTime="2026-10-30 18:45:16.000077854 +0000 UTC m=+680.247483892"
Oct 30 18:45:16.003322 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:16.003299    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-rjmdz"]
Oct 30 18:45:16.745091 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:16.745060    4888 generic.go:356] "Generic (PLEG): container finished" podID="cf71a4b7-6006-44ba-945c-bb8749af44ad" containerID="ee735a6244ff50f6b0d5fc8a91a7a09353311d638bff44afe7c9f112d9276a24" exitCode=0
Oct 30 18:45:16.745342 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:16.745147    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-59144a1b-openshift-default-78d555d5bc-lk4t7" event={"ID":"cf71a4b7-6006-44ba-945c-bb8749af44ad","Type":"ContainerDied","Data":"ee735a6244ff50f6b0d5fc8a91a7a09353311d638bff44afe7c9f112d9276a24"}
Oct 30 18:45:16.745718 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:16.745676    4888 scope.go:122] "RemoveContainer" containerID="ee735a6244ff50f6b0d5fc8a91a7a09353311d638bff44afe7c9f112d9276a24"
Oct 30 18:45:16.747809 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:16.747766    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:45:16.747920 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:16.747853    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:45:16.747920 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:16.747898    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:45:17.753749 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:17.753716    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-59144a1b-openshift-default-78d555d5bc-lk4t7" event={"ID":"cf71a4b7-6006-44ba-945c-bb8749af44ad","Type":"ContainerStarted","Data":"86c4440779a9ac1a1ef7a22332ca112e088b7faca1d90f3250aea8d81a925921"}
Oct 30 18:45:18.454005 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:18.453972    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-59144a1b-openshift-default-78d555d5bc-lk4t7"
Oct 30 18:45:18.455300 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:18.455259    4888 patch_prober.go:29] interesting pod/e2e-ait-59144a1b-openshift-default-78d555d5bc-lk4t7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body=
Oct 30 18:45:18.455431 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:18.455340    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-59144a1b-openshift-default-78d555d5bc-lk4t7" podUID="cf71a4b7-6006-44ba-945c-bb8749af44ad" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused"
Oct 30 18:45:18.887613 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:18.887580    4888 scope.go:122] "RemoveContainer" containerID="214773e35a1ed77e88e8198eebee68cf9ee6d18e94c0d004604a13fb2743c102"
Oct 30 18:45:19.453581 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:19.453544    4888 patch_prober.go:29] interesting pod/e2e-ait-59144a1b-openshift-default-78d555d5bc-lk4t7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body=
Oct 30 18:45:19.453778 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:19.453611    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-59144a1b-openshift-default-78d555d5bc-lk4t7" podUID="cf71a4b7-6006-44ba-945c-bb8749af44ad" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused"
Oct 30 18:45:19.733381 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:19.733355    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-620f2843-fd7f7f68f-vt65m"
Oct 30 18:45:19.779146 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:19.779115    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-59144a1b-29889765-qxkk7" event={"ID":"4aa64e3a-64c9-4cb1-89a7-19dfd4aa6706","Type":"ContainerStarted","Data":"5432d2f8bfc64057883737ec76d6e7f97df8df049c4a959d775bef0d9b97a076"}
Oct 30 18:45:19.887273 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:19.886809    4888 scope.go:122] "RemoveContainer" containerID="d69e51f6ec91c14327a876f3fa2b83c39b034b4a619b271a24d6ea4ec2197f39"
Oct 30 18:45:19.890122 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:19.890101    4888 scope.go:122] "RemoveContainer" containerID="7329a4ce384658edae5d1de06c01706e2f108d67fb2e2bfb2f7e72ea53005185"
Oct 30 18:45:20.000350 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:20.000143    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-59144a1b-29889765-qxkk7" podStartSLOduration=15.296859316 podStartE2EDuration="20.000125243s" totalImagesPullingTime="4.703265927s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:45:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:45:19.998943687 +0000 UTC m=+684.246349713" watchObservedRunningTime="2026-10-30 18:45:20.000125243 +0000 UTC m=+684.247531269"
Oct 30 18:45:20.453686 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:20.453650    4888 patch_prober.go:29] interesting pod/e2e-ait-59144a1b-openshift-default-78d555d5bc-lk4t7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body=
Oct 30 18:45:20.453864 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:20.453705    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-59144a1b-openshift-default-78d555d5bc-lk4t7" podUID="cf71a4b7-6006-44ba-945c-bb8749af44ad" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused"
Oct 30 18:45:20.799156 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:20.799063    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889765-qknhc" event={"ID":"21d9b47a-e5e0-4bd7-9a9d-384e1be64ad9","Type":"ContainerStarted","Data":"c9591eb22d47104f1f6b806e38605e2f382daed89b3a040b84db154393bbe8ac"}
Oct 30 18:45:20.800914 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:20.800891    4888 generic.go:356] "Generic (PLEG): container finished" podID="4aa64e3a-64c9-4cb1-89a7-19dfd4aa6706" containerID="5432d2f8bfc64057883737ec76d6e7f97df8df049c4a959d775bef0d9b97a076" exitCode=60
Oct 30 18:45:20.801057 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:20.800957    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-59144a1b-29889765-qxkk7" event={"ID":"4aa64e3a-64c9-4cb1-89a7-19dfd4aa6706","Type":"ContainerDied","Data":"5432d2f8bfc64057883737ec76d6e7f97df8df049c4a959d775bef0d9b97a076"}
Oct 30 18:45:20.801057 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:20.801011    4888 scope.go:122] "RemoveContainer" containerID="214773e35a1ed77e88e8198eebee68cf9ee6d18e94c0d004604a13fb2743c102"
Oct 30 18:45:20.801424 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:20.801411    4888 scope.go:122] "RemoveContainer" containerID="5432d2f8bfc64057883737ec76d6e7f97df8df049c4a959d775bef0d9b97a076"
Oct 30 18:45:20.801891 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:45:20.801750    4888 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-ait-59144a1b-29889765-qxkk7_odh-ai-gateway-infra(4aa64e3a-64c9-4cb1-89a7-19dfd4aa6706)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-59144a1b-29889765-qxkk7" podUID="4aa64e3a-64c9-4cb1-89a7-19dfd4aa6706"
Oct 30 18:45:20.803129 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:20.803108    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-529e04e2-29889765-vhkgl" event={"ID":"121702e5-40be-4cc8-94a7-897d1ad4c471","Type":"ContainerStarted","Data":"e44a4a522ca25ba1522c492fc726211f4860cc9b965977fd5ae446e25b7f21d7"}
Oct 30 18:45:20.921222 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:20.921179    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889765-qknhc" podStartSLOduration=16.36466087 podStartE2EDuration="20.92116315s" totalImagesPullingTime="4.55650228s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:45:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:45:20.91816016 +0000 UTC m=+685.165566187" watchObservedRunningTime="2026-10-30 18:45:20.92116315 +0000 UTC m=+685.168569174"
Oct 30 18:45:20.996183 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:20.996093    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-59144a1b-29889765-qxkk7"]
Oct 30 18:45:21.160360 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:21.160078    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-529e04e2-29889765-vhkgl" podStartSLOduration=16.591546771 podStartE2EDuration="21.160063532s" totalImagesPullingTime="4.568516761s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:45:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:45:21.158966789 +0000 UTC m=+685.406372813" watchObservedRunningTime="2026-10-30 18:45:21.160063532 +0000 UTC m=+685.407469596"
Oct 30 18:45:21.383309 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:21.371378    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-59144a1b-openshift-default-78d555d5bc-lk4t7"
Oct 30 18:45:21.453565 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:21.453486    4888 patch_prober.go:29] interesting pod/e2e-ait-59144a1b-openshift-default-78d555d5bc-lk4t7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body=
Oct 30 18:45:21.453565 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:21.453537    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-59144a1b-openshift-default-78d555d5bc-lk4t7" podUID="cf71a4b7-6006-44ba-945c-bb8749af44ad" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused"
Oct 30 18:45:21.811103 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:21.811017    4888 generic.go:356] "Generic (PLEG): container finished" podID="121702e5-40be-4cc8-94a7-897d1ad4c471" containerID="e44a4a522ca25ba1522c492fc726211f4860cc9b965977fd5ae446e25b7f21d7" exitCode=60
Oct 30 18:45:21.811278 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:21.811148    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-529e04e2-29889765-vhkgl" event={"ID":"121702e5-40be-4cc8-94a7-897d1ad4c471","Type":"ContainerDied","Data":"e44a4a522ca25ba1522c492fc726211f4860cc9b965977fd5ae446e25b7f21d7"}
Oct 30 18:45:21.811278 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:21.811203    4888 scope.go:122] "RemoveContainer" containerID="7329a4ce384658edae5d1de06c01706e2f108d67fb2e2bfb2f7e72ea53005185"
Oct 30 18:45:21.811901 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:21.811880    4888 scope.go:122] "RemoveContainer" containerID="e44a4a522ca25ba1522c492fc726211f4860cc9b965977fd5ae446e25b7f21d7"
Oct 30 18:45:21.812247 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:45:21.812224    4888 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-ait-529e04e2-29889765-vhkgl_odh-ai-gateway-infra(121702e5-40be-4cc8-94a7-897d1ad4c471)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-529e04e2-29889765-vhkgl" podUID="121702e5-40be-4cc8-94a7-897d1ad4c471"
Oct 30 18:45:21.814929 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:21.814907    4888 generic.go:356] "Generic (PLEG): container finished" podID="21d9b47a-e5e0-4bd7-9a9d-384e1be64ad9" containerID="c9591eb22d47104f1f6b806e38605e2f382daed89b3a040b84db154393bbe8ac" exitCode=60
Oct 30 18:45:21.815064 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:21.814966    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889765-qknhc" event={"ID":"21d9b47a-e5e0-4bd7-9a9d-384e1be64ad9","Type":"ContainerDied","Data":"c9591eb22d47104f1f6b806e38605e2f382daed89b3a040b84db154393bbe8ac"}
Oct 30 18:45:21.815248 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:21.815235    4888 scope.go:122] "RemoveContainer" containerID="c9591eb22d47104f1f6b806e38605e2f382daed89b3a040b84db154393bbe8ac"
Oct 30 18:45:21.815479 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:45:21.815463    4888 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-29889765-qknhc_odh-ai-gateway-infra(21d9b47a-e5e0-4bd7-9a9d-384e1be64ad9)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889765-qknhc" podUID="21d9b47a-e5e0-4bd7-9a9d-384e1be64ad9"
Oct 30 18:45:21.822752 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:21.822739    4888 scope.go:122] "RemoveContainer" containerID="d69e51f6ec91c14327a876f3fa2b83c39b034b4a619b271a24d6ea4ec2197f39"
Oct 30 18:45:21.941132 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:21.941102    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889765-qknhc"]
Oct 30 18:45:22.013174 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:22.013150    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-59144a1b-29889765-qxkk7"
Oct 30 18:45:22.043729 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:22.043704    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4aa64e3a-64c9-4cb1-89a7-19dfd4aa6706-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4aa64e3a-64c9-4cb1-89a7-19dfd4aa6706-maas-api-service-ca\") pod \"4aa64e3a-64c9-4cb1-89a7-19dfd4aa6706\" (UID: \"4aa64e3a-64c9-4cb1-89a7-19dfd4aa6706\") "
Oct 30 18:45:22.043888 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:22.043753    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4aa64e3a-64c9-4cb1-89a7-19dfd4aa6706-kube-api-access-sxzwr\" (UniqueName: \"kubernetes.io/projected/4aa64e3a-64c9-4cb1-89a7-19dfd4aa6706-kube-api-access-sxzwr\") pod \"4aa64e3a-64c9-4cb1-89a7-19dfd4aa6706\" (UID: \"4aa64e3a-64c9-4cb1-89a7-19dfd4aa6706\") "
Oct 30 18:45:22.044602 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:22.044552    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4aa64e3a-64c9-4cb1-89a7-19dfd4aa6706-maas-api-service-ca" pod "4aa64e3a-64c9-4cb1-89a7-19dfd4aa6706" (UID: "4aa64e3a-64c9-4cb1-89a7-19dfd4aa6706"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:45:22.044854 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:22.044703    4888 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4aa64e3a-64c9-4cb1-89a7-19dfd4aa6706-maas-api-service-ca\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:45:22.048551 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:22.048519    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4aa64e3a-64c9-4cb1-89a7-19dfd4aa6706-kube-api-access-sxzwr" pod "4aa64e3a-64c9-4cb1-89a7-19dfd4aa6706" (UID: "4aa64e3a-64c9-4cb1-89a7-19dfd4aa6706"). InnerVolumeSpecName "kube-api-access-sxzwr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:45:22.145819 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:22.145786    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sxzwr\" (UniqueName: \"kubernetes.io/projected/4aa64e3a-64c9-4cb1-89a7-19dfd4aa6706-kube-api-access-sxzwr\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:45:22.194419 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:22.194370    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-529e04e2-29889765-vhkgl"]
Oct 30 18:45:22.453703 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:22.453617    4888 patch_prober.go:29] interesting pod/e2e-ait-59144a1b-openshift-default-78d555d5bc-lk4t7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body=
Oct 30 18:45:22.453703 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:22.453673    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-59144a1b-openshift-default-78d555d5bc-lk4t7" podUID="cf71a4b7-6006-44ba-945c-bb8749af44ad" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused"
Oct 30 18:45:22.826270 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:22.826247    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-59144a1b-29889765-qxkk7"
Oct 30 18:45:22.826959 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:22.826243    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-59144a1b-29889765-qxkk7" event={"ID":"4aa64e3a-64c9-4cb1-89a7-19dfd4aa6706","Type":"ContainerDied","Data":"e437a70cf965669fed73fee3679e0cbcbdd7f9849f8b150266d6529c8e8a97dc"}
Oct 30 18:45:22.827082 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:22.827023    4888 scope.go:122] "RemoveContainer" containerID="5432d2f8bfc64057883737ec76d6e7f97df8df049c4a959d775bef0d9b97a076"
Oct 30 18:45:22.999342 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:22.999318    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889765-qknhc"
Oct 30 18:45:23.022989 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:23.022971    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-529e04e2-29889765-vhkgl"
Oct 30 18:45:23.055818 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:23.055792    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/121702e5-40be-4cc8-94a7-897d1ad4c471-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/121702e5-40be-4cc8-94a7-897d1ad4c471-maas-api-service-ca\") pod \"121702e5-40be-4cc8-94a7-897d1ad4c471\" (UID: \"121702e5-40be-4cc8-94a7-897d1ad4c471\") "
Oct 30 18:45:23.055929 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:23.055864    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/121702e5-40be-4cc8-94a7-897d1ad4c471-kube-api-access-mp2nf\" (UniqueName: \"kubernetes.io/projected/121702e5-40be-4cc8-94a7-897d1ad4c471-kube-api-access-mp2nf\") pod \"121702e5-40be-4cc8-94a7-897d1ad4c471\" (UID: \"121702e5-40be-4cc8-94a7-897d1ad4c471\") "
Oct 30 18:45:23.055929 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:23.055907    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/21d9b47a-e5e0-4bd7-9a9d-384e1be64ad9-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/21d9b47a-e5e0-4bd7-9a9d-384e1be64ad9-maas-api-service-ca\") pod \"21d9b47a-e5e0-4bd7-9a9d-384e1be64ad9\" (UID: \"21d9b47a-e5e0-4bd7-9a9d-384e1be64ad9\") "
Oct 30 18:45:23.056031 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:23.056010    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/21d9b47a-e5e0-4bd7-9a9d-384e1be64ad9-kube-api-access-nmdwg\" (UniqueName: \"kubernetes.io/projected/21d9b47a-e5e0-4bd7-9a9d-384e1be64ad9-kube-api-access-nmdwg\") pod \"21d9b47a-e5e0-4bd7-9a9d-384e1be64ad9\" (UID: \"21d9b47a-e5e0-4bd7-9a9d-384e1be64ad9\") "
Oct 30 18:45:23.056251 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:23.056224    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/121702e5-40be-4cc8-94a7-897d1ad4c471-maas-api-service-ca" pod "121702e5-40be-4cc8-94a7-897d1ad4c471" (UID: "121702e5-40be-4cc8-94a7-897d1ad4c471"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:45:23.056352 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:23.056325    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/21d9b47a-e5e0-4bd7-9a9d-384e1be64ad9-maas-api-service-ca" pod "21d9b47a-e5e0-4bd7-9a9d-384e1be64ad9" (UID: "21d9b47a-e5e0-4bd7-9a9d-384e1be64ad9"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:45:23.056469 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:23.056451    4888 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/21d9b47a-e5e0-4bd7-9a9d-384e1be64ad9-maas-api-service-ca\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:45:23.056469 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:23.056469    4888 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/121702e5-40be-4cc8-94a7-897d1ad4c471-maas-api-service-ca\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:45:23.057787 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:23.057768    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-59144a1b-29889765-qxkk7"]
Oct 30 18:45:23.058455 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:23.058417    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/21d9b47a-e5e0-4bd7-9a9d-384e1be64ad9-kube-api-access-nmdwg" pod "21d9b47a-e5e0-4bd7-9a9d-384e1be64ad9" (UID: "21d9b47a-e5e0-4bd7-9a9d-384e1be64ad9"). InnerVolumeSpecName "kube-api-access-nmdwg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:45:23.058561 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:23.058434    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/121702e5-40be-4cc8-94a7-897d1ad4c471-kube-api-access-mp2nf" pod "121702e5-40be-4cc8-94a7-897d1ad4c471" (UID: "121702e5-40be-4cc8-94a7-897d1ad4c471"). InnerVolumeSpecName "kube-api-access-mp2nf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:45:23.061937 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:23.061915    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-59144a1b-29889765-qxkk7"]
Oct 30 18:45:23.157321 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:23.157233    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mp2nf\" (UniqueName: \"kubernetes.io/projected/121702e5-40be-4cc8-94a7-897d1ad4c471-kube-api-access-mp2nf\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:45:23.157321 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:23.157257    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nmdwg\" (UniqueName: \"kubernetes.io/projected/21d9b47a-e5e0-4bd7-9a9d-384e1be64ad9-kube-api-access-nmdwg\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:45:23.458917 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:23.458841    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-59144a1b-openshift-default-78d555d5bc-lk4t7"
Oct 30 18:45:23.460038 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:23.460014    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-59144a1b-openshift-default-78d555d5bc-lk4t7"
Oct 30 18:45:23.841586 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:23.841546    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-529e04e2-29889765-vhkgl" event={"ID":"121702e5-40be-4cc8-94a7-897d1ad4c471","Type":"ContainerDied","Data":"c184bcc397425cf9cde4e28d3e468f9b46aa2bbf963318196bc302317c9b09d4"}
Oct 30 18:45:23.841790 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:23.841621    4888 scope.go:122] "RemoveContainer" containerID="e44a4a522ca25ba1522c492fc726211f4860cc9b965977fd5ae446e25b7f21d7"
Oct 30 18:45:23.841790 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:23.841716    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-529e04e2-29889765-vhkgl"
Oct 30 18:45:23.860595 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:23.860270    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889765-qknhc"
Oct 30 18:45:23.864835 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:23.864803    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889765-qknhc" event={"ID":"21d9b47a-e5e0-4bd7-9a9d-384e1be64ad9","Type":"ContainerDied","Data":"ec2a496b125aa0d378833e51513d9ecc0fb1afb7c7fb92f5a1550ae82ee24c3e"}
Oct 30 18:45:23.882912 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:23.882796    4888 scope.go:122] "RemoveContainer" containerID="c9591eb22d47104f1f6b806e38605e2f382daed89b3a040b84db154393bbe8ac"
Oct 30 18:45:23.894655 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:23.894621    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4aa64e3a-64c9-4cb1-89a7-19dfd4aa6706" path="/var/lib/kubelet/pods/4aa64e3a-64c9-4cb1-89a7-19dfd4aa6706/volumes"
Oct 30 18:45:24.016541 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:24.016511    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889765-qknhc"]
Oct 30 18:45:24.020325 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:24.020303    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889765-qknhc"]
Oct 30 18:45:24.137900 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:24.137811    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-529e04e2-29889765-vhkgl"]
Oct 30 18:45:24.144190 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:24.144163    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-529e04e2-29889765-vhkgl"]
Oct 30 18:45:25.904528 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:25.904495    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="121702e5-40be-4cc8-94a7-897d1ad4c471" path="/var/lib/kubelet/pods/121702e5-40be-4cc8-94a7-897d1ad4c471/volumes"
Oct 30 18:45:25.905038 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:25.905023    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="21d9b47a-e5e0-4bd7-9a9d-384e1be64ad9" path="/var/lib/kubelet/pods/21d9b47a-e5e0-4bd7-9a9d-384e1be64ad9/volumes"
Oct 30 18:45:26.396057 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:26.396011    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-def78c01-wrnkf"]
Oct 30 18:45:26.396562 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:26.396547    4888 cpu_manager.go:414] "containerMap: removing container" podUID="4aa64e3a-64c9-4cb1-89a7-19dfd4aa6706" containerName="cleanup"
Oct 30 18:45:26.396562 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:26.396563    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4aa64e3a-64c9-4cb1-89a7-19dfd4aa6706" containerName="cleanup"
Oct 30 18:45:26.396680 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:26.396575    4888 cpu_manager.go:414] "containerMap: removing container" podUID="21d9b47a-e5e0-4bd7-9a9d-384e1be64ad9" containerName="cleanup"
Oct 30 18:45:26.396680 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:26.396579    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="21d9b47a-e5e0-4bd7-9a9d-384e1be64ad9" containerName="cleanup"
Oct 30 18:45:26.396680 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:26.396592    4888 cpu_manager.go:414] "containerMap: removing container" podUID="4aa64e3a-64c9-4cb1-89a7-19dfd4aa6706" containerName="cleanup"
Oct 30 18:45:26.396680 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:26.396598    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4aa64e3a-64c9-4cb1-89a7-19dfd4aa6706" containerName="cleanup"
Oct 30 18:45:26.396680 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:26.396605    4888 cpu_manager.go:414] "containerMap: removing container" podUID="121702e5-40be-4cc8-94a7-897d1ad4c471" containerName="cleanup"
Oct 30 18:45:26.396680 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:26.396609    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="121702e5-40be-4cc8-94a7-897d1ad4c471" containerName="cleanup"
Oct 30 18:45:26.396680 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:26.396622    4888 cpu_manager.go:414] "containerMap: removing container" podUID="121702e5-40be-4cc8-94a7-897d1ad4c471" containerName="cleanup"
Oct 30 18:45:26.396680 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:26.396626    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="121702e5-40be-4cc8-94a7-897d1ad4c471" containerName="cleanup"
Oct 30 18:45:26.396680 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:26.396632    4888 cpu_manager.go:414] "containerMap: removing container" podUID="121702e5-40be-4cc8-94a7-897d1ad4c471" containerName="cleanup"
Oct 30 18:45:26.396680 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:26.396636    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="121702e5-40be-4cc8-94a7-897d1ad4c471" containerName="cleanup"
Oct 30 18:45:26.396680 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:26.396642    4888 cpu_manager.go:414] "containerMap: removing container" podUID="21d9b47a-e5e0-4bd7-9a9d-384e1be64ad9" containerName="cleanup"
Oct 30 18:45:26.396680 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:26.396646    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="21d9b47a-e5e0-4bd7-9a9d-384e1be64ad9" containerName="cleanup"
Oct 30 18:45:26.397020 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:26.396731    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="4aa64e3a-64c9-4cb1-89a7-19dfd4aa6706" containerName="cleanup"
Oct 30 18:45:26.397020 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:26.396742    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="4aa64e3a-64c9-4cb1-89a7-19dfd4aa6706" containerName="cleanup"
Oct 30 18:45:26.397020 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:26.396748    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="21d9b47a-e5e0-4bd7-9a9d-384e1be64ad9" containerName="cleanup"
Oct 30 18:45:26.397020 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:26.396754    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="121702e5-40be-4cc8-94a7-897d1ad4c471" containerName="cleanup"
Oct 30 18:45:26.397020 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:26.396761    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="121702e5-40be-4cc8-94a7-897d1ad4c471" containerName="cleanup"
Oct 30 18:45:26.397020 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:26.396765    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="121702e5-40be-4cc8-94a7-897d1ad4c471" containerName="cleanup"
Oct 30 18:45:26.397020 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:26.396771    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="21d9b47a-e5e0-4bd7-9a9d-384e1be64ad9" containerName="cleanup"
Oct 30 18:45:26.399945 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:26.399928    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-def78c01-wrnkf"
Oct 30 18:45:26.458122 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:26.458100    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 18:45:26.458264 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:26.458153    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-2qg7n\""
Oct 30 18:45:26.511864 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:26.511829    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-def78c01-wrnkf"]
Oct 30 18:45:26.513778 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:26.513749    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tszvn\" (UniqueName: \"kubernetes.io/projected/d09017d2-91c5-4f27-a84c-b60db41e36ea-kube-api-access-tszvn\") pod \"maas-api-revoke-keys-e2e-ait-def78c01-wrnkf\" (UID: \"d09017d2-91c5-4f27-a84c-b60db41e36ea\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-def78c01-wrnkf"
Oct 30 18:45:26.513967 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:26.513945    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d09017d2-91c5-4f27-a84c-b60db41e36ea-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-ait-def78c01-wrnkf\" (UID: \"d09017d2-91c5-4f27-a84c-b60db41e36ea\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-def78c01-wrnkf"
Oct 30 18:45:26.514075 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:26.514055    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d09017d2-91c5-4f27-a84c-b60db41e36ea-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-ait-def78c01-wrnkf\" (UID: \"d09017d2-91c5-4f27-a84c-b60db41e36ea\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-def78c01-wrnkf"
Oct 30 18:45:26.615030 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:26.614998    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d09017d2-91c5-4f27-a84c-b60db41e36ea-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-ait-def78c01-wrnkf\" (UID: \"d09017d2-91c5-4f27-a84c-b60db41e36ea\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-def78c01-wrnkf"
Oct 30 18:45:26.615189 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:26.615050    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tszvn\" (UniqueName: \"kubernetes.io/projected/d09017d2-91c5-4f27-a84c-b60db41e36ea-kube-api-access-tszvn\") pod \"maas-api-revoke-keys-e2e-ait-def78c01-wrnkf\" (UID: \"d09017d2-91c5-4f27-a84c-b60db41e36ea\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-def78c01-wrnkf"
Oct 30 18:45:26.615189 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:26.615127    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d09017d2-91c5-4f27-a84c-b60db41e36ea-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-ait-def78c01-wrnkf\" (UID: \"d09017d2-91c5-4f27-a84c-b60db41e36ea\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-def78c01-wrnkf"
Oct 30 18:45:26.615728 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:26.615708    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d09017d2-91c5-4f27-a84c-b60db41e36ea-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-ait-def78c01-wrnkf\" (UID: \"d09017d2-91c5-4f27-a84c-b60db41e36ea\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-def78c01-wrnkf"
Oct 30 18:45:26.617543 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:26.617526    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d09017d2-91c5-4f27-a84c-b60db41e36ea-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-ait-def78c01-wrnkf\" (UID: \"d09017d2-91c5-4f27-a84c-b60db41e36ea\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-def78c01-wrnkf"
Oct 30 18:45:26.694210 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:26.694142    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tszvn\" (UniqueName: \"kubernetes.io/projected/d09017d2-91c5-4f27-a84c-b60db41e36ea-kube-api-access-tszvn\") pod \"maas-api-revoke-keys-e2e-ait-def78c01-wrnkf\" (UID: \"d09017d2-91c5-4f27-a84c-b60db41e36ea\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-def78c01-wrnkf"
Oct 30 18:45:26.711849 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:26.711822    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-def78c01-wrnkf"
Oct 30 18:45:27.114442 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:27.114415    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-def78c01-wrnkf"]
Oct 30 18:45:27.115496 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:45:27.115472    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd09017d2_91c5_4f27_a84c_b60db41e36ea.slice/crio-4b13ae7090773749763919bfc447800e3065f9a7c373b9ba0360844cc996b3d8 WatchSource:0}: Error finding container 4b13ae7090773749763919bfc447800e3065f9a7c373b9ba0360844cc996b3d8: Status 404 returned error can't find the container with id 4b13ae7090773749763919bfc447800e3065f9a7c373b9ba0360844cc996b3d8
Oct 30 18:45:27.914518 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:27.914471    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-def78c01-wrnkf" event={"ID":"d09017d2-91c5-4f27-a84c-b60db41e36ea","Type":"ContainerStarted","Data":"118ed213b2333e41de22c2637eb89e67f7cc74ce5580bd5f03837b5939a0ced9"}
Oct 30 18:45:27.914703 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:27.914531    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-def78c01-wrnkf" event={"ID":"d09017d2-91c5-4f27-a84c-b60db41e36ea","Type":"ContainerStarted","Data":"4b13ae7090773749763919bfc447800e3065f9a7c373b9ba0360844cc996b3d8"}
Oct 30 18:45:28.047118 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:28.047073    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-def78c01-wrnkf" podStartSLOduration=2.047059231 podStartE2EDuration="2.047059231s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:45:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:45:28.04521471 +0000 UTC m=+692.292620736" watchObservedRunningTime="2026-10-30 18:45:28.047059231 +0000 UTC m=+692.294465256"
Oct 30 18:45:28.921939 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:28.921906    4888 generic.go:356] "Generic (PLEG): container finished" podID="d09017d2-91c5-4f27-a84c-b60db41e36ea" containerID="118ed213b2333e41de22c2637eb89e67f7cc74ce5580bd5f03837b5939a0ced9" exitCode=0
Oct 30 18:45:28.922368 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:28.921991    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-def78c01-wrnkf" event={"ID":"d09017d2-91c5-4f27-a84c-b60db41e36ea","Type":"ContainerDied","Data":"118ed213b2333e41de22c2637eb89e67f7cc74ce5580bd5f03837b5939a0ced9"}
Oct 30 18:45:30.110427 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:30.110402    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-def78c01-wrnkf"
Oct 30 18:45:30.155218 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:30.155189    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d09017d2-91c5-4f27-a84c-b60db41e36ea-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d09017d2-91c5-4f27-a84c-b60db41e36ea-maas-api-openshift-ca\") pod \"d09017d2-91c5-4f27-a84c-b60db41e36ea\" (UID: \"d09017d2-91c5-4f27-a84c-b60db41e36ea\") "
Oct 30 18:45:30.155418 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:30.155340    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d09017d2-91c5-4f27-a84c-b60db41e36ea-kube-api-access-tszvn\" (UniqueName: \"kubernetes.io/projected/d09017d2-91c5-4f27-a84c-b60db41e36ea-kube-api-access-tszvn\") pod \"d09017d2-91c5-4f27-a84c-b60db41e36ea\" (UID: \"d09017d2-91c5-4f27-a84c-b60db41e36ea\") "
Oct 30 18:45:30.155418 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:30.155366    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d09017d2-91c5-4f27-a84c-b60db41e36ea-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d09017d2-91c5-4f27-a84c-b60db41e36ea-maas-api-xks-ca\") pod \"d09017d2-91c5-4f27-a84c-b60db41e36ea\" (UID: \"d09017d2-91c5-4f27-a84c-b60db41e36ea\") "
Oct 30 18:45:30.155590 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:30.155561    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d09017d2-91c5-4f27-a84c-b60db41e36ea-maas-api-openshift-ca" pod "d09017d2-91c5-4f27-a84c-b60db41e36ea" (UID: "d09017d2-91c5-4f27-a84c-b60db41e36ea"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:45:30.155817 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:30.155798    4888 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d09017d2-91c5-4f27-a84c-b60db41e36ea-maas-api-openshift-ca\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:45:30.158013 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:30.157984    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d09017d2-91c5-4f27-a84c-b60db41e36ea-maas-api-xks-ca" pod "d09017d2-91c5-4f27-a84c-b60db41e36ea" (UID: "d09017d2-91c5-4f27-a84c-b60db41e36ea"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:45:30.158013 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:30.158000    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d09017d2-91c5-4f27-a84c-b60db41e36ea-kube-api-access-tszvn" pod "d09017d2-91c5-4f27-a84c-b60db41e36ea" (UID: "d09017d2-91c5-4f27-a84c-b60db41e36ea"). InnerVolumeSpecName "kube-api-access-tszvn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:45:30.257136 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:30.257034    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tszvn\" (UniqueName: \"kubernetes.io/projected/d09017d2-91c5-4f27-a84c-b60db41e36ea-kube-api-access-tszvn\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:45:30.257136 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:30.257074    4888 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d09017d2-91c5-4f27-a84c-b60db41e36ea-maas-api-xks-ca\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:45:30.938619 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:30.938577    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-def78c01-wrnkf"
Oct 30 18:45:30.938837 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:30.938629    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-def78c01-wrnkf" event={"ID":"d09017d2-91c5-4f27-a84c-b60db41e36ea","Type":"ContainerDied","Data":"4b13ae7090773749763919bfc447800e3065f9a7c373b9ba0360844cc996b3d8"}
Oct 30 18:45:30.938837 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:30.938693    4888 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4b13ae7090773749763919bfc447800e3065f9a7c373b9ba0360844cc996b3d8"
Oct 30 18:45:32.952737 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:32.952706    4888 generic.go:356] "Generic (PLEG): container finished" podID="87986146-ca55-47e6-8f17-9df4ed880eeb" containerID="30a2f64a02963e4beb48c6fdb8f56c71d76426dd791eb38175e528b4f2c91c2e" exitCode=0
Oct 30 18:45:32.953162 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:32.952776    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-def78c01-openshift-default-59959c464f-6qpnd" event={"ID":"87986146-ca55-47e6-8f17-9df4ed880eeb","Type":"ContainerDied","Data":"30a2f64a02963e4beb48c6fdb8f56c71d76426dd791eb38175e528b4f2c91c2e"}
Oct 30 18:45:32.953162 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:32.953118    4888 scope.go:122] "RemoveContainer" containerID="30a2f64a02963e4beb48c6fdb8f56c71d76426dd791eb38175e528b4f2c91c2e"
Oct 30 18:45:32.954372 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:32.954344    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:45:32.954474 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:32.954410    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:45:32.954474 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:32.954447    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:45:33.989453 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:33.989414    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-def78c01-openshift-default-59959c464f-6qpnd" event={"ID":"87986146-ca55-47e6-8f17-9df4ed880eeb","Type":"ContainerStarted","Data":"27c14b9c7a5c30a371888884f97f9d5eafae1d6dcccf733f2dbd8b15779c1d6d"}
Oct 30 18:45:34.755313 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:34.755256    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-def78c01-openshift-default-59959c464f-6qpnd"
Oct 30 18:45:34.756510 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:34.756482    4888 patch_prober.go:29] interesting pod/e2e-ait-def78c01-openshift-default-59959c464f-6qpnd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body=
Oct 30 18:45:34.756627 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:34.756532    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-def78c01-openshift-default-59959c464f-6qpnd" podUID="87986146-ca55-47e6-8f17-9df4ed880eeb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused"
Oct 30 18:45:35.622737 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:45:35.622704    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-jbsw6" containerName="extractor"
Oct 30 18:45:35.754963 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:35.754916    4888 patch_prober.go:29] interesting pod/e2e-ait-def78c01-openshift-default-59959c464f-6qpnd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body=
Oct 30 18:45:35.755152 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:35.754991    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-def78c01-openshift-default-59959c464f-6qpnd" podUID="87986146-ca55-47e6-8f17-9df4ed880eeb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused"
Oct 30 18:45:36.754630 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:36.754595    4888 patch_prober.go:29] interesting pod/e2e-ait-def78c01-openshift-default-59959c464f-6qpnd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body=
Oct 30 18:45:36.755071 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:36.754663    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-def78c01-openshift-default-59959c464f-6qpnd" podUID="87986146-ca55-47e6-8f17-9df4ed880eeb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused"
Oct 30 18:45:37.755367 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:37.755334    4888 patch_prober.go:29] interesting pod/e2e-ait-def78c01-openshift-default-59959c464f-6qpnd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body=
Oct 30 18:45:37.755783 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:37.755397    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-def78c01-openshift-default-59959c464f-6qpnd" podUID="87986146-ca55-47e6-8f17-9df4ed880eeb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused"
Oct 30 18:45:37.961179 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:37.961146    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-16a49490-5d9d5546d8-dgjqr"]
Oct 30 18:45:38.094873 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:38.094841    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-16a49490-779c98d64c-m8q2t"]
Oct 30 18:45:38.755130 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:38.755100    4888 patch_prober.go:29] interesting pod/e2e-ait-def78c01-openshift-default-59959c464f-6qpnd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body=
Oct 30 18:45:38.755321 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:38.755178    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-def78c01-openshift-default-59959c464f-6qpnd" podUID="87986146-ca55-47e6-8f17-9df4ed880eeb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused"
Oct 30 18:45:39.391801 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:39.391772    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-def78c01-openshift-default-59959c464f-6qpnd"
Oct 30 18:45:39.766058 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:39.765979    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-def78c01-openshift-default-59959c464f-6qpnd"
Oct 30 18:45:40.032508 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:40.032422    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-def78c01-openshift-default-59959c464f-6qpnd"
Oct 30 18:45:40.581785 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:40.581750    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-def78c01-6994fdb9d-kxfn6"]
Oct 30 18:45:40.691372 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:40.691336    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-def78c01-855988f75-dln2g"]
Oct 30 18:45:42.962049 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:42.962009    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-migrate-16a49490-5d9d5546d8-dgjqr" podUID="fcd5e44c-96f2-44e7-baea-b55634db50e6" containerName="payload-processing" containerID="cri-o://89336fe57d09afcfbf39f71718770360ab096c7a241b1955d0be1227dd1aeb01" gracePeriod=25
Oct 30 18:45:43.096060 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:43.096008    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-migrate-16a49490-779c98d64c-m8q2t" podUID="c7f9892e-42f6-4674-8a84-c1a4d2049807" containerName="payload-pre-processing" containerID="cri-o://dd2dd56b97aa944cbc505e79eda49819465492504f4f3242ffd3bd091956db06" gracePeriod=25
Oct 30 18:45:43.222392 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:43.222363    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-16a49490-5d9d5546d8-dgjqr"
Oct 30 18:45:43.299858 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:43.299825    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fcd5e44c-96f2-44e7-baea-b55634db50e6-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fcd5e44c-96f2-44e7-baea-b55634db50e6-plugins-config-volume\") pod \"fcd5e44c-96f2-44e7-baea-b55634db50e6\" (UID: \"fcd5e44c-96f2-44e7-baea-b55634db50e6\") "
Oct 30 18:45:43.300009 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:43.299874    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fcd5e44c-96f2-44e7-baea-b55634db50e6-kube-api-access-pp6pb\" (UniqueName: \"kubernetes.io/projected/fcd5e44c-96f2-44e7-baea-b55634db50e6-kube-api-access-pp6pb\") pod \"fcd5e44c-96f2-44e7-baea-b55634db50e6\" (UID: \"fcd5e44c-96f2-44e7-baea-b55634db50e6\") "
Oct 30 18:45:43.300423 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:43.300395    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fcd5e44c-96f2-44e7-baea-b55634db50e6-plugins-config-volume" pod "fcd5e44c-96f2-44e7-baea-b55634db50e6" (UID: "fcd5e44c-96f2-44e7-baea-b55634db50e6"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:45:43.302723 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:43.302692    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fcd5e44c-96f2-44e7-baea-b55634db50e6-kube-api-access-pp6pb" pod "fcd5e44c-96f2-44e7-baea-b55634db50e6" (UID: "fcd5e44c-96f2-44e7-baea-b55634db50e6"). InnerVolumeSpecName "kube-api-access-pp6pb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:45:43.344258 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:43.344233    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-16a49490-779c98d64c-m8q2t"
Oct 30 18:45:43.401087 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:43.401056    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c7f9892e-42f6-4674-8a84-c1a4d2049807-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c7f9892e-42f6-4674-8a84-c1a4d2049807-plugins-config-volume\") pod \"c7f9892e-42f6-4674-8a84-c1a4d2049807\" (UID: \"c7f9892e-42f6-4674-8a84-c1a4d2049807\") "
Oct 30 18:45:43.401269 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:43.401151    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c7f9892e-42f6-4674-8a84-c1a4d2049807-kube-api-access-w8h4n\" (UniqueName: \"kubernetes.io/projected/c7f9892e-42f6-4674-8a84-c1a4d2049807-kube-api-access-w8h4n\") pod \"c7f9892e-42f6-4674-8a84-c1a4d2049807\" (UID: \"c7f9892e-42f6-4674-8a84-c1a4d2049807\") "
Oct 30 18:45:43.401450 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:43.401425    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c7f9892e-42f6-4674-8a84-c1a4d2049807-plugins-config-volume" pod "c7f9892e-42f6-4674-8a84-c1a4d2049807" (UID: "c7f9892e-42f6-4674-8a84-c1a4d2049807"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:45:43.401521 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:43.401503    4888 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fcd5e44c-96f2-44e7-baea-b55634db50e6-plugins-config-volume\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:45:43.401521 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:43.401518    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pp6pb\" (UniqueName: \"kubernetes.io/projected/fcd5e44c-96f2-44e7-baea-b55634db50e6-kube-api-access-pp6pb\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:45:43.403481 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:43.403459    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c7f9892e-42f6-4674-8a84-c1a4d2049807-kube-api-access-w8h4n" pod "c7f9892e-42f6-4674-8a84-c1a4d2049807" (UID: "c7f9892e-42f6-4674-8a84-c1a4d2049807"). InnerVolumeSpecName "kube-api-access-w8h4n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:45:43.502063 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:43.501990    4888 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c7f9892e-42f6-4674-8a84-c1a4d2049807-plugins-config-volume\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:45:43.502063 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:43.502018    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w8h4n\" (UniqueName: \"kubernetes.io/projected/c7f9892e-42f6-4674-8a84-c1a4d2049807-kube-api-access-w8h4n\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:45:44.056252 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:44.056218    4888 generic.go:356] "Generic (PLEG): container finished" podID="c7f9892e-42f6-4674-8a84-c1a4d2049807" containerID="dd2dd56b97aa944cbc505e79eda49819465492504f4f3242ffd3bd091956db06" exitCode=0
Oct 30 18:45:44.056854 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:44.056296    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-16a49490-779c98d64c-m8q2t"
Oct 30 18:45:44.056854 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:44.056312    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-16a49490-779c98d64c-m8q2t" event={"ID":"c7f9892e-42f6-4674-8a84-c1a4d2049807","Type":"ContainerDied","Data":"dd2dd56b97aa944cbc505e79eda49819465492504f4f3242ffd3bd091956db06"}
Oct 30 18:45:44.056854 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:44.056374    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-16a49490-779c98d64c-m8q2t" event={"ID":"c7f9892e-42f6-4674-8a84-c1a4d2049807","Type":"ContainerDied","Data":"7ecd54f42a1a9c7895b22031b99a98ce1af9572c2bf999fccd57c2286bed85b6"}
Oct 30 18:45:44.056854 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:44.056409    4888 scope.go:122] "RemoveContainer" containerID="dd2dd56b97aa944cbc505e79eda49819465492504f4f3242ffd3bd091956db06"
Oct 30 18:45:44.057908 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:44.057852    4888 generic.go:356] "Generic (PLEG): container finished" podID="fcd5e44c-96f2-44e7-baea-b55634db50e6" containerID="89336fe57d09afcfbf39f71718770360ab096c7a241b1955d0be1227dd1aeb01" exitCode=0
Oct 30 18:45:44.057908 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:44.057887    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-16a49490-5d9d5546d8-dgjqr" event={"ID":"fcd5e44c-96f2-44e7-baea-b55634db50e6","Type":"ContainerDied","Data":"89336fe57d09afcfbf39f71718770360ab096c7a241b1955d0be1227dd1aeb01"}
Oct 30 18:45:44.058001 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:44.057924    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-16a49490-5d9d5546d8-dgjqr" event={"ID":"fcd5e44c-96f2-44e7-baea-b55634db50e6","Type":"ContainerDied","Data":"e75544568f3ba9f3a0b1a398d2de5dc68022bc26f06b470851f7e599a656ba45"}
Oct 30 18:45:44.058001 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:44.057941    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-16a49490-5d9d5546d8-dgjqr"
Oct 30 18:45:44.066688 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:44.066668    4888 scope.go:122] "RemoveContainer" containerID="dd2dd56b97aa944cbc505e79eda49819465492504f4f3242ffd3bd091956db06"
Oct 30 18:45:44.066988 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:45:44.066958    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dd2dd56b97aa944cbc505e79eda49819465492504f4f3242ffd3bd091956db06\": container with ID starting with dd2dd56b97aa944cbc505e79eda49819465492504f4f3242ffd3bd091956db06 not found: ID does not exist" containerID="dd2dd56b97aa944cbc505e79eda49819465492504f4f3242ffd3bd091956db06"
Oct 30 18:45:44.067101 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:44.066995    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dd2dd56b97aa944cbc505e79eda49819465492504f4f3242ffd3bd091956db06"} err="failed to get container status \"dd2dd56b97aa944cbc505e79eda49819465492504f4f3242ffd3bd091956db06\": rpc error: code = NotFound desc = could not find container \"dd2dd56b97aa944cbc505e79eda49819465492504f4f3242ffd3bd091956db06\": container with ID starting with dd2dd56b97aa944cbc505e79eda49819465492504f4f3242ffd3bd091956db06 not found: ID does not exist"
Oct 30 18:45:44.067101 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:44.067013    4888 scope.go:122] "RemoveContainer" containerID="89336fe57d09afcfbf39f71718770360ab096c7a241b1955d0be1227dd1aeb01"
Oct 30 18:45:44.075488 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:44.075470    4888 scope.go:122] "RemoveContainer" containerID="89336fe57d09afcfbf39f71718770360ab096c7a241b1955d0be1227dd1aeb01"
Oct 30 18:45:44.075743 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:45:44.075726    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"89336fe57d09afcfbf39f71718770360ab096c7a241b1955d0be1227dd1aeb01\": container with ID starting with 89336fe57d09afcfbf39f71718770360ab096c7a241b1955d0be1227dd1aeb01 not found: ID does not exist" containerID="89336fe57d09afcfbf39f71718770360ab096c7a241b1955d0be1227dd1aeb01"
Oct 30 18:45:44.075789 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:44.075754    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"89336fe57d09afcfbf39f71718770360ab096c7a241b1955d0be1227dd1aeb01"} err="failed to get container status \"89336fe57d09afcfbf39f71718770360ab096c7a241b1955d0be1227dd1aeb01\": rpc error: code = NotFound desc = could not find container \"89336fe57d09afcfbf39f71718770360ab096c7a241b1955d0be1227dd1aeb01\": container with ID starting with 89336fe57d09afcfbf39f71718770360ab096c7a241b1955d0be1227dd1aeb01 not found: ID does not exist"
Oct 30 18:45:44.224367 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:44.224272    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-16a49490-5d9d5546d8-dgjqr"]
Oct 30 18:45:44.275218 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:44.275186    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-16a49490-5d9d5546d8-dgjqr"]
Oct 30 18:45:44.387151 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:44.387120    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-16a49490-779c98d64c-m8q2t"]
Oct 30 18:45:44.393051 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:44.393026    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-16a49490-779c98d64c-m8q2t"]
Oct 30 18:45:45.582859 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:45.582819    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-def78c01-6994fdb9d-kxfn6" podUID="82903e3e-982c-4708-a10c-8328e324f7b6" containerName="payload-processing" containerID="cri-o://7a37934ea4e9b3d755990e8d6455612ba077aca80631c205bd5f7a17dac7785a" gracePeriod=25
Oct 30 18:45:45.693451 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:45.693403    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-def78c01-855988f75-dln2g" podUID="9c207780-577e-483c-b70c-ff0e44e2dc0f" containerName="payload-pre-processing" containerID="cri-o://3a280c94f8e2a5bfd41cb3e4fc14931ae808e19e5248e13ef2ea354ab2bb0e3f" gracePeriod=25
Oct 30 18:45:45.842334 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:45.842238    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-def78c01-6994fdb9d-kxfn6"
Oct 30 18:45:45.891340 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:45.891299    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c7f9892e-42f6-4674-8a84-c1a4d2049807" path="/var/lib/kubelet/pods/c7f9892e-42f6-4674-8a84-c1a4d2049807/volumes"
Oct 30 18:45:45.891755 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:45.891738    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fcd5e44c-96f2-44e7-baea-b55634db50e6" path="/var/lib/kubelet/pods/fcd5e44c-96f2-44e7-baea-b55634db50e6/volumes"
Oct 30 18:45:45.926697 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:45.926668    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/82903e3e-982c-4708-a10c-8328e324f7b6-kube-api-access-hh6zt\" (UniqueName: \"kubernetes.io/projected/82903e3e-982c-4708-a10c-8328e324f7b6-kube-api-access-hh6zt\") pod \"82903e3e-982c-4708-a10c-8328e324f7b6\" (UID: \"82903e3e-982c-4708-a10c-8328e324f7b6\") "
Oct 30 18:45:45.926882 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:45.926827    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/82903e3e-982c-4708-a10c-8328e324f7b6-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/82903e3e-982c-4708-a10c-8328e324f7b6-plugins-config-volume\") pod \"82903e3e-982c-4708-a10c-8328e324f7b6\" (UID: \"82903e3e-982c-4708-a10c-8328e324f7b6\") "
Oct 30 18:45:45.927157 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:45.927131    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/82903e3e-982c-4708-a10c-8328e324f7b6-plugins-config-volume" pod "82903e3e-982c-4708-a10c-8328e324f7b6" (UID: "82903e3e-982c-4708-a10c-8328e324f7b6"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:45:45.928943 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:45.928919    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/82903e3e-982c-4708-a10c-8328e324f7b6-kube-api-access-hh6zt" pod "82903e3e-982c-4708-a10c-8328e324f7b6" (UID: "82903e3e-982c-4708-a10c-8328e324f7b6"). InnerVolumeSpecName "kube-api-access-hh6zt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:45:45.949778 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:45.949758    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-def78c01-855988f75-dln2g"
Oct 30 18:45:46.027789 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:46.027763    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9c207780-577e-483c-b70c-ff0e44e2dc0f-kube-api-access-5jpfb\" (UniqueName: \"kubernetes.io/projected/9c207780-577e-483c-b70c-ff0e44e2dc0f-kube-api-access-5jpfb\") pod \"9c207780-577e-483c-b70c-ff0e44e2dc0f\" (UID: \"9c207780-577e-483c-b70c-ff0e44e2dc0f\") "
Oct 30 18:45:46.027956 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:46.027801    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9c207780-577e-483c-b70c-ff0e44e2dc0f-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9c207780-577e-483c-b70c-ff0e44e2dc0f-plugins-config-volume\") pod \"9c207780-577e-483c-b70c-ff0e44e2dc0f\" (UID: \"9c207780-577e-483c-b70c-ff0e44e2dc0f\") "
Oct 30 18:45:46.028143 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:46.028120    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hh6zt\" (UniqueName: \"kubernetes.io/projected/82903e3e-982c-4708-a10c-8328e324f7b6-kube-api-access-hh6zt\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:45:46.028143 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:46.028138    4888 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/82903e3e-982c-4708-a10c-8328e324f7b6-plugins-config-volume\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:45:46.028321 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:46.028212    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9c207780-577e-483c-b70c-ff0e44e2dc0f-plugins-config-volume" pod "9c207780-577e-483c-b70c-ff0e44e2dc0f" (UID: "9c207780-577e-483c-b70c-ff0e44e2dc0f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:45:46.030039 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:46.030020    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9c207780-577e-483c-b70c-ff0e44e2dc0f-kube-api-access-5jpfb" pod "9c207780-577e-483c-b70c-ff0e44e2dc0f" (UID: "9c207780-577e-483c-b70c-ff0e44e2dc0f"). InnerVolumeSpecName "kube-api-access-5jpfb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:45:46.077215 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:46.077181    4888 generic.go:356] "Generic (PLEG): container finished" podID="9c207780-577e-483c-b70c-ff0e44e2dc0f" containerID="3a280c94f8e2a5bfd41cb3e4fc14931ae808e19e5248e13ef2ea354ab2bb0e3f" exitCode=0
Oct 30 18:45:46.077417 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:46.077239    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-def78c01-855988f75-dln2g"
Oct 30 18:45:46.077417 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:46.077277    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-def78c01-855988f75-dln2g" event={"ID":"9c207780-577e-483c-b70c-ff0e44e2dc0f","Type":"ContainerDied","Data":"3a280c94f8e2a5bfd41cb3e4fc14931ae808e19e5248e13ef2ea354ab2bb0e3f"}
Oct 30 18:45:46.077417 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:46.077369    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-def78c01-855988f75-dln2g" event={"ID":"9c207780-577e-483c-b70c-ff0e44e2dc0f","Type":"ContainerDied","Data":"870752801e93969a0ee929e70e71238dc7623ae0ab0bdb9fd0239eb704b616df"}
Oct 30 18:45:46.077417 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:46.077400    4888 scope.go:122] "RemoveContainer" containerID="3a280c94f8e2a5bfd41cb3e4fc14931ae808e19e5248e13ef2ea354ab2bb0e3f"
Oct 30 18:45:46.090104 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:46.086687    4888 generic.go:356] "Generic (PLEG): container finished" podID="82903e3e-982c-4708-a10c-8328e324f7b6" containerID="7a37934ea4e9b3d755990e8d6455612ba077aca80631c205bd5f7a17dac7785a" exitCode=0
Oct 30 18:45:46.090104 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:46.086878    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-def78c01-6994fdb9d-kxfn6" event={"ID":"82903e3e-982c-4708-a10c-8328e324f7b6","Type":"ContainerDied","Data":"7a37934ea4e9b3d755990e8d6455612ba077aca80631c205bd5f7a17dac7785a"}
Oct 30 18:45:46.090104 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:46.086962    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-def78c01-6994fdb9d-kxfn6" event={"ID":"82903e3e-982c-4708-a10c-8328e324f7b6","Type":"ContainerDied","Data":"660e966cd64769e9edffd1d991c00c1530c5ac79b8f168d4b6378b57f8ca2b5f"}
Oct 30 18:45:46.090104 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:46.087044    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-def78c01-6994fdb9d-kxfn6"
Oct 30 18:45:46.093555 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:46.092109    4888 scope.go:122] "RemoveContainer" containerID="3a280c94f8e2a5bfd41cb3e4fc14931ae808e19e5248e13ef2ea354ab2bb0e3f"
Oct 30 18:45:46.093555 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:45:46.092513    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3a280c94f8e2a5bfd41cb3e4fc14931ae808e19e5248e13ef2ea354ab2bb0e3f\": container with ID starting with 3a280c94f8e2a5bfd41cb3e4fc14931ae808e19e5248e13ef2ea354ab2bb0e3f not found: ID does not exist" containerID="3a280c94f8e2a5bfd41cb3e4fc14931ae808e19e5248e13ef2ea354ab2bb0e3f"
Oct 30 18:45:46.093555 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:46.092546    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3a280c94f8e2a5bfd41cb3e4fc14931ae808e19e5248e13ef2ea354ab2bb0e3f"} err="failed to get container status \"3a280c94f8e2a5bfd41cb3e4fc14931ae808e19e5248e13ef2ea354ab2bb0e3f\": rpc error: code = NotFound desc = could not find container \"3a280c94f8e2a5bfd41cb3e4fc14931ae808e19e5248e13ef2ea354ab2bb0e3f\": container with ID starting with 3a280c94f8e2a5bfd41cb3e4fc14931ae808e19e5248e13ef2ea354ab2bb0e3f not found: ID does not exist"
Oct 30 18:45:46.093555 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:46.092563    4888 scope.go:122] "RemoveContainer" containerID="7a37934ea4e9b3d755990e8d6455612ba077aca80631c205bd5f7a17dac7785a"
Oct 30 18:45:46.105897 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:46.105875    4888 scope.go:122] "RemoveContainer" containerID="7a37934ea4e9b3d755990e8d6455612ba077aca80631c205bd5f7a17dac7785a"
Oct 30 18:45:46.106249 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:45:46.106228    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7a37934ea4e9b3d755990e8d6455612ba077aca80631c205bd5f7a17dac7785a\": container with ID starting with 7a37934ea4e9b3d755990e8d6455612ba077aca80631c205bd5f7a17dac7785a not found: ID does not exist" containerID="7a37934ea4e9b3d755990e8d6455612ba077aca80631c205bd5f7a17dac7785a"
Oct 30 18:45:46.106364 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:46.106259    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7a37934ea4e9b3d755990e8d6455612ba077aca80631c205bd5f7a17dac7785a"} err="failed to get container status \"7a37934ea4e9b3d755990e8d6455612ba077aca80631c205bd5f7a17dac7785a\": rpc error: code = NotFound desc = could not find container \"7a37934ea4e9b3d755990e8d6455612ba077aca80631c205bd5f7a17dac7785a\": container with ID starting with 7a37934ea4e9b3d755990e8d6455612ba077aca80631c205bd5f7a17dac7785a not found: ID does not exist"
Oct 30 18:45:46.129246 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:46.129224    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5jpfb\" (UniqueName: \"kubernetes.io/projected/9c207780-577e-483c-b70c-ff0e44e2dc0f-kube-api-access-5jpfb\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:45:46.129358 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:46.129250    4888 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9c207780-577e-483c-b70c-ff0e44e2dc0f-plugins-config-volume\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:45:46.260568 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:46.260537    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-def78c01-855988f75-dln2g"]
Oct 30 18:45:46.263907 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:46.263879    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-def78c01-855988f75-dln2g"]
Oct 30 18:45:46.376224 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:46.376143    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-def78c01-6994fdb9d-kxfn6"]
Oct 30 18:45:46.378579 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:46.378558    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-def78c01-6994fdb9d-kxfn6"]
Oct 30 18:45:47.367725 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:47.367692    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-59144a1b-569dd9dbd8-45qd4"]
Oct 30 18:45:47.499513 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:47.499480    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-59144a1b-7d9455dd67-dmj9h"]
Oct 30 18:45:47.889965 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:47.889935    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="82903e3e-982c-4708-a10c-8328e324f7b6" path="/var/lib/kubelet/pods/82903e3e-982c-4708-a10c-8328e324f7b6/volumes"
Oct 30 18:45:47.890322 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:47.890309    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9c207780-577e-483c-b70c-ff0e44e2dc0f" path="/var/lib/kubelet/pods/9c207780-577e-483c-b70c-ff0e44e2dc0f/volumes"
Oct 30 18:45:48.473151 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:48.473113    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-def78c01-wrnkf"]
Oct 30 18:45:48.479951 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:48.479926    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-def78c01-wrnkf"]
Oct 30 18:45:49.069859 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:49.069824    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-def78c01-openshift-default-59959c464f-6qpnd"]
Oct 30 18:45:49.070104 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:49.070078    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-def78c01-openshift-default-59959c464f-6qpnd" podUID="87986146-ca55-47e6-8f17-9df4ed880eeb" containerName="istio-proxy" containerID="cri-o://27c14b9c7a5c30a371888884f97f9d5eafae1d6dcccf733f2dbd8b15779c1d6d" gracePeriod=30
Oct 30 18:45:49.890161 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:49.890129    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d09017d2-91c5-4f27-a84c-b60db41e36ea" path="/var/lib/kubelet/pods/d09017d2-91c5-4f27-a84c-b60db41e36ea/volumes"
Oct 30 18:45:50.978313 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:50.978263    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-529e04e2-d954bc5c5-m5jvj"]
Oct 30 18:45:51.985781 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:51.985753    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-rjmdz"]
Oct 30 18:45:52.368005 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:52.367966    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-59144a1b-569dd9dbd8-45qd4" podUID="e1cf8b12-1a2a-4935-8c3a-97bfe3281cb6" containerName="payload-processing" containerID="cri-o://09e4971cdd8b96300a148b7b0276e7be874f6966cd3826671c7cb9424a60120a" gracePeriod=25
Oct 30 18:45:52.499945 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:52.499904    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-59144a1b-7d9455dd67-dmj9h" podUID="230b68f1-1eeb-4db0-a160-9248f8314240" containerName="payload-pre-processing" containerID="cri-o://11d8ab27b4ddc1570bc79f437c4c9b54b453f5a7599f917bb9df3731fc8f4006" gracePeriod=25
Oct 30 18:45:52.646679 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:52.646651    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-59144a1b-569dd9dbd8-45qd4"
Oct 30 18:45:52.721770 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:52.721728    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e1cf8b12-1a2a-4935-8c3a-97bfe3281cb6-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e1cf8b12-1a2a-4935-8c3a-97bfe3281cb6-plugins-config-volume\") pod \"e1cf8b12-1a2a-4935-8c3a-97bfe3281cb6\" (UID: \"e1cf8b12-1a2a-4935-8c3a-97bfe3281cb6\") "
Oct 30 18:45:52.721939 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:52.721915    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e1cf8b12-1a2a-4935-8c3a-97bfe3281cb6-kube-api-access-qxwjw\" (UniqueName: \"kubernetes.io/projected/e1cf8b12-1a2a-4935-8c3a-97bfe3281cb6-kube-api-access-qxwjw\") pod \"e1cf8b12-1a2a-4935-8c3a-97bfe3281cb6\" (UID: \"e1cf8b12-1a2a-4935-8c3a-97bfe3281cb6\") "
Oct 30 18:45:52.722165 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:52.722139    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e1cf8b12-1a2a-4935-8c3a-97bfe3281cb6-plugins-config-volume" pod "e1cf8b12-1a2a-4935-8c3a-97bfe3281cb6" (UID: "e1cf8b12-1a2a-4935-8c3a-97bfe3281cb6"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:45:52.722299 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:52.722262    4888 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e1cf8b12-1a2a-4935-8c3a-97bfe3281cb6-plugins-config-volume\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:45:52.727047 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:52.727001    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e1cf8b12-1a2a-4935-8c3a-97bfe3281cb6-kube-api-access-qxwjw" pod "e1cf8b12-1a2a-4935-8c3a-97bfe3281cb6" (UID: "e1cf8b12-1a2a-4935-8c3a-97bfe3281cb6"). InnerVolumeSpecName "kube-api-access-qxwjw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:45:52.769444 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:52.769416    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-59144a1b-7d9455dd67-dmj9h"
Oct 30 18:45:52.823191 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:52.823152    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qxwjw\" (UniqueName: \"kubernetes.io/projected/e1cf8b12-1a2a-4935-8c3a-97bfe3281cb6-kube-api-access-qxwjw\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:45:52.924610 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:52.924520    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/230b68f1-1eeb-4db0-a160-9248f8314240-kube-api-access-8g5nx\" (UniqueName: \"kubernetes.io/projected/230b68f1-1eeb-4db0-a160-9248f8314240-kube-api-access-8g5nx\") pod \"230b68f1-1eeb-4db0-a160-9248f8314240\" (UID: \"230b68f1-1eeb-4db0-a160-9248f8314240\") "
Oct 30 18:45:52.925469 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:52.925441    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/230b68f1-1eeb-4db0-a160-9248f8314240-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/230b68f1-1eeb-4db0-a160-9248f8314240-plugins-config-volume\") pod \"230b68f1-1eeb-4db0-a160-9248f8314240\" (UID: \"230b68f1-1eeb-4db0-a160-9248f8314240\") "
Oct 30 18:45:52.925919 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:52.925891    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/230b68f1-1eeb-4db0-a160-9248f8314240-plugins-config-volume" pod "230b68f1-1eeb-4db0-a160-9248f8314240" (UID: "230b68f1-1eeb-4db0-a160-9248f8314240"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:45:52.928191 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:52.928162    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/230b68f1-1eeb-4db0-a160-9248f8314240-kube-api-access-8g5nx" pod "230b68f1-1eeb-4db0-a160-9248f8314240" (UID: "230b68f1-1eeb-4db0-a160-9248f8314240"). InnerVolumeSpecName "kube-api-access-8g5nx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:45:53.026950 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:53.026917    4888 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/230b68f1-1eeb-4db0-a160-9248f8314240-plugins-config-volume\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:45:53.026950 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:53.026943    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8g5nx\" (UniqueName: \"kubernetes.io/projected/230b68f1-1eeb-4db0-a160-9248f8314240-kube-api-access-8g5nx\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:45:53.138643 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:53.138612    4888 generic.go:356] "Generic (PLEG): container finished" podID="e1cf8b12-1a2a-4935-8c3a-97bfe3281cb6" containerID="09e4971cdd8b96300a148b7b0276e7be874f6966cd3826671c7cb9424a60120a" exitCode=0
Oct 30 18:45:53.138844 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:53.138675    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-59144a1b-569dd9dbd8-45qd4"
Oct 30 18:45:53.138844 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:53.138700    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-59144a1b-569dd9dbd8-45qd4" event={"ID":"e1cf8b12-1a2a-4935-8c3a-97bfe3281cb6","Type":"ContainerDied","Data":"09e4971cdd8b96300a148b7b0276e7be874f6966cd3826671c7cb9424a60120a"}
Oct 30 18:45:53.138844 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:53.138794    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-59144a1b-569dd9dbd8-45qd4" event={"ID":"e1cf8b12-1a2a-4935-8c3a-97bfe3281cb6","Type":"ContainerDied","Data":"3efb010ad23340a9ee7ac6ac0ccdfe5671171547d32337593e38f25c42f59a6c"}
Oct 30 18:45:53.138844 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:53.138824    4888 scope.go:122] "RemoveContainer" containerID="09e4971cdd8b96300a148b7b0276e7be874f6966cd3826671c7cb9424a60120a"
Oct 30 18:45:53.140179 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:53.140153    4888 generic.go:356] "Generic (PLEG): container finished" podID="230b68f1-1eeb-4db0-a160-9248f8314240" containerID="11d8ab27b4ddc1570bc79f437c4c9b54b453f5a7599f917bb9df3731fc8f4006" exitCode=0
Oct 30 18:45:53.140320 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:53.140185    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-59144a1b-7d9455dd67-dmj9h" event={"ID":"230b68f1-1eeb-4db0-a160-9248f8314240","Type":"ContainerDied","Data":"11d8ab27b4ddc1570bc79f437c4c9b54b453f5a7599f917bb9df3731fc8f4006"}
Oct 30 18:45:53.140320 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:53.140220    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-59144a1b-7d9455dd67-dmj9h" event={"ID":"230b68f1-1eeb-4db0-a160-9248f8314240","Type":"ContainerDied","Data":"aa78764a6d7f32b0644dd828ca4434b27897d166ebbac085f45cbcbdc8837f20"}
Oct 30 18:45:53.140320 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:53.140251    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-59144a1b-7d9455dd67-dmj9h"
Oct 30 18:45:53.149866 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:53.149850    4888 scope.go:122] "RemoveContainer" containerID="09e4971cdd8b96300a148b7b0276e7be874f6966cd3826671c7cb9424a60120a"
Oct 30 18:45:53.150142 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:45:53.150121    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"09e4971cdd8b96300a148b7b0276e7be874f6966cd3826671c7cb9424a60120a\": container with ID starting with 09e4971cdd8b96300a148b7b0276e7be874f6966cd3826671c7cb9424a60120a not found: ID does not exist" containerID="09e4971cdd8b96300a148b7b0276e7be874f6966cd3826671c7cb9424a60120a"
Oct 30 18:45:53.150238 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:53.150148    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"09e4971cdd8b96300a148b7b0276e7be874f6966cd3826671c7cb9424a60120a"} err="failed to get container status \"09e4971cdd8b96300a148b7b0276e7be874f6966cd3826671c7cb9424a60120a\": rpc error: code = NotFound desc = could not find container \"09e4971cdd8b96300a148b7b0276e7be874f6966cd3826671c7cb9424a60120a\": container with ID starting with 09e4971cdd8b96300a148b7b0276e7be874f6966cd3826671c7cb9424a60120a not found: ID does not exist"
Oct 30 18:45:53.150238 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:53.150166    4888 scope.go:122] "RemoveContainer" containerID="11d8ab27b4ddc1570bc79f437c4c9b54b453f5a7599f917bb9df3731fc8f4006"
Oct 30 18:45:53.159180 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:53.159159    4888 scope.go:122] "RemoveContainer" containerID="11d8ab27b4ddc1570bc79f437c4c9b54b453f5a7599f917bb9df3731fc8f4006"
Oct 30 18:45:53.159966 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:45:53.159949    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"11d8ab27b4ddc1570bc79f437c4c9b54b453f5a7599f917bb9df3731fc8f4006\": container with ID starting with 11d8ab27b4ddc1570bc79f437c4c9b54b453f5a7599f917bb9df3731fc8f4006 not found: ID does not exist" containerID="11d8ab27b4ddc1570bc79f437c4c9b54b453f5a7599f917bb9df3731fc8f4006"
Oct 30 18:45:53.160040 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:53.159973    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"11d8ab27b4ddc1570bc79f437c4c9b54b453f5a7599f917bb9df3731fc8f4006"} err="failed to get container status \"11d8ab27b4ddc1570bc79f437c4c9b54b453f5a7599f917bb9df3731fc8f4006\": rpc error: code = NotFound desc = could not find container \"11d8ab27b4ddc1570bc79f437c4c9b54b453f5a7599f917bb9df3731fc8f4006\": container with ID starting with 11d8ab27b4ddc1570bc79f437c4c9b54b453f5a7599f917bb9df3731fc8f4006 not found: ID does not exist"
Oct 30 18:45:53.311191 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:53.311109    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-59144a1b-569dd9dbd8-45qd4"]
Oct 30 18:45:53.362342 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:53.362311    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-59144a1b-569dd9dbd8-45qd4"]
Oct 30 18:45:53.474977 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:53.474729    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-59144a1b-7d9455dd67-dmj9h"]
Oct 30 18:45:53.478217 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:53.478196    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-59144a1b-7d9455dd67-dmj9h"]
Oct 30 18:45:53.891795 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:53.891761    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="230b68f1-1eeb-4db0-a160-9248f8314240" path="/var/lib/kubelet/pods/230b68f1-1eeb-4db0-a160-9248f8314240/volumes"
Oct 30 18:45:53.892365 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:53.892348    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e1cf8b12-1a2a-4935-8c3a-97bfe3281cb6" path="/var/lib/kubelet/pods/e1cf8b12-1a2a-4935-8c3a-97bfe3281cb6/volumes"
Oct 30 18:45:54.334244 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:54.334217    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-def78c01-openshift-default-59959c464f-6qpnd"
Oct 30 18:45:54.440958 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:54.440879    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/87986146-ca55-47e6-8f17-9df4ed880eeb-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/87986146-ca55-47e6-8f17-9df4ed880eeb-credential-socket\") pod \"87986146-ca55-47e6-8f17-9df4ed880eeb\" (UID: \"87986146-ca55-47e6-8f17-9df4ed880eeb\") "
Oct 30 18:45:54.440958 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:54.440920    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/87986146-ca55-47e6-8f17-9df4ed880eeb-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/87986146-ca55-47e6-8f17-9df4ed880eeb-istio-podinfo\") pod \"87986146-ca55-47e6-8f17-9df4ed880eeb\" (UID: \"87986146-ca55-47e6-8f17-9df4ed880eeb\") "
Oct 30 18:45:54.441194 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:54.440976    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/87986146-ca55-47e6-8f17-9df4ed880eeb-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/87986146-ca55-47e6-8f17-9df4ed880eeb-workload-socket\") pod \"87986146-ca55-47e6-8f17-9df4ed880eeb\" (UID: \"87986146-ca55-47e6-8f17-9df4ed880eeb\") "
Oct 30 18:45:54.441194 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:54.441006    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/87986146-ca55-47e6-8f17-9df4ed880eeb-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/87986146-ca55-47e6-8f17-9df4ed880eeb-istio-data\") pod \"87986146-ca55-47e6-8f17-9df4ed880eeb\" (UID: \"87986146-ca55-47e6-8f17-9df4ed880eeb\") "
Oct 30 18:45:54.441194 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:54.441049    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/87986146-ca55-47e6-8f17-9df4ed880eeb-istio-token\" (UniqueName: \"kubernetes.io/projected/87986146-ca55-47e6-8f17-9df4ed880eeb-istio-token\") pod \"87986146-ca55-47e6-8f17-9df4ed880eeb\" (UID: \"87986146-ca55-47e6-8f17-9df4ed880eeb\") "
Oct 30 18:45:54.441194 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:54.441069    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/87986146-ca55-47e6-8f17-9df4ed880eeb-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/87986146-ca55-47e6-8f17-9df4ed880eeb-istio-envoy\") pod \"87986146-ca55-47e6-8f17-9df4ed880eeb\" (UID: \"87986146-ca55-47e6-8f17-9df4ed880eeb\") "
Oct 30 18:45:54.441194 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:54.441086    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/87986146-ca55-47e6-8f17-9df4ed880eeb-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/87986146-ca55-47e6-8f17-9df4ed880eeb-workload-certs\") pod \"87986146-ca55-47e6-8f17-9df4ed880eeb\" (UID: \"87986146-ca55-47e6-8f17-9df4ed880eeb\") "
Oct 30 18:45:54.441194 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:54.441108    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/87986146-ca55-47e6-8f17-9df4ed880eeb-kube-api-access-hvkjm\" (UniqueName: \"kubernetes.io/projected/87986146-ca55-47e6-8f17-9df4ed880eeb-kube-api-access-hvkjm\") pod \"87986146-ca55-47e6-8f17-9df4ed880eeb\" (UID: \"87986146-ca55-47e6-8f17-9df4ed880eeb\") "
Oct 30 18:45:54.441194 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:54.441138    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/87986146-ca55-47e6-8f17-9df4ed880eeb-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/87986146-ca55-47e6-8f17-9df4ed880eeb-istiod-ca-cert\") pod \"87986146-ca55-47e6-8f17-9df4ed880eeb\" (UID: \"87986146-ca55-47e6-8f17-9df4ed880eeb\") "
Oct 30 18:45:54.441601 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:54.441191    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/87986146-ca55-47e6-8f17-9df4ed880eeb-credential-socket" pod "87986146-ca55-47e6-8f17-9df4ed880eeb" (UID: "87986146-ca55-47e6-8f17-9df4ed880eeb"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:45:54.441601 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:54.441329    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/87986146-ca55-47e6-8f17-9df4ed880eeb-workload-socket" pod "87986146-ca55-47e6-8f17-9df4ed880eeb" (UID: "87986146-ca55-47e6-8f17-9df4ed880eeb"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:45:54.441601 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:54.441498    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/87986146-ca55-47e6-8f17-9df4ed880eeb-workload-certs" pod "87986146-ca55-47e6-8f17-9df4ed880eeb" (UID: "87986146-ca55-47e6-8f17-9df4ed880eeb"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:45:54.441601 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:54.441510    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/87986146-ca55-47e6-8f17-9df4ed880eeb-istio-data" pod "87986146-ca55-47e6-8f17-9df4ed880eeb" (UID: "87986146-ca55-47e6-8f17-9df4ed880eeb"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:45:54.441799 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:54.441628    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/87986146-ca55-47e6-8f17-9df4ed880eeb-istiod-ca-cert" pod "87986146-ca55-47e6-8f17-9df4ed880eeb" (UID: "87986146-ca55-47e6-8f17-9df4ed880eeb"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:45:54.441799 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:54.441640    4888 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/87986146-ca55-47e6-8f17-9df4ed880eeb-istio-data\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:45:54.441799 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:54.441659    4888 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/87986146-ca55-47e6-8f17-9df4ed880eeb-workload-certs\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:45:54.441799 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:54.441675    4888 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/87986146-ca55-47e6-8f17-9df4ed880eeb-credential-socket\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:45:54.441799 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:54.441688    4888 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/87986146-ca55-47e6-8f17-9df4ed880eeb-workload-socket\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:45:54.443408 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:54.443383    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/87986146-ca55-47e6-8f17-9df4ed880eeb-istio-podinfo" pod "87986146-ca55-47e6-8f17-9df4ed880eeb" (UID: "87986146-ca55-47e6-8f17-9df4ed880eeb"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 18:45:54.443688 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:54.443659    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/87986146-ca55-47e6-8f17-9df4ed880eeb-istio-token" pod "87986146-ca55-47e6-8f17-9df4ed880eeb" (UID: "87986146-ca55-47e6-8f17-9df4ed880eeb"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:45:54.443790 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:54.443721    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/87986146-ca55-47e6-8f17-9df4ed880eeb-istio-envoy" pod "87986146-ca55-47e6-8f17-9df4ed880eeb" (UID: "87986146-ca55-47e6-8f17-9df4ed880eeb"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:45:54.443790 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:54.443732    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/87986146-ca55-47e6-8f17-9df4ed880eeb-kube-api-access-hvkjm" pod "87986146-ca55-47e6-8f17-9df4ed880eeb" (UID: "87986146-ca55-47e6-8f17-9df4ed880eeb"). InnerVolumeSpecName "kube-api-access-hvkjm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:45:54.543448 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:54.543408    4888 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/87986146-ca55-47e6-8f17-9df4ed880eeb-istio-token\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:45:54.543448 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:54.543444    4888 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/87986146-ca55-47e6-8f17-9df4ed880eeb-istio-envoy\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:45:54.543684 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:54.543460    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hvkjm\" (UniqueName: \"kubernetes.io/projected/87986146-ca55-47e6-8f17-9df4ed880eeb-kube-api-access-hvkjm\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:45:54.543684 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:54.543476    4888 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/87986146-ca55-47e6-8f17-9df4ed880eeb-istiod-ca-cert\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:45:54.543684 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:54.543491    4888 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/87986146-ca55-47e6-8f17-9df4ed880eeb-istio-podinfo\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:45:55.165419 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:55.165386    4888 generic.go:356] "Generic (PLEG): container finished" podID="87986146-ca55-47e6-8f17-9df4ed880eeb" containerID="27c14b9c7a5c30a371888884f97f9d5eafae1d6dcccf733f2dbd8b15779c1d6d" exitCode=0
Oct 30 18:45:55.165577 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:55.165445    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-def78c01-openshift-default-59959c464f-6qpnd"
Oct 30 18:45:55.165577 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:55.165454    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-def78c01-openshift-default-59959c464f-6qpnd" event={"ID":"87986146-ca55-47e6-8f17-9df4ed880eeb","Type":"ContainerDied","Data":"27c14b9c7a5c30a371888884f97f9d5eafae1d6dcccf733f2dbd8b15779c1d6d"}
Oct 30 18:45:55.165577 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:55.165517    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-def78c01-openshift-default-59959c464f-6qpnd" event={"ID":"87986146-ca55-47e6-8f17-9df4ed880eeb","Type":"ContainerDied","Data":"257bcad6cc559f8c9c88a3bacdf3d6784a4e79a311e376849ed471078e3525de"}
Oct 30 18:45:55.165577 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:55.165538    4888 scope.go:122] "RemoveContainer" containerID="27c14b9c7a5c30a371888884f97f9d5eafae1d6dcccf733f2dbd8b15779c1d6d"
Oct 30 18:45:55.176972 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:55.176941    4888 scope.go:122] "RemoveContainer" containerID="30a2f64a02963e4beb48c6fdb8f56c71d76426dd791eb38175e528b4f2c91c2e"
Oct 30 18:45:55.186277 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:55.186120    4888 scope.go:122] "RemoveContainer" containerID="27c14b9c7a5c30a371888884f97f9d5eafae1d6dcccf733f2dbd8b15779c1d6d"
Oct 30 18:45:55.186455 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:45:55.186434    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"27c14b9c7a5c30a371888884f97f9d5eafae1d6dcccf733f2dbd8b15779c1d6d\": container with ID starting with 27c14b9c7a5c30a371888884f97f9d5eafae1d6dcccf733f2dbd8b15779c1d6d not found: ID does not exist" containerID="27c14b9c7a5c30a371888884f97f9d5eafae1d6dcccf733f2dbd8b15779c1d6d"
Oct 30 18:45:55.186541 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:55.186468    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"27c14b9c7a5c30a371888884f97f9d5eafae1d6dcccf733f2dbd8b15779c1d6d"} err="failed to get container status \"27c14b9c7a5c30a371888884f97f9d5eafae1d6dcccf733f2dbd8b15779c1d6d\": rpc error: code = NotFound desc = could not find container \"27c14b9c7a5c30a371888884f97f9d5eafae1d6dcccf733f2dbd8b15779c1d6d\": container with ID starting with 27c14b9c7a5c30a371888884f97f9d5eafae1d6dcccf733f2dbd8b15779c1d6d not found: ID does not exist"
Oct 30 18:45:55.186541 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:55.186493    4888 scope.go:122] "RemoveContainer" containerID="30a2f64a02963e4beb48c6fdb8f56c71d76426dd791eb38175e528b4f2c91c2e"
Oct 30 18:45:55.186744 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:45:55.186725    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"30a2f64a02963e4beb48c6fdb8f56c71d76426dd791eb38175e528b4f2c91c2e\": container with ID starting with 30a2f64a02963e4beb48c6fdb8f56c71d76426dd791eb38175e528b4f2c91c2e not found: ID does not exist" containerID="30a2f64a02963e4beb48c6fdb8f56c71d76426dd791eb38175e528b4f2c91c2e"
Oct 30 18:45:55.186782 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:55.186749    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"30a2f64a02963e4beb48c6fdb8f56c71d76426dd791eb38175e528b4f2c91c2e"} err="failed to get container status \"30a2f64a02963e4beb48c6fdb8f56c71d76426dd791eb38175e528b4f2c91c2e\": rpc error: code = NotFound desc = could not find container \"30a2f64a02963e4beb48c6fdb8f56c71d76426dd791eb38175e528b4f2c91c2e\": container with ID starting with 30a2f64a02963e4beb48c6fdb8f56c71d76426dd791eb38175e528b4f2c91c2e not found: ID does not exist"
Oct 30 18:45:55.339576 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:55.339541    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-def78c01-openshift-default-59959c464f-6qpnd"]
Oct 30 18:45:55.390341 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:55.390308    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-def78c01-openshift-default-59959c464f-6qpnd"]
Oct 30 18:45:55.891557 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:55.891528    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="87986146-ca55-47e6-8f17-9df4ed880eeb" path="/var/lib/kubelet/pods/87986146-ca55-47e6-8f17-9df4ed880eeb/volumes"
Oct 30 18:45:55.978979 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:55.978934    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-529e04e2-d954bc5c5-m5jvj" podUID="37620fd5-3a09-4ccb-8708-c51b3cae9520" containerName="payload-pre-processing" containerID="cri-o://c1f29e07d553f3519dc34f5299907005a90c635ef4b0640b8bb62d4bce2b9e59" gracePeriod=25
Oct 30 18:45:56.178075 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:56.177954    4888 generic.go:356] "Generic (PLEG): container finished" podID="37620fd5-3a09-4ccb-8708-c51b3cae9520" containerID="c1f29e07d553f3519dc34f5299907005a90c635ef4b0640b8bb62d4bce2b9e59" exitCode=0
Oct 30 18:45:56.178075 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:56.177983    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-529e04e2-d954bc5c5-m5jvj" event={"ID":"37620fd5-3a09-4ccb-8708-c51b3cae9520","Type":"ContainerDied","Data":"c1f29e07d553f3519dc34f5299907005a90c635ef4b0640b8bb62d4bce2b9e59"}
Oct 30 18:45:56.258210 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:56.258185    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-529e04e2-d954bc5c5-m5jvj"
Oct 30 18:45:56.361510 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:56.361482    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/37620fd5-3a09-4ccb-8708-c51b3cae9520-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/37620fd5-3a09-4ccb-8708-c51b3cae9520-plugins-config-volume\") pod \"37620fd5-3a09-4ccb-8708-c51b3cae9520\" (UID: \"37620fd5-3a09-4ccb-8708-c51b3cae9520\") "
Oct 30 18:45:56.361964 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:56.361527    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/37620fd5-3a09-4ccb-8708-c51b3cae9520-kube-api-access-64rdg\" (UniqueName: \"kubernetes.io/projected/37620fd5-3a09-4ccb-8708-c51b3cae9520-kube-api-access-64rdg\") pod \"37620fd5-3a09-4ccb-8708-c51b3cae9520\" (UID: \"37620fd5-3a09-4ccb-8708-c51b3cae9520\") "
Oct 30 18:45:56.361964 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:56.361793    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/37620fd5-3a09-4ccb-8708-c51b3cae9520-plugins-config-volume" pod "37620fd5-3a09-4ccb-8708-c51b3cae9520" (UID: "37620fd5-3a09-4ccb-8708-c51b3cae9520"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:45:56.363882 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:56.363859    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/37620fd5-3a09-4ccb-8708-c51b3cae9520-kube-api-access-64rdg" pod "37620fd5-3a09-4ccb-8708-c51b3cae9520" (UID: "37620fd5-3a09-4ccb-8708-c51b3cae9520"). InnerVolumeSpecName "kube-api-access-64rdg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:45:56.462557 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:56.462480    4888 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/37620fd5-3a09-4ccb-8708-c51b3cae9520-plugins-config-volume\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:45:56.462557 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:56.462515    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-64rdg\" (UniqueName: \"kubernetes.io/projected/37620fd5-3a09-4ccb-8708-c51b3cae9520-kube-api-access-64rdg\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:45:57.186776 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:57.186743    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-529e04e2-d954bc5c5-m5jvj" event={"ID":"37620fd5-3a09-4ccb-8708-c51b3cae9520","Type":"ContainerDied","Data":"54f11e85b206afa1534955dd392ec0e10eae1ecb5e2a72325095f81725f37577"}
Oct 30 18:45:57.186952 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:57.186810    4888 scope.go:122] "RemoveContainer" containerID="c1f29e07d553f3519dc34f5299907005a90c635ef4b0640b8bb62d4bce2b9e59"
Oct 30 18:45:57.186952 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:57.186917    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-529e04e2-d954bc5c5-m5jvj"
Oct 30 18:45:57.348537 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:57.348507    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-59144a1b-openshift-default-78d555d5bc-lk4t7"]
Oct 30 18:45:57.348735 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:57.348716    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-59144a1b-openshift-default-78d555d5bc-lk4t7" podUID="cf71a4b7-6006-44ba-945c-bb8749af44ad" containerName="istio-proxy" containerID="cri-o://86c4440779a9ac1a1ef7a22332ca112e088b7faca1d90f3250aea8d81a925921" gracePeriod=30
Oct 30 18:45:57.357040 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:57.356940    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-529e04e2-d954bc5c5-m5jvj"]
Oct 30 18:45:57.361852 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:57.361831    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-529e04e2-d954bc5c5-m5jvj"]
Oct 30 18:45:57.626455 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:57.626410    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-3b6013-84b5f84d8b-hfxgw"]
Oct 30 18:45:57.627080 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:57.627057    4888 cpu_manager.go:414] "containerMap: removing container" podUID="87986146-ca55-47e6-8f17-9df4ed880eeb" containerName="istio-proxy"
Oct 30 18:45:57.627191 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:57.627082    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="87986146-ca55-47e6-8f17-9df4ed880eeb" containerName="istio-proxy"
Oct 30 18:45:57.627191 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:57.627098    4888 cpu_manager.go:414] "containerMap: removing container" podUID="4aa64e3a-64c9-4cb1-89a7-19dfd4aa6706" containerName="cleanup"
Oct 30 18:45:57.627191 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:57.627105    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4aa64e3a-64c9-4cb1-89a7-19dfd4aa6706" containerName="cleanup"
Oct 30 18:45:57.627191 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:57.627120    4888 cpu_manager.go:414] "containerMap: removing container" podUID="fcd5e44c-96f2-44e7-baea-b55634db50e6" containerName="payload-processing"
Oct 30 18:45:57.627191 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:57.627128    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fcd5e44c-96f2-44e7-baea-b55634db50e6" containerName="payload-processing"
Oct 30 18:45:57.627191 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:57.627144    4888 cpu_manager.go:414] "containerMap: removing container" podUID="9c207780-577e-483c-b70c-ff0e44e2dc0f" containerName="payload-pre-processing"
Oct 30 18:45:57.627191 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:57.627153    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9c207780-577e-483c-b70c-ff0e44e2dc0f" containerName="payload-pre-processing"
Oct 30 18:45:57.627191 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:57.627187    4888 cpu_manager.go:414] "containerMap: removing container" podUID="e1cf8b12-1a2a-4935-8c3a-97bfe3281cb6" containerName="payload-processing"
Oct 30 18:45:57.627191 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:57.627194    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e1cf8b12-1a2a-4935-8c3a-97bfe3281cb6" containerName="payload-processing"
Oct 30 18:45:57.627531 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:57.627206    4888 cpu_manager.go:414] "containerMap: removing container" podUID="230b68f1-1eeb-4db0-a160-9248f8314240" containerName="payload-pre-processing"
Oct 30 18:45:57.627531 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:57.627214    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="230b68f1-1eeb-4db0-a160-9248f8314240" containerName="payload-pre-processing"
Oct 30 18:45:57.627531 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:57.627225    4888 cpu_manager.go:414] "containerMap: removing container" podUID="82903e3e-982c-4708-a10c-8328e324f7b6" containerName="payload-processing"
Oct 30 18:45:57.627531 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:57.627231    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="82903e3e-982c-4708-a10c-8328e324f7b6" containerName="payload-processing"
Oct 30 18:45:57.627531 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:57.627268    4888 cpu_manager.go:414] "containerMap: removing container" podUID="37620fd5-3a09-4ccb-8708-c51b3cae9520" containerName="payload-pre-processing"
Oct 30 18:45:57.627531 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:57.627274    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="37620fd5-3a09-4ccb-8708-c51b3cae9520" containerName="payload-pre-processing"
Oct 30 18:45:57.627531 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:57.627301    4888 cpu_manager.go:414] "containerMap: removing container" podUID="c7f9892e-42f6-4674-8a84-c1a4d2049807" containerName="payload-pre-processing"
Oct 30 18:45:57.627531 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:57.627307    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c7f9892e-42f6-4674-8a84-c1a4d2049807" containerName="payload-pre-processing"
Oct 30 18:45:57.627531 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:57.627315    4888 cpu_manager.go:414] "containerMap: removing container" podUID="d09017d2-91c5-4f27-a84c-b60db41e36ea" containerName="revoke-keys"
Oct 30 18:45:57.627531 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:57.627322    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d09017d2-91c5-4f27-a84c-b60db41e36ea" containerName="revoke-keys"
Oct 30 18:45:57.627531 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:57.627333    4888 cpu_manager.go:414] "containerMap: removing container" podUID="21d9b47a-e5e0-4bd7-9a9d-384e1be64ad9" containerName="cleanup"
Oct 30 18:45:57.627531 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:57.627340    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="21d9b47a-e5e0-4bd7-9a9d-384e1be64ad9" containerName="cleanup"
Oct 30 18:45:57.627531 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:57.627456    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="37620fd5-3a09-4ccb-8708-c51b3cae9520" containerName="payload-pre-processing"
Oct 30 18:45:57.627531 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:57.627469    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="c7f9892e-42f6-4674-8a84-c1a4d2049807" containerName="payload-pre-processing"
Oct 30 18:45:57.627531 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:57.627478    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="4aa64e3a-64c9-4cb1-89a7-19dfd4aa6706" containerName="cleanup"
Oct 30 18:45:57.627531 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:57.627486    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="21d9b47a-e5e0-4bd7-9a9d-384e1be64ad9" containerName="cleanup"
Oct 30 18:45:57.627531 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:57.627494    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="d09017d2-91c5-4f27-a84c-b60db41e36ea" containerName="revoke-keys"
Oct 30 18:45:57.627531 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:57.627501    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="87986146-ca55-47e6-8f17-9df4ed880eeb" containerName="istio-proxy"
Oct 30 18:45:57.627531 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:57.627512    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="87986146-ca55-47e6-8f17-9df4ed880eeb" containerName="istio-proxy"
Oct 30 18:45:57.627531 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:57.627521    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="fcd5e44c-96f2-44e7-baea-b55634db50e6" containerName="payload-processing"
Oct 30 18:45:57.627531 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:57.627530    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="9c207780-577e-483c-b70c-ff0e44e2dc0f" containerName="payload-pre-processing"
Oct 30 18:45:57.628387 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:57.627544    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="e1cf8b12-1a2a-4935-8c3a-97bfe3281cb6" containerName="payload-processing"
Oct 30 18:45:57.628387 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:57.627554    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="230b68f1-1eeb-4db0-a160-9248f8314240" containerName="payload-pre-processing"
Oct 30 18:45:57.628387 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:57.627562    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="82903e3e-982c-4708-a10c-8328e324f7b6" containerName="payload-processing"
Oct 30 18:45:57.632826 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:57.632795    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w3-3b6013-84b5f84d8b-hfxgw"
Oct 30 18:45:57.687461 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:57.687246    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w3-3b6013\""
Oct 30 18:45:57.687461 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:57.687418    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w3-3b6013-dockercfg-wr57f\""
Oct 30 18:45:57.690934 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:57.690911    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-3b6013-84b5f84d8b-hfxgw"]
Oct 30 18:45:57.777477 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:57.777448    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/337a7530-cf88-4466-818c-d025be569509-plugins-config-volume\") pod \"payload-processing-e2e-worker-w3-3b6013-84b5f84d8b-hfxgw\" (UID: \"337a7530-cf88-4466-818c-d025be569509\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-3b6013-84b5f84d8b-hfxgw"
Oct 30 18:45:57.777632 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:57.777560    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-24xql\" (UniqueName: \"kubernetes.io/projected/337a7530-cf88-4466-818c-d025be569509-kube-api-access-24xql\") pod \"payload-processing-e2e-worker-w3-3b6013-84b5f84d8b-hfxgw\" (UID: \"337a7530-cf88-4466-818c-d025be569509\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-3b6013-84b5f84d8b-hfxgw"
Oct 30 18:45:57.800766 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:57.800738    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-3b6013-57fbf8ccf5-bz979"]
Oct 30 18:45:57.801180 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:57.801165    4888 cpu_manager.go:414] "containerMap: removing container" podUID="87986146-ca55-47e6-8f17-9df4ed880eeb" containerName="istio-proxy"
Oct 30 18:45:57.801180 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:57.801180    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="87986146-ca55-47e6-8f17-9df4ed880eeb" containerName="istio-proxy"
Oct 30 18:45:57.811588 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:57.811567    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-3b6013-57fbf8ccf5-bz979"
Oct 30 18:45:57.865241 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:57.865209    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-3b6013-57fbf8ccf5-bz979"]
Oct 30 18:45:57.878946 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:57.878870    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/337a7530-cf88-4466-818c-d025be569509-plugins-config-volume\") pod \"payload-processing-e2e-worker-w3-3b6013-84b5f84d8b-hfxgw\" (UID: \"337a7530-cf88-4466-818c-d025be569509\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-3b6013-84b5f84d8b-hfxgw"
Oct 30 18:45:57.879052 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:57.878982    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-24xql\" (UniqueName: \"kubernetes.io/projected/337a7530-cf88-4466-818c-d025be569509-kube-api-access-24xql\") pod \"payload-processing-e2e-worker-w3-3b6013-84b5f84d8b-hfxgw\" (UID: \"337a7530-cf88-4466-818c-d025be569509\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-3b6013-84b5f84d8b-hfxgw"
Oct 30 18:45:57.879578 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:57.879560    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/337a7530-cf88-4466-818c-d025be569509-plugins-config-volume\") pod \"payload-processing-e2e-worker-w3-3b6013-84b5f84d8b-hfxgw\" (UID: \"337a7530-cf88-4466-818c-d025be569509\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-3b6013-84b5f84d8b-hfxgw"
Oct 30 18:45:57.889802 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:57.889772    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="37620fd5-3a09-4ccb-8708-c51b3cae9520" path="/var/lib/kubelet/pods/37620fd5-3a09-4ccb-8708-c51b3cae9520/volumes"
Oct 30 18:45:57.942188 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:57.942167    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-24xql\" (UniqueName: \"kubernetes.io/projected/337a7530-cf88-4466-818c-d025be569509-kube-api-access-24xql\") pod \"payload-processing-e2e-worker-w3-3b6013-84b5f84d8b-hfxgw\" (UID: \"337a7530-cf88-4466-818c-d025be569509\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-3b6013-84b5f84d8b-hfxgw"
Oct 30 18:45:57.950061 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:57.950040    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w3-3b6013-84b5f84d8b-hfxgw"
Oct 30 18:45:57.980551 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:57.980515    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/42a5ae09-5a33-4744-9e64-8e0f0084cd97-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w3-3b6013-57fbf8ccf5-bz979\" (UID: \"42a5ae09-5a33-4744-9e64-8e0f0084cd97\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-3b6013-57fbf8ccf5-bz979"
Oct 30 18:45:57.980667 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:57.980590    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t5z72\" (UniqueName: \"kubernetes.io/projected/42a5ae09-5a33-4744-9e64-8e0f0084cd97-kube-api-access-t5z72\") pod \"payload-pre-processing-e2e-worker-w3-3b6013-57fbf8ccf5-bz979\" (UID: \"42a5ae09-5a33-4744-9e64-8e0f0084cd97\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-3b6013-57fbf8ccf5-bz979"
Oct 30 18:45:58.081708 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:58.081676    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/42a5ae09-5a33-4744-9e64-8e0f0084cd97-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w3-3b6013-57fbf8ccf5-bz979\" (UID: \"42a5ae09-5a33-4744-9e64-8e0f0084cd97\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-3b6013-57fbf8ccf5-bz979"
Oct 30 18:45:58.081708 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:58.081708    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t5z72\" (UniqueName: \"kubernetes.io/projected/42a5ae09-5a33-4744-9e64-8e0f0084cd97-kube-api-access-t5z72\") pod \"payload-pre-processing-e2e-worker-w3-3b6013-57fbf8ccf5-bz979\" (UID: \"42a5ae09-5a33-4744-9e64-8e0f0084cd97\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-3b6013-57fbf8ccf5-bz979"
Oct 30 18:45:58.082464 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:58.082443    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/42a5ae09-5a33-4744-9e64-8e0f0084cd97-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w3-3b6013-57fbf8ccf5-bz979\" (UID: \"42a5ae09-5a33-4744-9e64-8e0f0084cd97\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-3b6013-57fbf8ccf5-bz979"
Oct 30 18:45:58.144339 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:58.144264    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t5z72\" (UniqueName: \"kubernetes.io/projected/42a5ae09-5a33-4744-9e64-8e0f0084cd97-kube-api-access-t5z72\") pod \"payload-pre-processing-e2e-worker-w3-3b6013-57fbf8ccf5-bz979\" (UID: \"42a5ae09-5a33-4744-9e64-8e0f0084cd97\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-3b6013-57fbf8ccf5-bz979"
Oct 30 18:45:58.330475 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:58.330456    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-3b6013-84b5f84d8b-hfxgw"]
Oct 30 18:45:58.331712 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:45:58.331689    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod337a7530_cf88_4466_818c_d025be569509.slice/crio-dae5121e63ac3736c431c53778eefbe3b6e72299987a75ff013a6bb1409222e1 WatchSource:0}: Error finding container dae5121e63ac3736c431c53778eefbe3b6e72299987a75ff013a6bb1409222e1: Status 404 returned error can't find the container with id dae5121e63ac3736c431c53778eefbe3b6e72299987a75ff013a6bb1409222e1
Oct 30 18:45:58.427325 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:58.427300    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-3b6013-57fbf8ccf5-bz979"
Oct 30 18:45:58.720111 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:58.720034    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-cgw6d"]
Oct 30 18:45:58.724997 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:58.724976    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-cgw6d"
Oct 30 18:45:58.781307 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:58.779487    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w2-4a7a52-openshift-default-dockercfg-9p592\""
Oct 30 18:45:58.791250 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:58.791215    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-cgw6d"]
Oct 30 18:45:58.808044 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:58.808016    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-3b6013-57fbf8ccf5-bz979"]
Oct 30 18:45:58.809753 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:45:58.809727    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42a5ae09_5a33_4744_9e64_8e0f0084cd97.slice/crio-44beab25f0159f8a6f392da563e7c5ac672e8b6b8493e12c1226b59805e34703 WatchSource:0}: Error finding container 44beab25f0159f8a6f392da563e7c5ac672e8b6b8493e12c1226b59805e34703: Status 404 returned error can't find the container with id 44beab25f0159f8a6f392da563e7c5ac672e8b6b8493e12c1226b59805e34703
Oct 30 18:45:58.890707 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:58.890675    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bae588a3-e65e-4f9b-b4c4-615aa878741e-istio-podinfo\") pod \"e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-cgw6d\" (UID: \"bae588a3-e65e-4f9b-b4c4-615aa878741e\") " pod="openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-cgw6d"
Oct 30 18:45:58.890707 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:58.890710    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bae588a3-e65e-4f9b-b4c4-615aa878741e-istio-envoy\") pod \"e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-cgw6d\" (UID: \"bae588a3-e65e-4f9b-b4c4-615aa878741e\") " pod="openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-cgw6d"
Oct 30 18:45:58.890916 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:58.890744    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bae588a3-e65e-4f9b-b4c4-615aa878741e-istio-token\") pod \"e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-cgw6d\" (UID: \"bae588a3-e65e-4f9b-b4c4-615aa878741e\") " pod="openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-cgw6d"
Oct 30 18:45:58.890916 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:58.890811    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-678m5\" (UniqueName: \"kubernetes.io/projected/bae588a3-e65e-4f9b-b4c4-615aa878741e-kube-api-access-678m5\") pod \"e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-cgw6d\" (UID: \"bae588a3-e65e-4f9b-b4c4-615aa878741e\") " pod="openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-cgw6d"
Oct 30 18:45:58.890916 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:58.890861    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bae588a3-e65e-4f9b-b4c4-615aa878741e-credential-socket\") pod \"e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-cgw6d\" (UID: \"bae588a3-e65e-4f9b-b4c4-615aa878741e\") " pod="openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-cgw6d"
Oct 30 18:45:58.891008 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:58.890920    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bae588a3-e65e-4f9b-b4c4-615aa878741e-workload-certs\") pod \"e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-cgw6d\" (UID: \"bae588a3-e65e-4f9b-b4c4-615aa878741e\") " pod="openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-cgw6d"
Oct 30 18:45:58.891008 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:58.890960    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bae588a3-e65e-4f9b-b4c4-615aa878741e-istiod-ca-cert\") pod \"e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-cgw6d\" (UID: \"bae588a3-e65e-4f9b-b4c4-615aa878741e\") " pod="openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-cgw6d"
Oct 30 18:45:58.891008 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:58.890992    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bae588a3-e65e-4f9b-b4c4-615aa878741e-istio-data\") pod \"e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-cgw6d\" (UID: \"bae588a3-e65e-4f9b-b4c4-615aa878741e\") " pod="openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-cgw6d"
Oct 30 18:45:58.891098 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:58.891044    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bae588a3-e65e-4f9b-b4c4-615aa878741e-workload-socket\") pod \"e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-cgw6d\" (UID: \"bae588a3-e65e-4f9b-b4c4-615aa878741e\") " pod="openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-cgw6d"
Oct 30 18:45:58.992550 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:58.992479    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bae588a3-e65e-4f9b-b4c4-615aa878741e-istio-token\") pod \"e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-cgw6d\" (UID: \"bae588a3-e65e-4f9b-b4c4-615aa878741e\") " pod="openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-cgw6d"
Oct 30 18:45:58.992550 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:58.992518    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-678m5\" (UniqueName: \"kubernetes.io/projected/bae588a3-e65e-4f9b-b4c4-615aa878741e-kube-api-access-678m5\") pod \"e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-cgw6d\" (UID: \"bae588a3-e65e-4f9b-b4c4-615aa878741e\") " pod="openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-cgw6d"
Oct 30 18:45:58.992550 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:58.992549    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bae588a3-e65e-4f9b-b4c4-615aa878741e-credential-socket\") pod \"e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-cgw6d\" (UID: \"bae588a3-e65e-4f9b-b4c4-615aa878741e\") " pod="openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-cgw6d"
Oct 30 18:45:58.992825 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:58.992667    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bae588a3-e65e-4f9b-b4c4-615aa878741e-workload-certs\") pod \"e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-cgw6d\" (UID: \"bae588a3-e65e-4f9b-b4c4-615aa878741e\") " pod="openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-cgw6d"
Oct 30 18:45:58.992825 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:58.992763    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bae588a3-e65e-4f9b-b4c4-615aa878741e-istiod-ca-cert\") pod \"e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-cgw6d\" (UID: \"bae588a3-e65e-4f9b-b4c4-615aa878741e\") " pod="openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-cgw6d"
Oct 30 18:45:58.992825 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:58.992816    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bae588a3-e65e-4f9b-b4c4-615aa878741e-istio-data\") pod \"e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-cgw6d\" (UID: \"bae588a3-e65e-4f9b-b4c4-615aa878741e\") " pod="openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-cgw6d"
Oct 30 18:45:58.992989 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:58.992857    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bae588a3-e65e-4f9b-b4c4-615aa878741e-workload-socket\") pod \"e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-cgw6d\" (UID: \"bae588a3-e65e-4f9b-b4c4-615aa878741e\") " pod="openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-cgw6d"
Oct 30 18:45:58.992989 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:58.992919    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bae588a3-e65e-4f9b-b4c4-615aa878741e-istio-podinfo\") pod \"e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-cgw6d\" (UID: \"bae588a3-e65e-4f9b-b4c4-615aa878741e\") " pod="openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-cgw6d"
Oct 30 18:45:58.992989 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:58.992947    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bae588a3-e65e-4f9b-b4c4-615aa878741e-istio-envoy\") pod \"e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-cgw6d\" (UID: \"bae588a3-e65e-4f9b-b4c4-615aa878741e\") " pod="openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-cgw6d"
Oct 30 18:45:58.992989 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:58.992962    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bae588a3-e65e-4f9b-b4c4-615aa878741e-credential-socket\") pod \"e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-cgw6d\" (UID: \"bae588a3-e65e-4f9b-b4c4-615aa878741e\") " pod="openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-cgw6d"
Oct 30 18:45:58.993221 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:58.993031    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bae588a3-e65e-4f9b-b4c4-615aa878741e-workload-certs\") pod \"e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-cgw6d\" (UID: \"bae588a3-e65e-4f9b-b4c4-615aa878741e\") " pod="openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-cgw6d"
Oct 30 18:45:58.993335 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:58.993316    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bae588a3-e65e-4f9b-b4c4-615aa878741e-workload-socket\") pod \"e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-cgw6d\" (UID: \"bae588a3-e65e-4f9b-b4c4-615aa878741e\") " pod="openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-cgw6d"
Oct 30 18:45:58.993408 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:58.993383    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bae588a3-e65e-4f9b-b4c4-615aa878741e-istio-data\") pod \"e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-cgw6d\" (UID: \"bae588a3-e65e-4f9b-b4c4-615aa878741e\") " pod="openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-cgw6d"
Oct 30 18:45:58.993637 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:58.993611    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bae588a3-e65e-4f9b-b4c4-615aa878741e-istiod-ca-cert\") pod \"e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-cgw6d\" (UID: \"bae588a3-e65e-4f9b-b4c4-615aa878741e\") " pod="openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-cgw6d"
Oct 30 18:45:58.995861 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:58.995840    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bae588a3-e65e-4f9b-b4c4-615aa878741e-istio-envoy\") pod \"e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-cgw6d\" (UID: \"bae588a3-e65e-4f9b-b4c4-615aa878741e\") " pod="openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-cgw6d"
Oct 30 18:45:58.996025 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:58.996010    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bae588a3-e65e-4f9b-b4c4-615aa878741e-istio-podinfo\") pod \"e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-cgw6d\" (UID: \"bae588a3-e65e-4f9b-b4c4-615aa878741e\") " pod="openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-cgw6d"
Oct 30 18:45:59.055033 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:59.054997    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-678m5\" (UniqueName: \"kubernetes.io/projected/bae588a3-e65e-4f9b-b4c4-615aa878741e-kube-api-access-678m5\") pod \"e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-cgw6d\" (UID: \"bae588a3-e65e-4f9b-b4c4-615aa878741e\") " pod="openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-cgw6d"
Oct 30 18:45:59.055513 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:59.055489    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bae588a3-e65e-4f9b-b4c4-615aa878741e-istio-token\") pod \"e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-cgw6d\" (UID: \"bae588a3-e65e-4f9b-b4c4-615aa878741e\") " pod="openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-cgw6d"
Oct 30 18:45:59.202598 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:59.202552    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w3-3b6013-84b5f84d8b-hfxgw" event={"ID":"337a7530-cf88-4466-818c-d025be569509","Type":"ContainerStarted","Data":"77c4fc34120888101629aa521c214162d2dd7365dc335fdc65ac5bbf29b2708b"}
Oct 30 18:45:59.202794 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:59.202735    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w3-3b6013-84b5f84d8b-hfxgw"
Oct 30 18:45:59.202794 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:59.202772    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w3-3b6013-84b5f84d8b-hfxgw" event={"ID":"337a7530-cf88-4466-818c-d025be569509","Type":"ContainerStarted","Data":"dae5121e63ac3736c431c53778eefbe3b6e72299987a75ff013a6bb1409222e1"}
Oct 30 18:45:59.204180 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:59.204157    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-3b6013-57fbf8ccf5-bz979" event={"ID":"42a5ae09-5a33-4744-9e64-8e0f0084cd97","Type":"ContainerStarted","Data":"db05e15f0e6b4bcf13a007cc06289b5ea825e1febc79948819c7f11b0d1a7e79"}
Oct 30 18:45:59.204338 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:59.204206    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-3b6013-57fbf8ccf5-bz979" event={"ID":"42a5ae09-5a33-4744-9e64-8e0f0084cd97","Type":"ContainerStarted","Data":"44beab25f0159f8a6f392da563e7c5ac672e8b6b8493e12c1226b59805e34703"}
Oct 30 18:45:59.204338 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:59.204233    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-3b6013-57fbf8ccf5-bz979"
Oct 30 18:45:59.321445 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:59.321362    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w3-3b6013-84b5f84d8b-hfxgw" podStartSLOduration=2.32134992 podStartE2EDuration="2.32134992s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:45:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:45:59.320973627 +0000 UTC m=+723.568379652" watchObservedRunningTime="2026-10-30 18:45:59.32134992 +0000 UTC m=+723.568755945"
Oct 30 18:45:59.338188 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:59.338166    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-cgw6d"
Oct 30 18:45:59.453203 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:59.453150    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-3b6013-57fbf8ccf5-bz979" podStartSLOduration=2.453136037 podStartE2EDuration="2.453136037s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:45:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:45:59.452015083 +0000 UTC m=+723.699421119" watchObservedRunningTime="2026-10-30 18:45:59.453136037 +0000 UTC m=+723.700542068"
Oct 30 18:45:59.775467 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:59.775443    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-cgw6d"]
Oct 30 18:45:59.776950 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:45:59.776930    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbae588a3_e65e_4f9b_b4c4_615aa878741e.slice/crio-0fb8ed749ce9dddab6ab0e08e6200f187183a23fc5fdc5442c793b20960189f3 WatchSource:0}: Error finding container 0fb8ed749ce9dddab6ab0e08e6200f187183a23fc5fdc5442c793b20960189f3: Status 404 returned error can't find the container with id 0fb8ed749ce9dddab6ab0e08e6200f187183a23fc5fdc5442c793b20960189f3
Oct 30 18:45:59.779448 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:59.779412    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:45:59.779533 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:59.779495    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:45:59.779576 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:45:59.779538    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:46:00.212312 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:00.212254    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-cgw6d" event={"ID":"bae588a3-e65e-4f9b-b4c4-615aa878741e","Type":"ContainerStarted","Data":"7c50df43789b05948bdbc79637a365e4bc455c0277c5f7755a9d5cff7caa60f3"}
Oct 30 18:46:00.212485 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:00.212344    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-cgw6d" event={"ID":"bae588a3-e65e-4f9b-b4c4-615aa878741e","Type":"ContainerStarted","Data":"0fb8ed749ce9dddab6ab0e08e6200f187183a23fc5fdc5442c793b20960189f3"}
Oct 30 18:46:00.337481 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:00.337420    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-cgw6d" podStartSLOduration=2.3374070319999998 podStartE2EDuration="2.337407032s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:45:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:46:00.335606956 +0000 UTC m=+724.583012981" watchObservedRunningTime="2026-10-30 18:46:00.337407032 +0000 UTC m=+724.584813056"
Oct 30 18:46:00.338647 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:00.338628    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-cgw6d"
Oct 30 18:46:00.344440 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:00.344422    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-cgw6d"
Oct 30 18:46:00.780368 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:00.780321    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-620f2843-699959648c-2vshr"]
Oct 30 18:46:01.222000 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:01.221972    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-cgw6d"
Oct 30 18:46:01.222834 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:01.222812    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-cgw6d"
Oct 30 18:46:02.621641 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:02.621584    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-59144a1b-openshift-default-78d555d5bc-lk4t7"
Oct 30 18:46:02.729017 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:02.728984    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cf71a4b7-6006-44ba-945c-bb8749af44ad-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf71a4b7-6006-44ba-945c-bb8749af44ad-credential-socket\") pod \"cf71a4b7-6006-44ba-945c-bb8749af44ad\" (UID: \"cf71a4b7-6006-44ba-945c-bb8749af44ad\") "
Oct 30 18:46:02.729181 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:02.729097    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cf71a4b7-6006-44ba-945c-bb8749af44ad-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cf71a4b7-6006-44ba-945c-bb8749af44ad-istio-data\") pod \"cf71a4b7-6006-44ba-945c-bb8749af44ad\" (UID: \"cf71a4b7-6006-44ba-945c-bb8749af44ad\") "
Oct 30 18:46:02.729181 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:02.729133    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/cf71a4b7-6006-44ba-945c-bb8749af44ad-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cf71a4b7-6006-44ba-945c-bb8749af44ad-istiod-ca-cert\") pod \"cf71a4b7-6006-44ba-945c-bb8749af44ad\" (UID: \"cf71a4b7-6006-44ba-945c-bb8749af44ad\") "
Oct 30 18:46:02.729254 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:02.729180    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/cf71a4b7-6006-44ba-945c-bb8749af44ad-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cf71a4b7-6006-44ba-945c-bb8749af44ad-istio-podinfo\") pod \"cf71a4b7-6006-44ba-945c-bb8749af44ad\" (UID: \"cf71a4b7-6006-44ba-945c-bb8749af44ad\") "
Oct 30 18:46:02.729254 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:02.729220    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cf71a4b7-6006-44ba-945c-bb8749af44ad-istio-token\" (UniqueName: \"kubernetes.io/projected/cf71a4b7-6006-44ba-945c-bb8749af44ad-istio-token\") pod \"cf71a4b7-6006-44ba-945c-bb8749af44ad\" (UID: \"cf71a4b7-6006-44ba-945c-bb8749af44ad\") "
Oct 30 18:46:02.729254 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:02.729249    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cf71a4b7-6006-44ba-945c-bb8749af44ad-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cf71a4b7-6006-44ba-945c-bb8749af44ad-workload-certs\") pod \"cf71a4b7-6006-44ba-945c-bb8749af44ad\" (UID: \"cf71a4b7-6006-44ba-945c-bb8749af44ad\") "
Oct 30 18:46:02.729438 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:02.729332    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf71a4b7-6006-44ba-945c-bb8749af44ad-credential-socket" pod "cf71a4b7-6006-44ba-945c-bb8749af44ad" (UID: "cf71a4b7-6006-44ba-945c-bb8749af44ad"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:46:02.729438 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:02.729344    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cf71a4b7-6006-44ba-945c-bb8749af44ad-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf71a4b7-6006-44ba-945c-bb8749af44ad-workload-socket\") pod \"cf71a4b7-6006-44ba-945c-bb8749af44ad\" (UID: \"cf71a4b7-6006-44ba-945c-bb8749af44ad\") "
Oct 30 18:46:02.729438 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:02.729409    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf71a4b7-6006-44ba-945c-bb8749af44ad-istio-data" pod "cf71a4b7-6006-44ba-945c-bb8749af44ad" (UID: "cf71a4b7-6006-44ba-945c-bb8749af44ad"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:46:02.729438 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:02.729426    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cf71a4b7-6006-44ba-945c-bb8749af44ad-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cf71a4b7-6006-44ba-945c-bb8749af44ad-istio-envoy\") pod \"cf71a4b7-6006-44ba-945c-bb8749af44ad\" (UID: \"cf71a4b7-6006-44ba-945c-bb8749af44ad\") "
Oct 30 18:46:02.729642 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:02.729472    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cf71a4b7-6006-44ba-945c-bb8749af44ad-kube-api-access-nsv9h\" (UniqueName: \"kubernetes.io/projected/cf71a4b7-6006-44ba-945c-bb8749af44ad-kube-api-access-nsv9h\") pod \"cf71a4b7-6006-44ba-945c-bb8749af44ad\" (UID: \"cf71a4b7-6006-44ba-945c-bb8749af44ad\") "
Oct 30 18:46:02.729642 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:02.729604    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf71a4b7-6006-44ba-945c-bb8749af44ad-workload-certs" pod "cf71a4b7-6006-44ba-945c-bb8749af44ad" (UID: "cf71a4b7-6006-44ba-945c-bb8749af44ad"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:46:02.729732 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:02.729638    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cf71a4b7-6006-44ba-945c-bb8749af44ad-istiod-ca-cert" pod "cf71a4b7-6006-44ba-945c-bb8749af44ad" (UID: "cf71a4b7-6006-44ba-945c-bb8749af44ad"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:46:02.729792 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:02.729772    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf71a4b7-6006-44ba-945c-bb8749af44ad-workload-socket" pod "cf71a4b7-6006-44ba-945c-bb8749af44ad" (UID: "cf71a4b7-6006-44ba-945c-bb8749af44ad"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:46:02.729958 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:02.729937    4888 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cf71a4b7-6006-44ba-945c-bb8749af44ad-istio-data\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:46:02.729958 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:02.729961    4888 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cf71a4b7-6006-44ba-945c-bb8749af44ad-istiod-ca-cert\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:46:02.730322 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:02.729974    4888 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cf71a4b7-6006-44ba-945c-bb8749af44ad-workload-certs\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:46:02.730322 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:02.729989    4888 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf71a4b7-6006-44ba-945c-bb8749af44ad-workload-socket\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:46:02.730322 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:02.730002    4888 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf71a4b7-6006-44ba-945c-bb8749af44ad-credential-socket\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:46:02.731766 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:02.731736    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cf71a4b7-6006-44ba-945c-bb8749af44ad-istio-token" pod "cf71a4b7-6006-44ba-945c-bb8749af44ad" (UID: "cf71a4b7-6006-44ba-945c-bb8749af44ad"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:46:02.731850 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:02.731777    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf71a4b7-6006-44ba-945c-bb8749af44ad-istio-envoy" pod "cf71a4b7-6006-44ba-945c-bb8749af44ad" (UID: "cf71a4b7-6006-44ba-945c-bb8749af44ad"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:46:02.731850 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:02.731808    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cf71a4b7-6006-44ba-945c-bb8749af44ad-kube-api-access-nsv9h" pod "cf71a4b7-6006-44ba-945c-bb8749af44ad" (UID: "cf71a4b7-6006-44ba-945c-bb8749af44ad"). InnerVolumeSpecName "kube-api-access-nsv9h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:46:02.731850 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:02.731831    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/cf71a4b7-6006-44ba-945c-bb8749af44ad-istio-podinfo" pod "cf71a4b7-6006-44ba-945c-bb8749af44ad" (UID: "cf71a4b7-6006-44ba-945c-bb8749af44ad"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 18:46:02.831124 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:02.831088    4888 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cf71a4b7-6006-44ba-945c-bb8749af44ad-istio-podinfo\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:46:02.831124 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:02.831126    4888 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cf71a4b7-6006-44ba-945c-bb8749af44ad-istio-token\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:46:02.831324 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:02.831142    4888 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cf71a4b7-6006-44ba-945c-bb8749af44ad-istio-envoy\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:46:02.831324 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:02.831156    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nsv9h\" (UniqueName: \"kubernetes.io/projected/cf71a4b7-6006-44ba-945c-bb8749af44ad-kube-api-access-nsv9h\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:46:03.237982 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:03.237953    4888 generic.go:356] "Generic (PLEG): container finished" podID="cf71a4b7-6006-44ba-945c-bb8749af44ad" containerID="86c4440779a9ac1a1ef7a22332ca112e088b7faca1d90f3250aea8d81a925921" exitCode=0
Oct 30 18:46:03.238223 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:03.238001    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-59144a1b-openshift-default-78d555d5bc-lk4t7"
Oct 30 18:46:03.238223 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:03.238027    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-59144a1b-openshift-default-78d555d5bc-lk4t7" event={"ID":"cf71a4b7-6006-44ba-945c-bb8749af44ad","Type":"ContainerDied","Data":"86c4440779a9ac1a1ef7a22332ca112e088b7faca1d90f3250aea8d81a925921"}
Oct 30 18:46:03.238223 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:03.238075    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-59144a1b-openshift-default-78d555d5bc-lk4t7" event={"ID":"cf71a4b7-6006-44ba-945c-bb8749af44ad","Type":"ContainerDied","Data":"f20bace6c42b409cc2a263e3dae90c01fb28e0af5ddc8b7a33fea16205f9c2a1"}
Oct 30 18:46:03.238223 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:03.238096    4888 scope.go:122] "RemoveContainer" containerID="86c4440779a9ac1a1ef7a22332ca112e088b7faca1d90f3250aea8d81a925921"
Oct 30 18:46:03.250125 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:03.250107    4888 scope.go:122] "RemoveContainer" containerID="ee735a6244ff50f6b0d5fc8a91a7a09353311d638bff44afe7c9f112d9276a24"
Oct 30 18:46:03.259576 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:03.259555    4888 scope.go:122] "RemoveContainer" containerID="86c4440779a9ac1a1ef7a22332ca112e088b7faca1d90f3250aea8d81a925921"
Oct 30 18:46:03.259887 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:46:03.259868    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"86c4440779a9ac1a1ef7a22332ca112e088b7faca1d90f3250aea8d81a925921\": container with ID starting with 86c4440779a9ac1a1ef7a22332ca112e088b7faca1d90f3250aea8d81a925921 not found: ID does not exist" containerID="86c4440779a9ac1a1ef7a22332ca112e088b7faca1d90f3250aea8d81a925921"
Oct 30 18:46:03.259941 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:03.259904    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"86c4440779a9ac1a1ef7a22332ca112e088b7faca1d90f3250aea8d81a925921"} err="failed to get container status \"86c4440779a9ac1a1ef7a22332ca112e088b7faca1d90f3250aea8d81a925921\": rpc error: code = NotFound desc = could not find container \"86c4440779a9ac1a1ef7a22332ca112e088b7faca1d90f3250aea8d81a925921\": container with ID starting with 86c4440779a9ac1a1ef7a22332ca112e088b7faca1d90f3250aea8d81a925921 not found: ID does not exist"
Oct 30 18:46:03.259941 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:03.259921    4888 scope.go:122] "RemoveContainer" containerID="ee735a6244ff50f6b0d5fc8a91a7a09353311d638bff44afe7c9f112d9276a24"
Oct 30 18:46:03.260187 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:46:03.260170    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ee735a6244ff50f6b0d5fc8a91a7a09353311d638bff44afe7c9f112d9276a24\": container with ID starting with ee735a6244ff50f6b0d5fc8a91a7a09353311d638bff44afe7c9f112d9276a24 not found: ID does not exist" containerID="ee735a6244ff50f6b0d5fc8a91a7a09353311d638bff44afe7c9f112d9276a24"
Oct 30 18:46:03.260241 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:03.260194    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ee735a6244ff50f6b0d5fc8a91a7a09353311d638bff44afe7c9f112d9276a24"} err="failed to get container status \"ee735a6244ff50f6b0d5fc8a91a7a09353311d638bff44afe7c9f112d9276a24\": rpc error: code = NotFound desc = could not find container \"ee735a6244ff50f6b0d5fc8a91a7a09353311d638bff44afe7c9f112d9276a24\": container with ID starting with ee735a6244ff50f6b0d5fc8a91a7a09353311d638bff44afe7c9f112d9276a24 not found: ID does not exist"
Oct 30 18:46:03.416092 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:03.416057    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-59144a1b-openshift-default-78d555d5bc-lk4t7"]
Oct 30 18:46:03.465798 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:03.465769    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-59144a1b-openshift-default-78d555d5bc-lk4t7"]
Oct 30 18:46:03.897223 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:03.897185    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="cf71a4b7-6006-44ba-945c-bb8749af44ad" path="/var/lib/kubelet/pods/cf71a4b7-6006-44ba-945c-bb8749af44ad/volumes"
Oct 30 18:46:05.015619 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:05.015585    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-2ade09-85459998c9-cksnq"]
Oct 30 18:46:05.016317 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:05.016263    4888 cpu_manager.go:414] "containerMap: removing container" podUID="cf71a4b7-6006-44ba-945c-bb8749af44ad" containerName="istio-proxy"
Oct 30 18:46:05.016317 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:05.016301    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cf71a4b7-6006-44ba-945c-bb8749af44ad" containerName="istio-proxy"
Oct 30 18:46:05.016496 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:05.016480    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="cf71a4b7-6006-44ba-945c-bb8749af44ad" containerName="istio-proxy"
Oct 30 18:46:05.016553 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:05.016505    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="cf71a4b7-6006-44ba-945c-bb8749af44ad" containerName="istio-proxy"
Oct 30 18:46:05.021405 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:05.021384    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-2ade09-85459998c9-cksnq"
Oct 30 18:46:05.052224 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:05.052196    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e7d8237d-903d-4136-8e74-b90346f5f619-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-2ade09-85459998c9-cksnq\" (UID: \"e7d8237d-903d-4136-8e74-b90346f5f619\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-2ade09-85459998c9-cksnq"
Oct 30 18:46:05.052385 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:05.052372    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vctb2\" (UniqueName: \"kubernetes.io/projected/e7d8237d-903d-4136-8e74-b90346f5f619-kube-api-access-vctb2\") pod \"payload-pre-processing-e2e-worker-w0-2ade09-85459998c9-cksnq\" (UID: \"e7d8237d-903d-4136-8e74-b90346f5f619\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-2ade09-85459998c9-cksnq"
Oct 30 18:46:05.076981 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:05.076949    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-2ade09-85459998c9-cksnq"]
Oct 30 18:46:05.125564 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:05.125530    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w0-2ade09\""
Oct 30 18:46:05.149593 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:05.149561    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w0-2ade09-dockercfg-rzhzt\""
Oct 30 18:46:05.153201 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:05.153163    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vctb2\" (UniqueName: \"kubernetes.io/projected/e7d8237d-903d-4136-8e74-b90346f5f619-kube-api-access-vctb2\") pod \"payload-pre-processing-e2e-worker-w0-2ade09-85459998c9-cksnq\" (UID: \"e7d8237d-903d-4136-8e74-b90346f5f619\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-2ade09-85459998c9-cksnq"
Oct 30 18:46:05.153380 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:05.153329    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e7d8237d-903d-4136-8e74-b90346f5f619-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-2ade09-85459998c9-cksnq\" (UID: \"e7d8237d-903d-4136-8e74-b90346f5f619\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-2ade09-85459998c9-cksnq"
Oct 30 18:46:05.157513 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:05.154445    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e7d8237d-903d-4136-8e74-b90346f5f619-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-2ade09-85459998c9-cksnq\" (UID: \"e7d8237d-903d-4136-8e74-b90346f5f619\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-2ade09-85459998c9-cksnq"
Oct 30 18:46:05.179237 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:05.179203    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-2ade09-867957c796-stmnm"]
Oct 30 18:46:05.179844 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:05.179820    4888 cpu_manager.go:414] "containerMap: removing container" podUID="cf71a4b7-6006-44ba-945c-bb8749af44ad" containerName="istio-proxy"
Oct 30 18:46:05.180000 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:05.179845    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cf71a4b7-6006-44ba-945c-bb8749af44ad" containerName="istio-proxy"
Oct 30 18:46:05.183543 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:05.183523    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-2ade09-867957c796-stmnm"
Oct 30 18:46:05.213404 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:05.213376    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-2ade09-867957c796-stmnm"]
Oct 30 18:46:05.218121 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:05.218090    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vctb2\" (UniqueName: \"kubernetes.io/projected/e7d8237d-903d-4136-8e74-b90346f5f619-kube-api-access-vctb2\") pod \"payload-pre-processing-e2e-worker-w0-2ade09-85459998c9-cksnq\" (UID: \"e7d8237d-903d-4136-8e74-b90346f5f619\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-2ade09-85459998c9-cksnq"
Oct 30 18:46:05.254021 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:05.253995    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/771fbee7-b699-4a2b-a2cd-a2f78bc8e198-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-2ade09-867957c796-stmnm\" (UID: \"771fbee7-b699-4a2b-a2cd-a2f78bc8e198\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-2ade09-867957c796-stmnm"
Oct 30 18:46:05.254146 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:05.254119    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tzmhf\" (UniqueName: \"kubernetes.io/projected/771fbee7-b699-4a2b-a2cd-a2f78bc8e198-kube-api-access-tzmhf\") pod \"payload-processing-e2e-worker-w0-2ade09-867957c796-stmnm\" (UID: \"771fbee7-b699-4a2b-a2cd-a2f78bc8e198\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-2ade09-867957c796-stmnm"
Oct 30 18:46:05.334977 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:05.334953    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-2ade09-85459998c9-cksnq"
Oct 30 18:46:05.355349 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:05.355327    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/771fbee7-b699-4a2b-a2cd-a2f78bc8e198-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-2ade09-867957c796-stmnm\" (UID: \"771fbee7-b699-4a2b-a2cd-a2f78bc8e198\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-2ade09-867957c796-stmnm"
Oct 30 18:46:05.355455 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:05.355440    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tzmhf\" (UniqueName: \"kubernetes.io/projected/771fbee7-b699-4a2b-a2cd-a2f78bc8e198-kube-api-access-tzmhf\") pod \"payload-processing-e2e-worker-w0-2ade09-867957c796-stmnm\" (UID: \"771fbee7-b699-4a2b-a2cd-a2f78bc8e198\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-2ade09-867957c796-stmnm"
Oct 30 18:46:05.356011 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:05.355983    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/771fbee7-b699-4a2b-a2cd-a2f78bc8e198-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-2ade09-867957c796-stmnm\" (UID: \"771fbee7-b699-4a2b-a2cd-a2f78bc8e198\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-2ade09-867957c796-stmnm"
Oct 30 18:46:05.418113 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:05.418013    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tzmhf\" (UniqueName: \"kubernetes.io/projected/771fbee7-b699-4a2b-a2cd-a2f78bc8e198-kube-api-access-tzmhf\") pod \"payload-processing-e2e-worker-w0-2ade09-867957c796-stmnm\" (UID: \"771fbee7-b699-4a2b-a2cd-a2f78bc8e198\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-2ade09-867957c796-stmnm"
Oct 30 18:46:05.496750 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:05.496713    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-2ade09-867957c796-stmnm"
Oct 30 18:46:05.618577 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:46:05.618478    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-jbsw6" containerName="extractor"
Oct 30 18:46:05.710096 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:05.710068    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-2ade09-85459998c9-cksnq"]
Oct 30 18:46:05.711149 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:46:05.711122    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7d8237d_903d_4136_8e74_b90346f5f619.slice/crio-34f3b6512127594436ebf00b2394410c65f99c51a7ab17a1a7606b2f50052761 WatchSource:0}: Error finding container 34f3b6512127594436ebf00b2394410c65f99c51a7ab17a1a7606b2f50052761: Status 404 returned error can't find the container with id 34f3b6512127594436ebf00b2394410c65f99c51a7ab17a1a7606b2f50052761
Oct 30 18:46:05.780663 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:05.780630    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-derive-620f2843-699959648c-2vshr" podUID="3fa35762-2452-4ca7-8b74-25ebda4d8d83" containerName="payload-pre-processing" containerID="cri-o://a92254a2d29ae91e09490996ff6d5728bf531588ce5f518c5b24814eeba8ce80" gracePeriod=25
Oct 30 18:46:05.787406 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:05.787383    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-3b6013-f879fcb57-dx298"]
Oct 30 18:46:05.791049 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:05.791026    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-3b6013-f879fcb57-dx298"
Oct 30 18:46:05.846187 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:05.846156    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w3-3b6013-serving-cert\""
Oct 30 18:46:05.850201 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:05.850173    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-3b6013-f879fcb57-dx298"]
Oct 30 18:46:05.859621 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:05.859597    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hlwhk\" (UniqueName: \"kubernetes.io/projected/1598af0f-006e-4459-9508-75aaa889964d-kube-api-access-hlwhk\") pod \"maas-api-e2e-worker-w3-3b6013-f879fcb57-dx298\" (UID: \"1598af0f-006e-4459-9508-75aaa889964d\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-3b6013-f879fcb57-dx298"
Oct 30 18:46:05.859777 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:05.859662    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1598af0f-006e-4459-9508-75aaa889964d-metrics-certs\") pod \"maas-api-e2e-worker-w3-3b6013-f879fcb57-dx298\" (UID: \"1598af0f-006e-4459-9508-75aaa889964d\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-3b6013-f879fcb57-dx298"
Oct 30 18:46:05.859847 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:05.859790    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1598af0f-006e-4459-9508-75aaa889964d-maas-api-tls\") pod \"maas-api-e2e-worker-w3-3b6013-f879fcb57-dx298\" (UID: \"1598af0f-006e-4459-9508-75aaa889964d\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-3b6013-f879fcb57-dx298"
Oct 30 18:46:05.859847 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:05.859824    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1598af0f-006e-4459-9508-75aaa889964d-openshift-service-ca\") pod \"maas-api-e2e-worker-w3-3b6013-f879fcb57-dx298\" (UID: \"1598af0f-006e-4459-9508-75aaa889964d\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-3b6013-f879fcb57-dx298"
Oct 30 18:46:05.877456 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:05.877434    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-2ade09-867957c796-stmnm"]
Oct 30 18:46:05.879777 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:46:05.879756    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod771fbee7_b699_4a2b_a2cd_a2f78bc8e198.slice/crio-df52c879a2bc5510ee680d79439fc92ae0e2395d6b45a9fdf83a9ef3542faf70 WatchSource:0}: Error finding container df52c879a2bc5510ee680d79439fc92ae0e2395d6b45a9fdf83a9ef3542faf70: Status 404 returned error can't find the container with id df52c879a2bc5510ee680d79439fc92ae0e2395d6b45a9fdf83a9ef3542faf70
Oct 30 18:46:05.961624 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:05.961594    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1598af0f-006e-4459-9508-75aaa889964d-maas-api-tls\") pod \"maas-api-e2e-worker-w3-3b6013-f879fcb57-dx298\" (UID: \"1598af0f-006e-4459-9508-75aaa889964d\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-3b6013-f879fcb57-dx298"
Oct 30 18:46:05.961755 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:05.961658    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1598af0f-006e-4459-9508-75aaa889964d-openshift-service-ca\") pod \"maas-api-e2e-worker-w3-3b6013-f879fcb57-dx298\" (UID: \"1598af0f-006e-4459-9508-75aaa889964d\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-3b6013-f879fcb57-dx298"
Oct 30 18:46:05.961755 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:05.961727    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hlwhk\" (UniqueName: \"kubernetes.io/projected/1598af0f-006e-4459-9508-75aaa889964d-kube-api-access-hlwhk\") pod \"maas-api-e2e-worker-w3-3b6013-f879fcb57-dx298\" (UID: \"1598af0f-006e-4459-9508-75aaa889964d\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-3b6013-f879fcb57-dx298"
Oct 30 18:46:05.961839 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:46:05.961772    4888 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w3-3b6013-serving-cert: secret "maas-api-e2e-worker-w3-3b6013-serving-cert" not found
Oct 30 18:46:05.961839 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:05.961817    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1598af0f-006e-4459-9508-75aaa889964d-metrics-certs\") pod \"maas-api-e2e-worker-w3-3b6013-f879fcb57-dx298\" (UID: \"1598af0f-006e-4459-9508-75aaa889964d\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-3b6013-f879fcb57-dx298"
Oct 30 18:46:05.961923 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:46:05.961843    4888 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1598af0f-006e-4459-9508-75aaa889964d-maas-api-tls podName:1598af0f-006e-4459-9508-75aaa889964d nodeName:}" failed. No retries permitted until 2026-10-30 18:46:06.46182039 +0000 UTC m=+730.709226414 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/1598af0f-006e-4459-9508-75aaa889964d-maas-api-tls") pod "maas-api-e2e-worker-w3-3b6013-f879fcb57-dx298" (UID: "1598af0f-006e-4459-9508-75aaa889964d") : secret "maas-api-e2e-worker-w3-3b6013-serving-cert" not found
Oct 30 18:46:05.963742 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:05.963714    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1598af0f-006e-4459-9508-75aaa889964d-openshift-service-ca\") pod \"maas-api-e2e-worker-w3-3b6013-f879fcb57-dx298\" (UID: \"1598af0f-006e-4459-9508-75aaa889964d\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-3b6013-f879fcb57-dx298"
Oct 30 18:46:05.964754 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:05.964709    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1598af0f-006e-4459-9508-75aaa889964d-metrics-certs\") pod \"maas-api-e2e-worker-w3-3b6013-f879fcb57-dx298\" (UID: \"1598af0f-006e-4459-9508-75aaa889964d\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-3b6013-f879fcb57-dx298"
Oct 30 18:46:06.023255 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:06.023232    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-620f2843-699959648c-2vshr"
Oct 30 18:46:06.025976 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:06.025954    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hlwhk\" (UniqueName: \"kubernetes.io/projected/1598af0f-006e-4459-9508-75aaa889964d-kube-api-access-hlwhk\") pod \"maas-api-e2e-worker-w3-3b6013-f879fcb57-dx298\" (UID: \"1598af0f-006e-4459-9508-75aaa889964d\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-3b6013-f879fcb57-dx298"
Oct 30 18:46:06.062904 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:06.062868    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3fa35762-2452-4ca7-8b74-25ebda4d8d83-kube-api-access-kf6kj\" (UniqueName: \"kubernetes.io/projected/3fa35762-2452-4ca7-8b74-25ebda4d8d83-kube-api-access-kf6kj\") pod \"3fa35762-2452-4ca7-8b74-25ebda4d8d83\" (UID: \"3fa35762-2452-4ca7-8b74-25ebda4d8d83\") "
Oct 30 18:46:06.063032 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:06.062949    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3fa35762-2452-4ca7-8b74-25ebda4d8d83-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3fa35762-2452-4ca7-8b74-25ebda4d8d83-plugins-config-volume\") pod \"3fa35762-2452-4ca7-8b74-25ebda4d8d83\" (UID: \"3fa35762-2452-4ca7-8b74-25ebda4d8d83\") "
Oct 30 18:46:06.063441 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:06.063403    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3fa35762-2452-4ca7-8b74-25ebda4d8d83-plugins-config-volume" pod "3fa35762-2452-4ca7-8b74-25ebda4d8d83" (UID: "3fa35762-2452-4ca7-8b74-25ebda4d8d83"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:46:06.064972 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:06.064948    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3fa35762-2452-4ca7-8b74-25ebda4d8d83-kube-api-access-kf6kj" pod "3fa35762-2452-4ca7-8b74-25ebda4d8d83" (UID: "3fa35762-2452-4ca7-8b74-25ebda4d8d83"). InnerVolumeSpecName "kube-api-access-kf6kj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:46:06.163910 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:06.163841    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kf6kj\" (UniqueName: \"kubernetes.io/projected/3fa35762-2452-4ca7-8b74-25ebda4d8d83-kube-api-access-kf6kj\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:46:06.163910 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:06.163867    4888 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3fa35762-2452-4ca7-8b74-25ebda4d8d83-plugins-config-volume\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:46:06.257932 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:06.257901    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-2ade09-85459998c9-cksnq" event={"ID":"e7d8237d-903d-4136-8e74-b90346f5f619","Type":"ContainerStarted","Data":"1d49bc4b6f14c5398972d57b8d01621d3e0c5c7cb11d650bb459c60fd18eb8b4"}
Oct 30 18:46:06.258128 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:06.257950    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-2ade09-85459998c9-cksnq" event={"ID":"e7d8237d-903d-4136-8e74-b90346f5f619","Type":"ContainerStarted","Data":"34f3b6512127594436ebf00b2394410c65f99c51a7ab17a1a7606b2f50052761"}
Oct 30 18:46:06.258128 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:06.258028    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-2ade09-85459998c9-cksnq"
Oct 30 18:46:06.259342 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:06.259319    4888 generic.go:356] "Generic (PLEG): container finished" podID="3fa35762-2452-4ca7-8b74-25ebda4d8d83" containerID="a92254a2d29ae91e09490996ff6d5728bf531588ce5f518c5b24814eeba8ce80" exitCode=0
Oct 30 18:46:06.259478 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:06.259347    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-620f2843-699959648c-2vshr" event={"ID":"3fa35762-2452-4ca7-8b74-25ebda4d8d83","Type":"ContainerDied","Data":"a92254a2d29ae91e09490996ff6d5728bf531588ce5f518c5b24814eeba8ce80"}
Oct 30 18:46:06.259478 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:06.259360    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-620f2843-699959648c-2vshr"
Oct 30 18:46:06.259478 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:06.259393    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-620f2843-699959648c-2vshr" event={"ID":"3fa35762-2452-4ca7-8b74-25ebda4d8d83","Type":"ContainerDied","Data":"5aa7f7c6972ac07fa28339f1ce87958df8757ce60638097b92fe80e0bec31e5c"}
Oct 30 18:46:06.259478 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:06.259422    4888 scope.go:122] "RemoveContainer" containerID="a92254a2d29ae91e09490996ff6d5728bf531588ce5f518c5b24814eeba8ce80"
Oct 30 18:46:06.261357 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:06.261315    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-2ade09-867957c796-stmnm" event={"ID":"771fbee7-b699-4a2b-a2cd-a2f78bc8e198","Type":"ContainerStarted","Data":"57f9536cea451c7dce5cf63b5237a082999a01f7b392219964ffdd7b9a156620"}
Oct 30 18:46:06.261460 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:06.261374    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-2ade09-867957c796-stmnm" event={"ID":"771fbee7-b699-4a2b-a2cd-a2f78bc8e198","Type":"ContainerStarted","Data":"df52c879a2bc5510ee680d79439fc92ae0e2395d6b45a9fdf83a9ef3542faf70"}
Oct 30 18:46:06.261460 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:06.261404    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w0-2ade09-867957c796-stmnm"
Oct 30 18:46:06.269488 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:06.269473    4888 scope.go:122] "RemoveContainer" containerID="a92254a2d29ae91e09490996ff6d5728bf531588ce5f518c5b24814eeba8ce80"
Oct 30 18:46:06.269737 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:46:06.269720    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a92254a2d29ae91e09490996ff6d5728bf531588ce5f518c5b24814eeba8ce80\": container with ID starting with a92254a2d29ae91e09490996ff6d5728bf531588ce5f518c5b24814eeba8ce80 not found: ID does not exist" containerID="a92254a2d29ae91e09490996ff6d5728bf531588ce5f518c5b24814eeba8ce80"
Oct 30 18:46:06.269777 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:06.269744    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a92254a2d29ae91e09490996ff6d5728bf531588ce5f518c5b24814eeba8ce80"} err="failed to get container status \"a92254a2d29ae91e09490996ff6d5728bf531588ce5f518c5b24814eeba8ce80\": rpc error: code = NotFound desc = could not find container \"a92254a2d29ae91e09490996ff6d5728bf531588ce5f518c5b24814eeba8ce80\": container with ID starting with a92254a2d29ae91e09490996ff6d5728bf531588ce5f518c5b24814eeba8ce80 not found: ID does not exist"
Oct 30 18:46:06.376883 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:06.376832    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-2ade09-85459998c9-cksnq" podStartSLOduration=2.376818482 podStartE2EDuration="2.376818482s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:46:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:46:06.376172157 +0000 UTC m=+730.623578182" watchObservedRunningTime="2026-10-30 18:46:06.376818482 +0000 UTC m=+730.624224554"
Oct 30 18:46:06.466535 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:06.466444    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1598af0f-006e-4459-9508-75aaa889964d-maas-api-tls\") pod \"maas-api-e2e-worker-w3-3b6013-f879fcb57-dx298\" (UID: \"1598af0f-006e-4459-9508-75aaa889964d\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-3b6013-f879fcb57-dx298"
Oct 30 18:46:06.469006 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:06.468982    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1598af0f-006e-4459-9508-75aaa889964d-maas-api-tls\") pod \"maas-api-e2e-worker-w3-3b6013-f879fcb57-dx298\" (UID: \"1598af0f-006e-4459-9508-75aaa889964d\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-3b6013-f879fcb57-dx298"
Oct 30 18:46:06.491097 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:06.491051    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-620f2843-699959648c-2vshr"]
Oct 30 18:46:06.498116 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:06.498086    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-620f2843-699959648c-2vshr"]
Oct 30 18:46:06.615110 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:06.615057    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w0-2ade09-867957c796-stmnm" podStartSLOduration=1.6150435440000002 podStartE2EDuration="1.615043544s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:46:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:46:06.612495837 +0000 UTC m=+730.859901867" watchObservedRunningTime="2026-10-30 18:46:06.615043544 +0000 UTC m=+730.862449583"
Oct 30 18:46:06.767369 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:06.767258    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-3b6013-f879fcb57-dx298"
Oct 30 18:46:07.147959 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:07.147930    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-3b6013-f879fcb57-dx298"]
Oct 30 18:46:07.148966 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:46:07.148944    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1598af0f_006e_4459_9508_75aaa889964d.slice/crio-0ba924fdb28ab90637deca8de429f5046210824f0ddae92f76f02fbd41bca26b WatchSource:0}: Error finding container 0ba924fdb28ab90637deca8de429f5046210824f0ddae92f76f02fbd41bca26b: Status 404 returned error can't find the container with id 0ba924fdb28ab90637deca8de429f5046210824f0ddae92f76f02fbd41bca26b
Oct 30 18:46:07.268197 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:07.268161    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-3b6013-f879fcb57-dx298" event={"ID":"1598af0f-006e-4459-9508-75aaa889964d","Type":"ContainerStarted","Data":"0ba924fdb28ab90637deca8de429f5046210824f0ddae92f76f02fbd41bca26b"}
Oct 30 18:46:07.891303 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:07.891249    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3fa35762-2452-4ca7-8b74-25ebda4d8d83" path="/var/lib/kubelet/pods/3fa35762-2452-4ca7-8b74-25ebda4d8d83/volumes"
Oct 30 18:46:08.277348 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:08.277316    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-3b6013-f879fcb57-dx298" event={"ID":"1598af0f-006e-4459-9508-75aaa889964d","Type":"ContainerStarted","Data":"4573630803e53259ab91a8724d49972edf0628652ee2a03d23e4d8254001d0b6"}
Oct 30 18:46:08.277768 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:08.277456    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-3b6013-f879fcb57-dx298"
Oct 30 18:46:08.399542 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:08.399456    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-3b6013-f879fcb57-dx298" podStartSLOduration=2.453574343 podStartE2EDuration="3.399441327s" totalImagesPullingTime="945.866984ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:46:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:46:08.398480231 +0000 UTC m=+732.645886255" watchObservedRunningTime="2026-10-30 18:46:08.399441327 +0000 UTC m=+732.646847374"
Oct 30 18:46:10.215903 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:10.215874    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w3-3b6013-84b5f84d8b-hfxgw"
Oct 30 18:46:10.216329 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:10.216042    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-3b6013-57fbf8ccf5-bz979"
Oct 30 18:46:11.166327 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:11.161085    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-2ade09-d8b4fccb4-mjj8l"]
Oct 30 18:46:11.166327 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:11.162248    4888 cpu_manager.go:414] "containerMap: removing container" podUID="3fa35762-2452-4ca7-8b74-25ebda4d8d83" containerName="payload-pre-processing"
Oct 30 18:46:11.166327 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:11.162268    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3fa35762-2452-4ca7-8b74-25ebda4d8d83" containerName="payload-pre-processing"
Oct 30 18:46:11.166327 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:11.162613    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="3fa35762-2452-4ca7-8b74-25ebda4d8d83" containerName="payload-pre-processing"
Oct 30 18:46:11.168679 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:11.168657    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-2ade09-d8b4fccb4-mjj8l"
Oct 30 18:46:11.216816 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:11.216788    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ddf8afd2-ed32-44bb-87ce-c26cde475f08-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-2ade09-d8b4fccb4-mjj8l\" (UID: \"ddf8afd2-ed32-44bb-87ce-c26cde475f08\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-2ade09-d8b4fccb4-mjj8l"
Oct 30 18:46:11.217173 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:11.216838    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cx52k\" (UniqueName: \"kubernetes.io/projected/ddf8afd2-ed32-44bb-87ce-c26cde475f08-kube-api-access-cx52k\") pod \"maas-api-e2e-worker-w0-2ade09-d8b4fccb4-mjj8l\" (UID: \"ddf8afd2-ed32-44bb-87ce-c26cde475f08\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-2ade09-d8b4fccb4-mjj8l"
Oct 30 18:46:11.217173 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:11.216872    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ddf8afd2-ed32-44bb-87ce-c26cde475f08-metrics-certs\") pod \"maas-api-e2e-worker-w0-2ade09-d8b4fccb4-mjj8l\" (UID: \"ddf8afd2-ed32-44bb-87ce-c26cde475f08\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-2ade09-d8b4fccb4-mjj8l"
Oct 30 18:46:11.217173 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:11.217011    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ddf8afd2-ed32-44bb-87ce-c26cde475f08-maas-api-tls\") pod \"maas-api-e2e-worker-w0-2ade09-d8b4fccb4-mjj8l\" (UID: \"ddf8afd2-ed32-44bb-87ce-c26cde475f08\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-2ade09-d8b4fccb4-mjj8l"
Oct 30 18:46:11.224468 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:11.224443    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-2ade09-d8b4fccb4-mjj8l"]
Oct 30 18:46:11.275397 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:11.275369    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w0-2ade09-serving-cert\""
Oct 30 18:46:11.277230 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:11.277205    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-rjmdz"]
Oct 30 18:46:11.317944 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:11.317917    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ddf8afd2-ed32-44bb-87ce-c26cde475f08-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-2ade09-d8b4fccb4-mjj8l\" (UID: \"ddf8afd2-ed32-44bb-87ce-c26cde475f08\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-2ade09-d8b4fccb4-mjj8l"
Oct 30 18:46:11.318109 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:11.317956    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cx52k\" (UniqueName: \"kubernetes.io/projected/ddf8afd2-ed32-44bb-87ce-c26cde475f08-kube-api-access-cx52k\") pod \"maas-api-e2e-worker-w0-2ade09-d8b4fccb4-mjj8l\" (UID: \"ddf8afd2-ed32-44bb-87ce-c26cde475f08\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-2ade09-d8b4fccb4-mjj8l"
Oct 30 18:46:11.318109 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:11.318082    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ddf8afd2-ed32-44bb-87ce-c26cde475f08-metrics-certs\") pod \"maas-api-e2e-worker-w0-2ade09-d8b4fccb4-mjj8l\" (UID: \"ddf8afd2-ed32-44bb-87ce-c26cde475f08\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-2ade09-d8b4fccb4-mjj8l"
Oct 30 18:46:11.318234 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:11.318211    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ddf8afd2-ed32-44bb-87ce-c26cde475f08-maas-api-tls\") pod \"maas-api-e2e-worker-w0-2ade09-d8b4fccb4-mjj8l\" (UID: \"ddf8afd2-ed32-44bb-87ce-c26cde475f08\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-2ade09-d8b4fccb4-mjj8l"
Oct 30 18:46:11.318390 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:46:11.318366    4888 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w0-2ade09-serving-cert: secret "maas-api-e2e-worker-w0-2ade09-serving-cert" not found
Oct 30 18:46:11.318553 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:46:11.318467    4888 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ddf8afd2-ed32-44bb-87ce-c26cde475f08-maas-api-tls podName:ddf8afd2-ed32-44bb-87ce-c26cde475f08 nodeName:}" failed. No retries permitted until 2026-10-30 18:46:11.818447109 +0000 UTC m=+736.065853130 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/ddf8afd2-ed32-44bb-87ce-c26cde475f08-maas-api-tls") pod "maas-api-e2e-worker-w0-2ade09-d8b4fccb4-mjj8l" (UID: "ddf8afd2-ed32-44bb-87ce-c26cde475f08") : secret "maas-api-e2e-worker-w0-2ade09-serving-cert" not found
Oct 30 18:46:11.318684 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:11.318667    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ddf8afd2-ed32-44bb-87ce-c26cde475f08-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-2ade09-d8b4fccb4-mjj8l\" (UID: \"ddf8afd2-ed32-44bb-87ce-c26cde475f08\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-2ade09-d8b4fccb4-mjj8l"
Oct 30 18:46:11.320661 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:11.320642    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ddf8afd2-ed32-44bb-87ce-c26cde475f08-metrics-certs\") pod \"maas-api-e2e-worker-w0-2ade09-d8b4fccb4-mjj8l\" (UID: \"ddf8afd2-ed32-44bb-87ce-c26cde475f08\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-2ade09-d8b4fccb4-mjj8l"
Oct 30 18:46:11.382798 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:11.382777    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cx52k\" (UniqueName: \"kubernetes.io/projected/ddf8afd2-ed32-44bb-87ce-c26cde475f08-kube-api-access-cx52k\") pod \"maas-api-e2e-worker-w0-2ade09-d8b4fccb4-mjj8l\" (UID: \"ddf8afd2-ed32-44bb-87ce-c26cde475f08\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-2ade09-d8b4fccb4-mjj8l"
Oct 30 18:46:11.824467 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:11.824435    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ddf8afd2-ed32-44bb-87ce-c26cde475f08-maas-api-tls\") pod \"maas-api-e2e-worker-w0-2ade09-d8b4fccb4-mjj8l\" (UID: \"ddf8afd2-ed32-44bb-87ce-c26cde475f08\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-2ade09-d8b4fccb4-mjj8l"
Oct 30 18:46:11.826953 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:11.826928    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ddf8afd2-ed32-44bb-87ce-c26cde475f08-maas-api-tls\") pod \"maas-api-e2e-worker-w0-2ade09-d8b4fccb4-mjj8l\" (UID: \"ddf8afd2-ed32-44bb-87ce-c26cde475f08\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-2ade09-d8b4fccb4-mjj8l"
Oct 30 18:46:12.082160 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:12.082080    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-2ade09-d8b4fccb4-mjj8l"
Oct 30 18:46:12.463479 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:12.463450    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-2ade09-d8b4fccb4-mjj8l"]
Oct 30 18:46:12.464423 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:46:12.464401    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podddf8afd2_ed32_44bb_87ce_c26cde475f08.slice/crio-dc6565fc2b862e5daaa7dd9e5b5e2669eccfeaf90bef568b8b5554499dcd3ea2 WatchSource:0}: Error finding container dc6565fc2b862e5daaa7dd9e5b5e2669eccfeaf90bef568b8b5554499dcd3ea2: Status 404 returned error can't find the container with id dc6565fc2b862e5daaa7dd9e5b5e2669eccfeaf90bef568b8b5554499dcd3ea2
Oct 30 18:46:13.309032 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:13.308997    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-2ade09-d8b4fccb4-mjj8l" event={"ID":"ddf8afd2-ed32-44bb-87ce-c26cde475f08","Type":"ContainerStarted","Data":"dc6565fc2b862e5daaa7dd9e5b5e2669eccfeaf90bef568b8b5554499dcd3ea2"}
Oct 30 18:46:14.290438 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:14.290414    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-3b6013-f879fcb57-dx298"
Oct 30 18:46:14.317052 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:14.317025    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-2ade09-d8b4fccb4-mjj8l" event={"ID":"ddf8afd2-ed32-44bb-87ce-c26cde475f08","Type":"ContainerStarted","Data":"8108d78fc0d001365fb731a353b144c815deac503635bf95ff6d36870d902edb"}
Oct 30 18:46:14.317191 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:14.317072    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-2ade09-d8b4fccb4-mjj8l"
Oct 30 18:46:14.533334 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:14.533265    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-2ade09-d8b4fccb4-mjj8l" podStartSLOduration=2.663729409 podStartE2EDuration="3.533253098s" totalImagesPullingTime="869.523689ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:46:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:46:14.532371875 +0000 UTC m=+738.779777899" watchObservedRunningTime="2026-10-30 18:46:14.533253098 +0000 UTC m=+738.780659124"
Oct 30 18:46:17.272498 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:17.272468    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w0-2ade09-867957c796-stmnm"
Oct 30 18:46:17.272887 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:17.272532    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-2ade09-85459998c9-cksnq"
Oct 30 18:46:20.329304 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:20.329251    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-2ade09-d8b4fccb4-mjj8l"
Oct 30 18:46:21.101578 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:21.101545    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-620f2843-fd7f7f68f-vt65m"]
Oct 30 18:46:21.101802 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:21.101760    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-620f2843-fd7f7f68f-vt65m" podUID="790c0c38-1b9e-4fbc-b765-a7d280ac665c" containerName="maas-api" containerID="cri-o://4f145df9c33b492e96215dc4eee656f62eb551a3713a5f9aefdd2774a2767123" gracePeriod=30
Oct 30 18:46:21.359795 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:21.359726    4888 generic.go:356] "Generic (PLEG): container finished" podID="790c0c38-1b9e-4fbc-b765-a7d280ac665c" containerID="4f145df9c33b492e96215dc4eee656f62eb551a3713a5f9aefdd2774a2767123" exitCode=0
Oct 30 18:46:21.360119 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:21.359797    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-620f2843-fd7f7f68f-vt65m" event={"ID":"790c0c38-1b9e-4fbc-b765-a7d280ac665c","Type":"ContainerDied","Data":"4f145df9c33b492e96215dc4eee656f62eb551a3713a5f9aefdd2774a2767123"}
Oct 30 18:46:21.360119 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:21.359862    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-620f2843-fd7f7f68f-vt65m" event={"ID":"790c0c38-1b9e-4fbc-b765-a7d280ac665c","Type":"ContainerDied","Data":"d6c4c1a7886150f9066aef93078c6692fb540c5ac4b779e8e0cfcc90164a7e3e"}
Oct 30 18:46:21.360119 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:21.359890    4888 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d6c4c1a7886150f9066aef93078c6692fb540c5ac4b779e8e0cfcc90164a7e3e"
Oct 30 18:46:21.367065 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:21.367047    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-620f2843-fd7f7f68f-vt65m"
Oct 30 18:46:21.441845 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:21.441811    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/790c0c38-1b9e-4fbc-b765-a7d280ac665c-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/790c0c38-1b9e-4fbc-b765-a7d280ac665c-maas-api-tls\") pod \"790c0c38-1b9e-4fbc-b765-a7d280ac665c\" (UID: \"790c0c38-1b9e-4fbc-b765-a7d280ac665c\") "
Oct 30 18:46:21.441999 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:21.441982    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/790c0c38-1b9e-4fbc-b765-a7d280ac665c-kube-api-access-whp5r\" (UniqueName: \"kubernetes.io/projected/790c0c38-1b9e-4fbc-b765-a7d280ac665c-kube-api-access-whp5r\") pod \"790c0c38-1b9e-4fbc-b765-a7d280ac665c\" (UID: \"790c0c38-1b9e-4fbc-b765-a7d280ac665c\") "
Oct 30 18:46:21.442039 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:21.442024    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/790c0c38-1b9e-4fbc-b765-a7d280ac665c-metrics-certs\" (UniqueName: \"kubernetes.io/secret/790c0c38-1b9e-4fbc-b765-a7d280ac665c-metrics-certs\") pod \"790c0c38-1b9e-4fbc-b765-a7d280ac665c\" (UID: \"790c0c38-1b9e-4fbc-b765-a7d280ac665c\") "
Oct 30 18:46:21.442129 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:21.442103    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/790c0c38-1b9e-4fbc-b765-a7d280ac665c-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/790c0c38-1b9e-4fbc-b765-a7d280ac665c-openshift-service-ca\") pod \"790c0c38-1b9e-4fbc-b765-a7d280ac665c\" (UID: \"790c0c38-1b9e-4fbc-b765-a7d280ac665c\") "
Oct 30 18:46:21.442513 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:21.442483    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/790c0c38-1b9e-4fbc-b765-a7d280ac665c-openshift-service-ca" pod "790c0c38-1b9e-4fbc-b765-a7d280ac665c" (UID: "790c0c38-1b9e-4fbc-b765-a7d280ac665c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:46:21.444176 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:21.444142    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/790c0c38-1b9e-4fbc-b765-a7d280ac665c-maas-api-tls" pod "790c0c38-1b9e-4fbc-b765-a7d280ac665c" (UID: "790c0c38-1b9e-4fbc-b765-a7d280ac665c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:46:21.444268 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:21.444185    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/790c0c38-1b9e-4fbc-b765-a7d280ac665c-kube-api-access-whp5r" pod "790c0c38-1b9e-4fbc-b765-a7d280ac665c" (UID: "790c0c38-1b9e-4fbc-b765-a7d280ac665c"). InnerVolumeSpecName "kube-api-access-whp5r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:46:21.444268 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:21.444203    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/790c0c38-1b9e-4fbc-b765-a7d280ac665c-metrics-certs" pod "790c0c38-1b9e-4fbc-b765-a7d280ac665c" (UID: "790c0c38-1b9e-4fbc-b765-a7d280ac665c"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:46:21.543948 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:21.543916    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-whp5r\" (UniqueName: \"kubernetes.io/projected/790c0c38-1b9e-4fbc-b765-a7d280ac665c-kube-api-access-whp5r\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:46:21.543948 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:21.543944    4888 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/790c0c38-1b9e-4fbc-b765-a7d280ac665c-metrics-certs\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:46:21.543948 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:21.543955    4888 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/790c0c38-1b9e-4fbc-b765-a7d280ac665c-openshift-service-ca\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:46:21.544182 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:21.543964    4888 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/790c0c38-1b9e-4fbc-b765-a7d280ac665c-maas-api-tls\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:46:22.188319 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:22.188216    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-e7929d-7cbc876b55-5mnbp"]
Oct 30 18:46:22.188779 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:22.188760    4888 cpu_manager.go:414] "containerMap: removing container" podUID="790c0c38-1b9e-4fbc-b765-a7d280ac665c" containerName="maas-api"
Oct 30 18:46:22.188883 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:22.188782    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="790c0c38-1b9e-4fbc-b765-a7d280ac665c" containerName="maas-api"
Oct 30 18:46:22.188970 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:22.188956    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="790c0c38-1b9e-4fbc-b765-a7d280ac665c" containerName="maas-api"
Oct 30 18:46:22.192271 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:22.192251    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-e7929d-7cbc876b55-5mnbp"
Oct 30 18:46:22.249482 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:22.249235    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w4-e7929d-dockercfg-djtfh\""
Oct 30 18:46:22.249803 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:22.249718    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w4-e7929d\""
Oct 30 18:46:22.250898 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:22.250870    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q6s2j\" (UniqueName: \"kubernetes.io/projected/b13a4e48-bc35-4bff-9840-640c9a1d6c92-kube-api-access-q6s2j\") pod \"payload-pre-processing-e2e-worker-w4-e7929d-7cbc876b55-5mnbp\" (UID: \"b13a4e48-bc35-4bff-9840-640c9a1d6c92\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-e7929d-7cbc876b55-5mnbp"
Oct 30 18:46:22.251094 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:22.251067    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b13a4e48-bc35-4bff-9840-640c9a1d6c92-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w4-e7929d-7cbc876b55-5mnbp\" (UID: \"b13a4e48-bc35-4bff-9840-640c9a1d6c92\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-e7929d-7cbc876b55-5mnbp"
Oct 30 18:46:22.253268 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:22.253248    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-e7929d-7cbc876b55-5mnbp"]
Oct 30 18:46:22.351964 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:22.351918    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q6s2j\" (UniqueName: \"kubernetes.io/projected/b13a4e48-bc35-4bff-9840-640c9a1d6c92-kube-api-access-q6s2j\") pod \"payload-pre-processing-e2e-worker-w4-e7929d-7cbc876b55-5mnbp\" (UID: \"b13a4e48-bc35-4bff-9840-640c9a1d6c92\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-e7929d-7cbc876b55-5mnbp"
Oct 30 18:46:22.352340 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:22.352309    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b13a4e48-bc35-4bff-9840-640c9a1d6c92-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w4-e7929d-7cbc876b55-5mnbp\" (UID: \"b13a4e48-bc35-4bff-9840-640c9a1d6c92\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-e7929d-7cbc876b55-5mnbp"
Oct 30 18:46:22.353430 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:22.353405    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b13a4e48-bc35-4bff-9840-640c9a1d6c92-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w4-e7929d-7cbc876b55-5mnbp\" (UID: \"b13a4e48-bc35-4bff-9840-640c9a1d6c92\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-e7929d-7cbc876b55-5mnbp"
Oct 30 18:46:22.368155 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:22.368119    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-620f2843-fd7f7f68f-vt65m"
Oct 30 18:46:22.416169 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:22.416138    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q6s2j\" (UniqueName: \"kubernetes.io/projected/b13a4e48-bc35-4bff-9840-640c9a1d6c92-kube-api-access-q6s2j\") pod \"payload-pre-processing-e2e-worker-w4-e7929d-7cbc876b55-5mnbp\" (UID: \"b13a4e48-bc35-4bff-9840-640c9a1d6c92\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-e7929d-7cbc876b55-5mnbp"
Oct 30 18:46:22.505875 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:22.505795    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-e7929d-7cbc876b55-5mnbp"
Oct 30 18:46:22.536896 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:22.536865    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-620f2843-fd7f7f68f-vt65m"]
Oct 30 18:46:22.540952 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:22.540927    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-620f2843-fd7f7f68f-vt65m"]
Oct 30 18:46:22.884104 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:22.882269    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-e7929d-7cbc876b55-5mnbp"]
Oct 30 18:46:22.892455 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:46:22.892421    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb13a4e48_bc35_4bff_9840_640c9a1d6c92.slice/crio-066e40aad7bdb3a4aa680993bcd5bb5f1a6875ecfd4627f798997ac30490f660 WatchSource:0}: Error finding container 066e40aad7bdb3a4aa680993bcd5bb5f1a6875ecfd4627f798997ac30490f660: Status 404 returned error can't find the container with id 066e40aad7bdb3a4aa680993bcd5bb5f1a6875ecfd4627f798997ac30490f660
Oct 30 18:46:23.375763 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:23.375734    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-e7929d-7cbc876b55-5mnbp" event={"ID":"b13a4e48-bc35-4bff-9840-640c9a1d6c92","Type":"ContainerStarted","Data":"90d3b1437af10ab33425c776de11ff80083fde4297aa09d707d7a668ad21d8c6"}
Oct 30 18:46:23.376125 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:23.375789    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-e7929d-7cbc876b55-5mnbp" event={"ID":"b13a4e48-bc35-4bff-9840-640c9a1d6c92","Type":"ContainerStarted","Data":"066e40aad7bdb3a4aa680993bcd5bb5f1a6875ecfd4627f798997ac30490f660"}
Oct 30 18:46:23.376125 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:23.375814    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-e7929d-7cbc876b55-5mnbp"
Oct 30 18:46:23.515104 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:23.515040    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-e7929d-7cbc876b55-5mnbp" podStartSLOduration=1.5150206750000001 podStartE2EDuration="1.515020675s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:46:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:46:23.513175536 +0000 UTC m=+747.760581563" watchObservedRunningTime="2026-10-30 18:46:23.515020675 +0000 UTC m=+747.762426701"
Oct 30 18:46:23.897345 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:23.897314    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="790c0c38-1b9e-4fbc-b765-a7d280ac665c" path="/var/lib/kubelet/pods/790c0c38-1b9e-4fbc-b765-a7d280ac665c/volumes"
Oct 30 18:46:24.760148 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:24.760118    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-msdwn"]
Oct 30 18:46:24.764140 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:24.764126    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-msdwn"
Oct 30 18:46:24.826609 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:24.826573    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w1-fd61e3-openshift-default-dockercfg-tn6nh\""
Oct 30 18:46:24.879115 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:24.879080    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f49068b0-4dea-487c-a31c-c61704119dad-workload-socket\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-msdwn\" (UID: \"f49068b0-4dea-487c-a31c-c61704119dad\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-msdwn"
Oct 30 18:46:24.879115 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:24.879111    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f49068b0-4dea-487c-a31c-c61704119dad-istio-data\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-msdwn\" (UID: \"f49068b0-4dea-487c-a31c-c61704119dad\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-msdwn"
Oct 30 18:46:24.879395 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:24.879134    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f49068b0-4dea-487c-a31c-c61704119dad-workload-certs\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-msdwn\" (UID: \"f49068b0-4dea-487c-a31c-c61704119dad\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-msdwn"
Oct 30 18:46:24.879395 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:24.879195    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f49068b0-4dea-487c-a31c-c61704119dad-istio-podinfo\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-msdwn\" (UID: \"f49068b0-4dea-487c-a31c-c61704119dad\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-msdwn"
Oct 30 18:46:24.879395 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:24.879367    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f49068b0-4dea-487c-a31c-c61704119dad-istiod-ca-cert\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-msdwn\" (UID: \"f49068b0-4dea-487c-a31c-c61704119dad\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-msdwn"
Oct 30 18:46:24.879547 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:24.879401    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f49068b0-4dea-487c-a31c-c61704119dad-istio-token\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-msdwn\" (UID: \"f49068b0-4dea-487c-a31c-c61704119dad\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-msdwn"
Oct 30 18:46:24.879547 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:24.879466    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f49068b0-4dea-487c-a31c-c61704119dad-istio-envoy\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-msdwn\" (UID: \"f49068b0-4dea-487c-a31c-c61704119dad\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-msdwn"
Oct 30 18:46:24.879641 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:24.879546    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f49068b0-4dea-487c-a31c-c61704119dad-credential-socket\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-msdwn\" (UID: \"f49068b0-4dea-487c-a31c-c61704119dad\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-msdwn"
Oct 30 18:46:24.879641 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:24.879566    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-msdwn"]
Oct 30 18:46:24.879641 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:24.879626    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wm52q\" (UniqueName: \"kubernetes.io/projected/f49068b0-4dea-487c-a31c-c61704119dad-kube-api-access-wm52q\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-msdwn\" (UID: \"f49068b0-4dea-487c-a31c-c61704119dad\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-msdwn"
Oct 30 18:46:24.980764 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:24.980727    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f49068b0-4dea-487c-a31c-c61704119dad-istio-podinfo\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-msdwn\" (UID: \"f49068b0-4dea-487c-a31c-c61704119dad\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-msdwn"
Oct 30 18:46:24.980977 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:24.980790    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f49068b0-4dea-487c-a31c-c61704119dad-istiod-ca-cert\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-msdwn\" (UID: \"f49068b0-4dea-487c-a31c-c61704119dad\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-msdwn"
Oct 30 18:46:24.980977 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:24.980820    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f49068b0-4dea-487c-a31c-c61704119dad-istio-token\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-msdwn\" (UID: \"f49068b0-4dea-487c-a31c-c61704119dad\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-msdwn"
Oct 30 18:46:24.980977 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:24.980877    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f49068b0-4dea-487c-a31c-c61704119dad-istio-envoy\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-msdwn\" (UID: \"f49068b0-4dea-487c-a31c-c61704119dad\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-msdwn"
Oct 30 18:46:24.981159 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:24.980990    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f49068b0-4dea-487c-a31c-c61704119dad-credential-socket\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-msdwn\" (UID: \"f49068b0-4dea-487c-a31c-c61704119dad\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-msdwn"
Oct 30 18:46:24.981159 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:24.981037    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wm52q\" (UniqueName: \"kubernetes.io/projected/f49068b0-4dea-487c-a31c-c61704119dad-kube-api-access-wm52q\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-msdwn\" (UID: \"f49068b0-4dea-487c-a31c-c61704119dad\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-msdwn"
Oct 30 18:46:24.981159 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:24.981104    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f49068b0-4dea-487c-a31c-c61704119dad-workload-socket\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-msdwn\" (UID: \"f49068b0-4dea-487c-a31c-c61704119dad\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-msdwn"
Oct 30 18:46:24.981159 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:24.981130    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f49068b0-4dea-487c-a31c-c61704119dad-istio-data\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-msdwn\" (UID: \"f49068b0-4dea-487c-a31c-c61704119dad\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-msdwn"
Oct 30 18:46:24.981395 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:24.981166    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f49068b0-4dea-487c-a31c-c61704119dad-workload-certs\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-msdwn\" (UID: \"f49068b0-4dea-487c-a31c-c61704119dad\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-msdwn"
Oct 30 18:46:24.981561 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:24.981534    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f49068b0-4dea-487c-a31c-c61704119dad-credential-socket\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-msdwn\" (UID: \"f49068b0-4dea-487c-a31c-c61704119dad\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-msdwn"
Oct 30 18:46:24.981790 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:24.981595    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f49068b0-4dea-487c-a31c-c61704119dad-workload-certs\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-msdwn\" (UID: \"f49068b0-4dea-487c-a31c-c61704119dad\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-msdwn"
Oct 30 18:46:24.981790 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:24.981619    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f49068b0-4dea-487c-a31c-c61704119dad-istio-data\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-msdwn\" (UID: \"f49068b0-4dea-487c-a31c-c61704119dad\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-msdwn"
Oct 30 18:46:24.981790 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:24.981656    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f49068b0-4dea-487c-a31c-c61704119dad-workload-socket\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-msdwn\" (UID: \"f49068b0-4dea-487c-a31c-c61704119dad\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-msdwn"
Oct 30 18:46:24.982024 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:24.981999    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f49068b0-4dea-487c-a31c-c61704119dad-istiod-ca-cert\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-msdwn\" (UID: \"f49068b0-4dea-487c-a31c-c61704119dad\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-msdwn"
Oct 30 18:46:24.983417 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:24.983394    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f49068b0-4dea-487c-a31c-c61704119dad-istio-envoy\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-msdwn\" (UID: \"f49068b0-4dea-487c-a31c-c61704119dad\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-msdwn"
Oct 30 18:46:24.983731 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:24.983712    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f49068b0-4dea-487c-a31c-c61704119dad-istio-podinfo\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-msdwn\" (UID: \"f49068b0-4dea-487c-a31c-c61704119dad\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-msdwn"
Oct 30 18:46:25.045388 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:25.045314    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f49068b0-4dea-487c-a31c-c61704119dad-istio-token\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-msdwn\" (UID: \"f49068b0-4dea-487c-a31c-c61704119dad\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-msdwn"
Oct 30 18:46:25.045512 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:25.045386    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wm52q\" (UniqueName: \"kubernetes.io/projected/f49068b0-4dea-487c-a31c-c61704119dad-kube-api-access-wm52q\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-msdwn\" (UID: \"f49068b0-4dea-487c-a31c-c61704119dad\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-msdwn"
Oct 30 18:46:25.077006 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:25.076969    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-msdwn"
Oct 30 18:46:25.512379 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:25.512352    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-msdwn"]
Oct 30 18:46:25.513427 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:46:25.513386    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf49068b0_4dea_487c_a31c_c61704119dad.slice/crio-504fb43091830fbd5741e6969177104c225dc833e9ec65e997280cf50f627901 WatchSource:0}: Error finding container 504fb43091830fbd5741e6969177104c225dc833e9ec65e997280cf50f627901: Status 404 returned error can't find the container with id 504fb43091830fbd5741e6969177104c225dc833e9ec65e997280cf50f627901
Oct 30 18:46:25.515898 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:25.515864    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:46:25.515998 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:25.515950    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:46:25.515998 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:25.515985    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:46:26.398579 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:26.398543    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-msdwn" event={"ID":"f49068b0-4dea-487c-a31c-c61704119dad","Type":"ContainerStarted","Data":"f7e4b2330162644e18b81c185c852390d98e3ec3f146b0a5e16ea2db1a8e781c"}
Oct 30 18:46:26.398986 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:26.398596    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-msdwn" event={"ID":"f49068b0-4dea-487c-a31c-c61704119dad","Type":"ContainerStarted","Data":"504fb43091830fbd5741e6969177104c225dc833e9ec65e997280cf50f627901"}
Oct 30 18:46:26.573919 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:26.573872    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-msdwn" podStartSLOduration=2.573858445 podStartE2EDuration="2.573858445s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:46:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:46:26.571142023 +0000 UTC m=+750.818548072" watchObservedRunningTime="2026-10-30 18:46:26.573858445 +0000 UTC m=+750.821264470"
Oct 30 18:46:27.077380 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:27.077345    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-msdwn"
Oct 30 18:46:27.083112 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:27.083085    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-msdwn"
Oct 30 18:46:27.404897 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:27.404810    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-msdwn"
Oct 30 18:46:27.405727 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:27.405709    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-msdwn"
Oct 30 18:46:30.089457 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:30.089424    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-rjmdz"]
Oct 30 18:46:34.393087 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:34.393043    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-e7929d-7cbc876b55-5mnbp"
Oct 30 18:46:35.624345 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:46:35.624248    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-jbsw6" containerName="extractor"
Oct 30 18:46:39.484722 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:39.484682    4888 generic.go:356] "Generic (PLEG): container finished" podID="bae588a3-e65e-4f9b-b4c4-615aa878741e" containerID="7c50df43789b05948bdbc79637a365e4bc455c0277c5f7755a9d5cff7caa60f3" exitCode=0
Oct 30 18:46:39.485172 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:39.484763    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-cgw6d" event={"ID":"bae588a3-e65e-4f9b-b4c4-615aa878741e","Type":"ContainerDied","Data":"7c50df43789b05948bdbc79637a365e4bc455c0277c5f7755a9d5cff7caa60f3"}
Oct 30 18:46:39.485172 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:39.485121    4888 scope.go:122] "RemoveContainer" containerID="7c50df43789b05948bdbc79637a365e4bc455c0277c5f7755a9d5cff7caa60f3"
Oct 30 18:46:39.486441 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:39.486418    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:46:39.486516 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:39.486482    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:46:39.486555 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:39.486524    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:46:40.500068 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:40.500038    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-cgw6d" event={"ID":"bae588a3-e65e-4f9b-b4c4-615aa878741e","Type":"ContainerStarted","Data":"035c09ad84c4ee952cd6931ec51f499f1ed45e4ba21766a9506710ac92814e57"}
Oct 30 18:46:41.339305 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:41.339249    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-cgw6d"
Oct 30 18:46:41.339609 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:41.339585    4888 patch_prober.go:29] interesting pod/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-cgw6d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body=
Oct 30 18:46:41.339710 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:41.339637    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-cgw6d" podUID="bae588a3-e65e-4f9b-b4c4-615aa878741e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused"
Oct 30 18:46:42.338926 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:42.338894    4888 patch_prober.go:29] interesting pod/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-cgw6d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body=
Oct 30 18:46:42.339347 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:42.338945    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-cgw6d" podUID="bae588a3-e65e-4f9b-b4c4-615aa878741e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused"
Oct 30 18:46:43.338886 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:43.338852    4888 patch_prober.go:29] interesting pod/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-cgw6d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body=
Oct 30 18:46:43.339055 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:43.338905    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-cgw6d" podUID="bae588a3-e65e-4f9b-b4c4-615aa878741e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused"
Oct 30 18:46:44.338880 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:44.338833    4888 patch_prober.go:29] interesting pod/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-cgw6d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body=
Oct 30 18:46:44.339065 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:44.338903    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-cgw6d" podUID="bae588a3-e65e-4f9b-b4c4-615aa878741e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused"
Oct 30 18:46:45.339463 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:45.339432    4888 patch_prober.go:29] interesting pod/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-cgw6d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body=
Oct 30 18:46:45.339862 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:45.339480    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-cgw6d" podUID="bae588a3-e65e-4f9b-b4c4-615aa878741e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused"
Oct 30 18:46:46.221834 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:46.221807    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-cgw6d"
Oct 30 18:46:46.343519 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:46.343491    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-cgw6d"
Oct 30 18:46:46.539176 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:46.539090    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-cgw6d"
Oct 30 18:46:47.682615 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:47.682583    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-rjmdz"]
Oct 30 18:46:53.628136 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:53.628098    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-fd61e3-7647c5bb9f-h4th6"]
Oct 30 18:46:53.633536 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:53.633513    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-fd61e3-7647c5bb9f-h4th6"
Oct 30 18:46:53.688333 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:53.688307    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w1-fd61e3-serving-cert\""
Oct 30 18:46:53.717563 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:53.717538    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a4fd7ad8-cdda-470a-86bf-c0c0df93fda3-maas-api-tls\") pod \"maas-api-e2e-worker-w1-fd61e3-7647c5bb9f-h4th6\" (UID: \"a4fd7ad8-cdda-470a-86bf-c0c0df93fda3\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-fd61e3-7647c5bb9f-h4th6"
Oct 30 18:46:53.717700 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:53.717579    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tllbc\" (UniqueName: \"kubernetes.io/projected/a4fd7ad8-cdda-470a-86bf-c0c0df93fda3-kube-api-access-tllbc\") pod \"maas-api-e2e-worker-w1-fd61e3-7647c5bb9f-h4th6\" (UID: \"a4fd7ad8-cdda-470a-86bf-c0c0df93fda3\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-fd61e3-7647c5bb9f-h4th6"
Oct 30 18:46:53.717771 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:53.717704    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a4fd7ad8-cdda-470a-86bf-c0c0df93fda3-metrics-certs\") pod \"maas-api-e2e-worker-w1-fd61e3-7647c5bb9f-h4th6\" (UID: \"a4fd7ad8-cdda-470a-86bf-c0c0df93fda3\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-fd61e3-7647c5bb9f-h4th6"
Oct 30 18:46:53.717830 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:53.717764    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a4fd7ad8-cdda-470a-86bf-c0c0df93fda3-openshift-service-ca\") pod \"maas-api-e2e-worker-w1-fd61e3-7647c5bb9f-h4th6\" (UID: \"a4fd7ad8-cdda-470a-86bf-c0c0df93fda3\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-fd61e3-7647c5bb9f-h4th6"
Oct 30 18:46:53.742068 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:53.742043    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-fd61e3-7647c5bb9f-h4th6"]
Oct 30 18:46:53.818508 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:53.818475    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a4fd7ad8-cdda-470a-86bf-c0c0df93fda3-openshift-service-ca\") pod \"maas-api-e2e-worker-w1-fd61e3-7647c5bb9f-h4th6\" (UID: \"a4fd7ad8-cdda-470a-86bf-c0c0df93fda3\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-fd61e3-7647c5bb9f-h4th6"
Oct 30 18:46:53.818651 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:53.818550    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a4fd7ad8-cdda-470a-86bf-c0c0df93fda3-maas-api-tls\") pod \"maas-api-e2e-worker-w1-fd61e3-7647c5bb9f-h4th6\" (UID: \"a4fd7ad8-cdda-470a-86bf-c0c0df93fda3\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-fd61e3-7647c5bb9f-h4th6"
Oct 30 18:46:53.818651 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:53.818582    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tllbc\" (UniqueName: \"kubernetes.io/projected/a4fd7ad8-cdda-470a-86bf-c0c0df93fda3-kube-api-access-tllbc\") pod \"maas-api-e2e-worker-w1-fd61e3-7647c5bb9f-h4th6\" (UID: \"a4fd7ad8-cdda-470a-86bf-c0c0df93fda3\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-fd61e3-7647c5bb9f-h4th6"
Oct 30 18:46:53.818651 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:53.818633    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a4fd7ad8-cdda-470a-86bf-c0c0df93fda3-metrics-certs\") pod \"maas-api-e2e-worker-w1-fd61e3-7647c5bb9f-h4th6\" (UID: \"a4fd7ad8-cdda-470a-86bf-c0c0df93fda3\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-fd61e3-7647c5bb9f-h4th6"
Oct 30 18:46:53.818782 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:46:53.818684    4888 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w1-fd61e3-serving-cert: secret "maas-api-e2e-worker-w1-fd61e3-serving-cert" not found
Oct 30 18:46:53.818782 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:46:53.818764    4888 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a4fd7ad8-cdda-470a-86bf-c0c0df93fda3-maas-api-tls podName:a4fd7ad8-cdda-470a-86bf-c0c0df93fda3 nodeName:}" failed. No retries permitted until 2026-10-30 18:46:54.318747144 +0000 UTC m=+778.566153166 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/a4fd7ad8-cdda-470a-86bf-c0c0df93fda3-maas-api-tls") pod "maas-api-e2e-worker-w1-fd61e3-7647c5bb9f-h4th6" (UID: "a4fd7ad8-cdda-470a-86bf-c0c0df93fda3") : secret "maas-api-e2e-worker-w1-fd61e3-serving-cert" not found
Oct 30 18:46:53.819220 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:53.819194    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a4fd7ad8-cdda-470a-86bf-c0c0df93fda3-openshift-service-ca\") pod \"maas-api-e2e-worker-w1-fd61e3-7647c5bb9f-h4th6\" (UID: \"a4fd7ad8-cdda-470a-86bf-c0c0df93fda3\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-fd61e3-7647c5bb9f-h4th6"
Oct 30 18:46:53.821263 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:53.821241    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a4fd7ad8-cdda-470a-86bf-c0c0df93fda3-metrics-certs\") pod \"maas-api-e2e-worker-w1-fd61e3-7647c5bb9f-h4th6\" (UID: \"a4fd7ad8-cdda-470a-86bf-c0c0df93fda3\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-fd61e3-7647c5bb9f-h4th6"
Oct 30 18:46:53.885897 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:53.885795    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tllbc\" (UniqueName: \"kubernetes.io/projected/a4fd7ad8-cdda-470a-86bf-c0c0df93fda3-kube-api-access-tllbc\") pod \"maas-api-e2e-worker-w1-fd61e3-7647c5bb9f-h4th6\" (UID: \"a4fd7ad8-cdda-470a-86bf-c0c0df93fda3\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-fd61e3-7647c5bb9f-h4th6"
Oct 30 18:46:54.324600 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:54.324567    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a4fd7ad8-cdda-470a-86bf-c0c0df93fda3-maas-api-tls\") pod \"maas-api-e2e-worker-w1-fd61e3-7647c5bb9f-h4th6\" (UID: \"a4fd7ad8-cdda-470a-86bf-c0c0df93fda3\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-fd61e3-7647c5bb9f-h4th6"
Oct 30 18:46:54.327344 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:54.327318    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a4fd7ad8-cdda-470a-86bf-c0c0df93fda3-maas-api-tls\") pod \"maas-api-e2e-worker-w1-fd61e3-7647c5bb9f-h4th6\" (UID: \"a4fd7ad8-cdda-470a-86bf-c0c0df93fda3\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-fd61e3-7647c5bb9f-h4th6"
Oct 30 18:46:54.547474 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:54.547442    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-fd61e3-7647c5bb9f-h4th6"
Oct 30 18:46:54.926758 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:54.926729    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-fd61e3-7647c5bb9f-h4th6"]
Oct 30 18:46:54.927104 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:46:54.926981    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4fd7ad8_cdda_470a_86bf_c0c0df93fda3.slice/crio-8e266367c9811bbf35b7c2bf773fb1ea8889b27fdce3c433f442553d16ff4e21 WatchSource:0}: Error finding container 8e266367c9811bbf35b7c2bf773fb1ea8889b27fdce3c433f442553d16ff4e21: Status 404 returned error can't find the container with id 8e266367c9811bbf35b7c2bf773fb1ea8889b27fdce3c433f442553d16ff4e21
Oct 30 18:46:55.599297 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:55.599261    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-fd61e3-7647c5bb9f-h4th6" event={"ID":"a4fd7ad8-cdda-470a-86bf-c0c0df93fda3","Type":"ContainerStarted","Data":"8e266367c9811bbf35b7c2bf773fb1ea8889b27fdce3c433f442553d16ff4e21"}
Oct 30 18:46:56.606212 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:56.606183    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-fd61e3-7647c5bb9f-h4th6" event={"ID":"a4fd7ad8-cdda-470a-86bf-c0c0df93fda3","Type":"ContainerStarted","Data":"d212141157e251729a834d30ecbf004ac79b3a9093f6d80f1391bae628e0d560"}
Oct 30 18:46:56.606630 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:56.606249    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-fd61e3-7647c5bb9f-h4th6"
Oct 30 18:46:56.729915 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:56.729865    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-fd61e3-7647c5bb9f-h4th6" podStartSLOduration=2.874154531 podStartE2EDuration="3.729846948s" totalImagesPullingTime="855.692417ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:46:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:46:56.728515893 +0000 UTC m=+780.975921917" watchObservedRunningTime="2026-10-30 18:46:56.729846948 +0000 UTC m=+780.977253006"
Oct 30 18:46:59.222277 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:46:59.222235    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-rjmdz"]
Oct 30 18:47:02.619419 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:47:02.619392    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-fd61e3-7647c5bb9f-h4th6"
Oct 30 18:47:05.620424 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:47:05.620385    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-jbsw6" containerName="extractor"
Oct 30 18:47:08.682907 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:47:08.682875    4888 generic.go:356] "Generic (PLEG): container finished" podID="f49068b0-4dea-487c-a31c-c61704119dad" containerID="f7e4b2330162644e18b81c185c852390d98e3ec3f146b0a5e16ea2db1a8e781c" exitCode=0
Oct 30 18:47:08.683471 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:47:08.682952    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-msdwn" event={"ID":"f49068b0-4dea-487c-a31c-c61704119dad","Type":"ContainerDied","Data":"f7e4b2330162644e18b81c185c852390d98e3ec3f146b0a5e16ea2db1a8e781c"}
Oct 30 18:47:08.683542 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:47:08.683493    4888 scope.go:122] "RemoveContainer" containerID="f7e4b2330162644e18b81c185c852390d98e3ec3f146b0a5e16ea2db1a8e781c"
Oct 30 18:47:08.684960 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:47:08.684934    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:47:08.685061 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:47:08.685001    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:47:08.685061 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:47:08.685031    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:47:09.704061 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:47:09.704021    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-msdwn" event={"ID":"f49068b0-4dea-487c-a31c-c61704119dad","Type":"ContainerStarted","Data":"77d084411a7aed534925ca3711c62beb55728f847f45ec1544e4a454432b7e1a"}
Oct 30 18:47:10.092609 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:47:10.080972    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-msdwn"
Oct 30 18:47:10.092609 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:47:10.081111    4888 patch_prober.go:29] interesting pod/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-msdwn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused" start-of-body=
Oct 30 18:47:10.092609 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:47:10.081169    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-msdwn" podUID="f49068b0-4dea-487c-a31c-c61704119dad" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused"
Oct 30 18:47:11.083746 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:47:11.083706    4888 patch_prober.go:29] interesting pod/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-msdwn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused" start-of-body=
Oct 30 18:47:11.084441 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:47:11.084400    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-msdwn" podUID="f49068b0-4dea-487c-a31c-c61704119dad" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused"
Oct 30 18:47:12.084310 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:47:12.080707    4888 patch_prober.go:29] interesting pod/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-msdwn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused" start-of-body=
Oct 30 18:47:12.084310 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:47:12.080766    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-msdwn" podUID="f49068b0-4dea-487c-a31c-c61704119dad" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused"
Oct 30 18:47:12.405745 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:47:12.405653    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-msdwn"
Oct 30 18:47:13.081320 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:47:13.079357    4888 patch_prober.go:29] interesting pod/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-msdwn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused" start-of-body=
Oct 30 18:47:13.081320 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:47:13.079413    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-msdwn" podUID="f49068b0-4dea-487c-a31c-c61704119dad" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused"
Oct 30 18:47:14.080310 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:47:14.078653    4888 patch_prober.go:29] interesting pod/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-msdwn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused" start-of-body=
Oct 30 18:47:14.080310 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:47:14.078711    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-msdwn" podUID="f49068b0-4dea-487c-a31c-c61704119dad" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused"
Oct 30 18:47:15.077910 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:47:15.077871    4888 patch_prober.go:29] interesting pod/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-msdwn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused" start-of-body=
Oct 30 18:47:15.078096 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:47:15.077947    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-msdwn" podUID="f49068b0-4dea-487c-a31c-c61704119dad" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused"
Oct 30 18:47:16.081760 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:47:16.081725    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-msdwn"
Oct 30 18:47:16.082934 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:47:16.082916    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-msdwn"
Oct 30 18:47:20.419482 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:47:20.419448    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-rjmdz"]
Oct 30 18:47:35.617368 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:47:35.617334    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-jbsw6" containerName="extractor"
Oct 30 18:47:39.387229 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:47:39.387198    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w3-3b6013/premium-w3-3b6013-kserve-74df5fb95d-k747z"]
Oct 30 18:47:39.389995 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:47:39.389976    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-3b6013/premium-w3-3b6013-kserve-74df5fb95d-k747z"
Oct 30 18:47:39.494939 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:47:39.494909    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-3b6013\"/\"premium-w3-3b6013-kserve-self-signed-certs\""
Oct 30 18:47:39.495186 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:47:39.495167    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-3b6013\"/\"default-dockercfg-dfsp2\""
Oct 30 18:47:39.495263 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:47:39.495215    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w3-3b6013\"/\"openshift-service-ca.crt\""
Oct 30 18:47:39.495263 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:47:39.495217    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w3-3b6013\"/\"kube-root-ca.crt\""
Oct 30 18:47:39.498169 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:47:39.498149    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-3b6013/premium-w3-3b6013-kserve-74df5fb95d-k747z"]
Oct 30 18:47:39.506545 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:47:39.506521    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a9309776-20d5-4c06-bcae-04b8e54493e8-tls-certs\") pod \"premium-w3-3b6013-kserve-74df5fb95d-k747z\" (UID: \"a9309776-20d5-4c06-bcae-04b8e54493e8\") " pod="e2e-models-e2e-worker-w3-3b6013/premium-w3-3b6013-kserve-74df5fb95d-k747z"
Oct 30 18:47:39.506662 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:47:39.506570    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a9309776-20d5-4c06-bcae-04b8e54493e8-home\") pod \"premium-w3-3b6013-kserve-74df5fb95d-k747z\" (UID: \"a9309776-20d5-4c06-bcae-04b8e54493e8\") " pod="e2e-models-e2e-worker-w3-3b6013/premium-w3-3b6013-kserve-74df5fb95d-k747z"
Oct 30 18:47:39.506732 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:47:39.506662    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a9309776-20d5-4c06-bcae-04b8e54493e8-model-cache\") pod \"premium-w3-3b6013-kserve-74df5fb95d-k747z\" (UID: \"a9309776-20d5-4c06-bcae-04b8e54493e8\") " pod="e2e-models-e2e-worker-w3-3b6013/premium-w3-3b6013-kserve-74df5fb95d-k747z"
Oct 30 18:47:39.506732 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:47:39.506715    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l4dd9\" (UniqueName: \"kubernetes.io/projected/a9309776-20d5-4c06-bcae-04b8e54493e8-kube-api-access-l4dd9\") pod \"premium-w3-3b6013-kserve-74df5fb95d-k747z\" (UID: \"a9309776-20d5-4c06-bcae-04b8e54493e8\") " pod="e2e-models-e2e-worker-w3-3b6013/premium-w3-3b6013-kserve-74df5fb95d-k747z"
Oct 30 18:47:39.506947 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:47:39.506876    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a9309776-20d5-4c06-bcae-04b8e54493e8-dshm\") pod \"premium-w3-3b6013-kserve-74df5fb95d-k747z\" (UID: \"a9309776-20d5-4c06-bcae-04b8e54493e8\") " pod="e2e-models-e2e-worker-w3-3b6013/premium-w3-3b6013-kserve-74df5fb95d-k747z"
Oct 30 18:47:39.506947 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:47:39.506925    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a9309776-20d5-4c06-bcae-04b8e54493e8-tmp-dir\") pod \"premium-w3-3b6013-kserve-74df5fb95d-k747z\" (UID: \"a9309776-20d5-4c06-bcae-04b8e54493e8\") " pod="e2e-models-e2e-worker-w3-3b6013/premium-w3-3b6013-kserve-74df5fb95d-k747z"
Oct 30 18:47:39.608452 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:47:39.608413    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a9309776-20d5-4c06-bcae-04b8e54493e8-tmp-dir\") pod \"premium-w3-3b6013-kserve-74df5fb95d-k747z\" (UID: \"a9309776-20d5-4c06-bcae-04b8e54493e8\") " pod="e2e-models-e2e-worker-w3-3b6013/premium-w3-3b6013-kserve-74df5fb95d-k747z"
Oct 30 18:47:39.608647 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:47:39.608488    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a9309776-20d5-4c06-bcae-04b8e54493e8-tls-certs\") pod \"premium-w3-3b6013-kserve-74df5fb95d-k747z\" (UID: \"a9309776-20d5-4c06-bcae-04b8e54493e8\") " pod="e2e-models-e2e-worker-w3-3b6013/premium-w3-3b6013-kserve-74df5fb95d-k747z"
Oct 30 18:47:39.608647 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:47:39.608547    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a9309776-20d5-4c06-bcae-04b8e54493e8-home\") pod \"premium-w3-3b6013-kserve-74df5fb95d-k747z\" (UID: \"a9309776-20d5-4c06-bcae-04b8e54493e8\") " pod="e2e-models-e2e-worker-w3-3b6013/premium-w3-3b6013-kserve-74df5fb95d-k747z"
Oct 30 18:47:39.608647 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:47:39.608594    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a9309776-20d5-4c06-bcae-04b8e54493e8-model-cache\") pod \"premium-w3-3b6013-kserve-74df5fb95d-k747z\" (UID: \"a9309776-20d5-4c06-bcae-04b8e54493e8\") " pod="e2e-models-e2e-worker-w3-3b6013/premium-w3-3b6013-kserve-74df5fb95d-k747z"
Oct 30 18:47:39.608647 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:47:39.608630    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l4dd9\" (UniqueName: \"kubernetes.io/projected/a9309776-20d5-4c06-bcae-04b8e54493e8-kube-api-access-l4dd9\") pod \"premium-w3-3b6013-kserve-74df5fb95d-k747z\" (UID: \"a9309776-20d5-4c06-bcae-04b8e54493e8\") " pod="e2e-models-e2e-worker-w3-3b6013/premium-w3-3b6013-kserve-74df5fb95d-k747z"
Oct 30 18:47:39.608856 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:47:39.608726    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a9309776-20d5-4c06-bcae-04b8e54493e8-dshm\") pod \"premium-w3-3b6013-kserve-74df5fb95d-k747z\" (UID: \"a9309776-20d5-4c06-bcae-04b8e54493e8\") " pod="e2e-models-e2e-worker-w3-3b6013/premium-w3-3b6013-kserve-74df5fb95d-k747z"
Oct 30 18:47:39.608918 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:47:39.608896    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a9309776-20d5-4c06-bcae-04b8e54493e8-tmp-dir\") pod \"premium-w3-3b6013-kserve-74df5fb95d-k747z\" (UID: \"a9309776-20d5-4c06-bcae-04b8e54493e8\") " pod="e2e-models-e2e-worker-w3-3b6013/premium-w3-3b6013-kserve-74df5fb95d-k747z"
Oct 30 18:47:39.608960 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:47:39.608915    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a9309776-20d5-4c06-bcae-04b8e54493e8-home\") pod \"premium-w3-3b6013-kserve-74df5fb95d-k747z\" (UID: \"a9309776-20d5-4c06-bcae-04b8e54493e8\") " pod="e2e-models-e2e-worker-w3-3b6013/premium-w3-3b6013-kserve-74df5fb95d-k747z"
Oct 30 18:47:39.609005 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:47:39.608988    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a9309776-20d5-4c06-bcae-04b8e54493e8-model-cache\") pod \"premium-w3-3b6013-kserve-74df5fb95d-k747z\" (UID: \"a9309776-20d5-4c06-bcae-04b8e54493e8\") " pod="e2e-models-e2e-worker-w3-3b6013/premium-w3-3b6013-kserve-74df5fb95d-k747z"
Oct 30 18:47:39.610922 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:47:39.610903    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a9309776-20d5-4c06-bcae-04b8e54493e8-dshm\") pod \"premium-w3-3b6013-kserve-74df5fb95d-k747z\" (UID: \"a9309776-20d5-4c06-bcae-04b8e54493e8\") " pod="e2e-models-e2e-worker-w3-3b6013/premium-w3-3b6013-kserve-74df5fb95d-k747z"
Oct 30 18:47:39.611232 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:47:39.611216    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a9309776-20d5-4c06-bcae-04b8e54493e8-tls-certs\") pod \"premium-w3-3b6013-kserve-74df5fb95d-k747z\" (UID: \"a9309776-20d5-4c06-bcae-04b8e54493e8\") " pod="e2e-models-e2e-worker-w3-3b6013/premium-w3-3b6013-kserve-74df5fb95d-k747z"
Oct 30 18:47:39.671471 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:47:39.671410    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l4dd9\" (UniqueName: \"kubernetes.io/projected/a9309776-20d5-4c06-bcae-04b8e54493e8-kube-api-access-l4dd9\") pod \"premium-w3-3b6013-kserve-74df5fb95d-k747z\" (UID: \"a9309776-20d5-4c06-bcae-04b8e54493e8\") " pod="e2e-models-e2e-worker-w3-3b6013/premium-w3-3b6013-kserve-74df5fb95d-k747z"
Oct 30 18:47:39.700921 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:47:39.700897    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-3b6013/premium-w3-3b6013-kserve-74df5fb95d-k747z"
Oct 30 18:47:40.091157 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:47:40.091122    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda9309776_20d5_4c06_bcae_04b8e54493e8.slice/crio-0b6cfeb036c0dac3dae47741a8d8eb9840cf17d8f68c2b8ec66069d49d694c6c WatchSource:0}: Error finding container 0b6cfeb036c0dac3dae47741a8d8eb9840cf17d8f68c2b8ec66069d49d694c6c: Status 404 returned error can't find the container with id 0b6cfeb036c0dac3dae47741a8d8eb9840cf17d8f68c2b8ec66069d49d694c6c
Oct 30 18:47:40.091672 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:47:40.091647    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-3b6013/premium-w3-3b6013-kserve-74df5fb95d-k747z"]
Oct 30 18:47:40.902802 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:47:40.902771    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-3b6013/premium-w3-3b6013-kserve-74df5fb95d-k747z" event={"ID":"a9309776-20d5-4c06-bcae-04b8e54493e8","Type":"ContainerStarted","Data":"b8f7a11879729dfdaeb18f36e51d7057e97444b00a31e47fbb3c2e005ae8f364"}
Oct 30 18:47:40.903331 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:47:40.902812    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-3b6013/premium-w3-3b6013-kserve-74df5fb95d-k747z" event={"ID":"a9309776-20d5-4c06-bcae-04b8e54493e8","Type":"ContainerStarted","Data":"0b6cfeb036c0dac3dae47741a8d8eb9840cf17d8f68c2b8ec66069d49d694c6c"}
Oct 30 18:47:41.023232 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:47:41.023183    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w3-3b6013/premium-w3-3b6013-kserve-74df5fb95d-k747z" podStartSLOduration=2.023170454 podStartE2EDuration="2.023170454s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:47:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:47:41.021881559 +0000 UTC m=+825.269287584" watchObservedRunningTime="2026-10-30 18:47:41.023170454 +0000 UTC m=+825.270576478"
Oct 30 18:47:41.701841 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:47:41.701802    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-3b6013/premium-w3-3b6013-kserve-74df5fb95d-k747z"
Oct 30 18:47:45.127404 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:47:45.127367    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-rjmdz"]
Oct 30 18:47:49.701801 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:47:49.701763    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w3-3b6013/premium-w3-3b6013-kserve-74df5fb95d-k747z"
Oct 30 18:47:49.714844 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:47:49.714816    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w3-3b6013/premium-w3-3b6013-kserve-74df5fb95d-k747z"
Oct 30 18:47:49.973348 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:47:49.973240    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-3b6013/premium-w3-3b6013-kserve-74df5fb95d-k747z"
Oct 30 18:48:02.040263 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:48:02.040222    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-rjmdz"]
Oct 30 18:48:05.618002 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:48:05.617924    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-jbsw6" containerName="extractor"
Oct 30 18:48:24.591036 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:48:24.591003    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w2-4a7a52/premium-w2-4a7a52-kserve-746d9579f6-hn7hl"]
Oct 30 18:48:24.595084 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:48:24.595066    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-4a7a52/premium-w2-4a7a52-kserve-746d9579f6-hn7hl"
Oct 30 18:48:24.667883 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:48:24.667847    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-4a7a52/premium-w2-4a7a52-kserve-746d9579f6-hn7hl"]
Oct 30 18:48:24.700853 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:48:24.700819    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w2-4a7a52\"/\"openshift-service-ca.crt\""
Oct 30 18:48:24.700853 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:48:24.700840    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w2-4a7a52\"/\"kube-root-ca.crt\""
Oct 30 18:48:24.701072 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:48:24.701015    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-4a7a52\"/\"premium-w2-4a7a52-kserve-self-signed-certs\""
Oct 30 18:48:24.701072 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:48:24.701024    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-4a7a52\"/\"default-dockercfg-qzpjw\""
Oct 30 18:48:24.765352 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:48:24.765318    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3160b54f-1df9-48de-80d9-8d8a91d47f56-home\") pod \"premium-w2-4a7a52-kserve-746d9579f6-hn7hl\" (UID: \"3160b54f-1df9-48de-80d9-8d8a91d47f56\") " pod="e2e-models-e2e-worker-w2-4a7a52/premium-w2-4a7a52-kserve-746d9579f6-hn7hl"
Oct 30 18:48:24.765567 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:48:24.765432    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3160b54f-1df9-48de-80d9-8d8a91d47f56-tmp-dir\") pod \"premium-w2-4a7a52-kserve-746d9579f6-hn7hl\" (UID: \"3160b54f-1df9-48de-80d9-8d8a91d47f56\") " pod="e2e-models-e2e-worker-w2-4a7a52/premium-w2-4a7a52-kserve-746d9579f6-hn7hl"
Oct 30 18:48:24.765567 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:48:24.765484    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3160b54f-1df9-48de-80d9-8d8a91d47f56-tls-certs\") pod \"premium-w2-4a7a52-kserve-746d9579f6-hn7hl\" (UID: \"3160b54f-1df9-48de-80d9-8d8a91d47f56\") " pod="e2e-models-e2e-worker-w2-4a7a52/premium-w2-4a7a52-kserve-746d9579f6-hn7hl"
Oct 30 18:48:24.765668 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:48:24.765627    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rdnzg\" (UniqueName: \"kubernetes.io/projected/3160b54f-1df9-48de-80d9-8d8a91d47f56-kube-api-access-rdnzg\") pod \"premium-w2-4a7a52-kserve-746d9579f6-hn7hl\" (UID: \"3160b54f-1df9-48de-80d9-8d8a91d47f56\") " pod="e2e-models-e2e-worker-w2-4a7a52/premium-w2-4a7a52-kserve-746d9579f6-hn7hl"
Oct 30 18:48:24.765795 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:48:24.765774    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3160b54f-1df9-48de-80d9-8d8a91d47f56-dshm\") pod \"premium-w2-4a7a52-kserve-746d9579f6-hn7hl\" (UID: \"3160b54f-1df9-48de-80d9-8d8a91d47f56\") " pod="e2e-models-e2e-worker-w2-4a7a52/premium-w2-4a7a52-kserve-746d9579f6-hn7hl"
Oct 30 18:48:24.765871 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:48:24.765836    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3160b54f-1df9-48de-80d9-8d8a91d47f56-model-cache\") pod \"premium-w2-4a7a52-kserve-746d9579f6-hn7hl\" (UID: \"3160b54f-1df9-48de-80d9-8d8a91d47f56\") " pod="e2e-models-e2e-worker-w2-4a7a52/premium-w2-4a7a52-kserve-746d9579f6-hn7hl"
Oct 30 18:48:24.866703 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:48:24.866636    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3160b54f-1df9-48de-80d9-8d8a91d47f56-model-cache\") pod \"premium-w2-4a7a52-kserve-746d9579f6-hn7hl\" (UID: \"3160b54f-1df9-48de-80d9-8d8a91d47f56\") " pod="e2e-models-e2e-worker-w2-4a7a52/premium-w2-4a7a52-kserve-746d9579f6-hn7hl"
Oct 30 18:48:24.866703 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:48:24.866672    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3160b54f-1df9-48de-80d9-8d8a91d47f56-home\") pod \"premium-w2-4a7a52-kserve-746d9579f6-hn7hl\" (UID: \"3160b54f-1df9-48de-80d9-8d8a91d47f56\") " pod="e2e-models-e2e-worker-w2-4a7a52/premium-w2-4a7a52-kserve-746d9579f6-hn7hl"
Oct 30 18:48:24.866703 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:48:24.866701    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3160b54f-1df9-48de-80d9-8d8a91d47f56-tmp-dir\") pod \"premium-w2-4a7a52-kserve-746d9579f6-hn7hl\" (UID: \"3160b54f-1df9-48de-80d9-8d8a91d47f56\") " pod="e2e-models-e2e-worker-w2-4a7a52/premium-w2-4a7a52-kserve-746d9579f6-hn7hl"
Oct 30 18:48:24.866934 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:48:24.866818    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3160b54f-1df9-48de-80d9-8d8a91d47f56-tls-certs\") pod \"premium-w2-4a7a52-kserve-746d9579f6-hn7hl\" (UID: \"3160b54f-1df9-48de-80d9-8d8a91d47f56\") " pod="e2e-models-e2e-worker-w2-4a7a52/premium-w2-4a7a52-kserve-746d9579f6-hn7hl"
Oct 30 18:48:24.866934 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:48:24.866899    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rdnzg\" (UniqueName: \"kubernetes.io/projected/3160b54f-1df9-48de-80d9-8d8a91d47f56-kube-api-access-rdnzg\") pod \"premium-w2-4a7a52-kserve-746d9579f6-hn7hl\" (UID: \"3160b54f-1df9-48de-80d9-8d8a91d47f56\") " pod="e2e-models-e2e-worker-w2-4a7a52/premium-w2-4a7a52-kserve-746d9579f6-hn7hl"
Oct 30 18:48:24.867034 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:48:24.867020    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3160b54f-1df9-48de-80d9-8d8a91d47f56-dshm\") pod \"premium-w2-4a7a52-kserve-746d9579f6-hn7hl\" (UID: \"3160b54f-1df9-48de-80d9-8d8a91d47f56\") " pod="e2e-models-e2e-worker-w2-4a7a52/premium-w2-4a7a52-kserve-746d9579f6-hn7hl"
Oct 30 18:48:24.867104 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:48:24.867084    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3160b54f-1df9-48de-80d9-8d8a91d47f56-tmp-dir\") pod \"premium-w2-4a7a52-kserve-746d9579f6-hn7hl\" (UID: \"3160b54f-1df9-48de-80d9-8d8a91d47f56\") " pod="e2e-models-e2e-worker-w2-4a7a52/premium-w2-4a7a52-kserve-746d9579f6-hn7hl"
Oct 30 18:48:24.867145 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:48:24.867120    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3160b54f-1df9-48de-80d9-8d8a91d47f56-home\") pod \"premium-w2-4a7a52-kserve-746d9579f6-hn7hl\" (UID: \"3160b54f-1df9-48de-80d9-8d8a91d47f56\") " pod="e2e-models-e2e-worker-w2-4a7a52/premium-w2-4a7a52-kserve-746d9579f6-hn7hl"
Oct 30 18:48:24.867189 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:48:24.867161    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3160b54f-1df9-48de-80d9-8d8a91d47f56-model-cache\") pod \"premium-w2-4a7a52-kserve-746d9579f6-hn7hl\" (UID: \"3160b54f-1df9-48de-80d9-8d8a91d47f56\") " pod="e2e-models-e2e-worker-w2-4a7a52/premium-w2-4a7a52-kserve-746d9579f6-hn7hl"
Oct 30 18:48:24.869357 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:48:24.869331    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3160b54f-1df9-48de-80d9-8d8a91d47f56-dshm\") pod \"premium-w2-4a7a52-kserve-746d9579f6-hn7hl\" (UID: \"3160b54f-1df9-48de-80d9-8d8a91d47f56\") " pod="e2e-models-e2e-worker-w2-4a7a52/premium-w2-4a7a52-kserve-746d9579f6-hn7hl"
Oct 30 18:48:24.869615 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:48:24.869597    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3160b54f-1df9-48de-80d9-8d8a91d47f56-tls-certs\") pod \"premium-w2-4a7a52-kserve-746d9579f6-hn7hl\" (UID: \"3160b54f-1df9-48de-80d9-8d8a91d47f56\") " pod="e2e-models-e2e-worker-w2-4a7a52/premium-w2-4a7a52-kserve-746d9579f6-hn7hl"
Oct 30 18:48:24.929831 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:48:24.929806    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rdnzg\" (UniqueName: \"kubernetes.io/projected/3160b54f-1df9-48de-80d9-8d8a91d47f56-kube-api-access-rdnzg\") pod \"premium-w2-4a7a52-kserve-746d9579f6-hn7hl\" (UID: \"3160b54f-1df9-48de-80d9-8d8a91d47f56\") " pod="e2e-models-e2e-worker-w2-4a7a52/premium-w2-4a7a52-kserve-746d9579f6-hn7hl"
Oct 30 18:48:25.207328 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:48:25.207224    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-4a7a52/premium-w2-4a7a52-kserve-746d9579f6-hn7hl"
Oct 30 18:48:25.584330 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:48:25.584302    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-4a7a52/premium-w2-4a7a52-kserve-746d9579f6-hn7hl"]
Oct 30 18:48:25.584828 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:48:25.584804    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3160b54f_1df9_48de_80d9_8d8a91d47f56.slice/crio-3af21bf02037d46de9ad04acffabe4ae29a165ac4755545840fd1b105d3c8572 WatchSource:0}: Error finding container 3af21bf02037d46de9ad04acffabe4ae29a165ac4755545840fd1b105d3c8572: Status 404 returned error can't find the container with id 3af21bf02037d46de9ad04acffabe4ae29a165ac4755545840fd1b105d3c8572
Oct 30 18:48:26.197125 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:48:26.197086    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-4a7a52/premium-w2-4a7a52-kserve-746d9579f6-hn7hl" event={"ID":"3160b54f-1df9-48de-80d9-8d8a91d47f56","Type":"ContainerStarted","Data":"4d848c8bd5b08337a6dd1112a40e2bde01626862f7ed0a59e6f00bcd0353203d"}
Oct 30 18:48:26.197125 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:48:26.197124    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-4a7a52/premium-w2-4a7a52-kserve-746d9579f6-hn7hl" event={"ID":"3160b54f-1df9-48de-80d9-8d8a91d47f56","Type":"ContainerStarted","Data":"3af21bf02037d46de9ad04acffabe4ae29a165ac4755545840fd1b105d3c8572"}
Oct 30 18:48:26.207906 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:48:26.207880    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-4a7a52/premium-w2-4a7a52-kserve-746d9579f6-hn7hl"
Oct 30 18:48:26.319012 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:48:26.318972    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w2-4a7a52/premium-w2-4a7a52-kserve-746d9579f6-hn7hl" podStartSLOduration=2.318958486 podStartE2EDuration="2.318958486s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:48:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:48:26.318204175 +0000 UTC m=+870.565610202" watchObservedRunningTime="2026-10-30 18:48:26.318958486 +0000 UTC m=+870.566364540"
Oct 30 18:48:28.639743 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:48:28.639701    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-rjmdz"]
Oct 30 18:48:35.207762 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:48:35.207725    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w2-4a7a52/premium-w2-4a7a52-kserve-746d9579f6-hn7hl"
Oct 30 18:48:35.220683 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:48:35.220658    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w2-4a7a52/premium-w2-4a7a52-kserve-746d9579f6-hn7hl"
Oct 30 18:48:35.290950 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:48:35.290919    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-4a7a52/premium-w2-4a7a52-kserve-746d9579f6-hn7hl"
Oct 30 18:48:35.617916 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:48:35.617884    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-jbsw6" containerName="extractor"
Oct 30 18:48:54.644250 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:48:54.644214    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-rjmdz"]
Oct 30 18:48:56.138817 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:48:56.138789    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-7x4rb_382e8b31-a7fa-43f3-9f15-4c61627b0c04/console-operator/2.log"
Oct 30 18:48:56.176308 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:48:56.171763    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-7x4rb_382e8b31-a7fa-43f3-9f15-4c61627b0c04/console-operator/2.log"
Oct 30 18:48:56.210906 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:48:56.210879    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xfcdl_3f3fe764-2652-46c9-8e54-7882c0e6bcf1/ovn-acl-logging/0.log"
Oct 30 18:48:56.212553 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:48:56.212533    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xfcdl_3f3fe764-2652-46c9-8e54-7882c0e6bcf1/ovn-acl-logging/0.log"
Oct 30 18:49:05.619033 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:49:05.619000    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-jbsw6" containerName="extractor"
Oct 30 18:49:17.213650 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:49:17.213607    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-rjmdz"]
Oct 30 18:49:23.796104 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:49:23.796071    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-66db998887-wlrgd"]
Oct 30 18:49:23.796549 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:49:23.796278    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-wlrgd" podUID="36c8e780-ee5d-41e1-899f-99b9614c84ad" containerName="istio-proxy" containerID="cri-o://dbad0b77cb998b7ee7ebd31f8462c53b27dd7ceae36ea003f51590f332358a1a" gracePeriod=30
Oct 30 18:49:29.052519 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:49:29.052489    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-wlrgd"
Oct 30 18:49:29.205874 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:49:29.205836    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/36c8e780-ee5d-41e1-899f-99b9614c84ad-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/36c8e780-ee5d-41e1-899f-99b9614c84ad-credential-socket\") pod \"36c8e780-ee5d-41e1-899f-99b9614c84ad\" (UID: \"36c8e780-ee5d-41e1-899f-99b9614c84ad\") "
Oct 30 18:49:29.206071 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:49:29.205892    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/36c8e780-ee5d-41e1-899f-99b9614c84ad-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/36c8e780-ee5d-41e1-899f-99b9614c84ad-istiod-ca-cert\") pod \"36c8e780-ee5d-41e1-899f-99b9614c84ad\" (UID: \"36c8e780-ee5d-41e1-899f-99b9614c84ad\") "
Oct 30 18:49:29.206071 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:49:29.205934    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/36c8e780-ee5d-41e1-899f-99b9614c84ad-istio-token\" (UniqueName: \"kubernetes.io/projected/36c8e780-ee5d-41e1-899f-99b9614c84ad-istio-token\") pod \"36c8e780-ee5d-41e1-899f-99b9614c84ad\" (UID: \"36c8e780-ee5d-41e1-899f-99b9614c84ad\") "
Oct 30 18:49:29.206071 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:49:29.205962    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/36c8e780-ee5d-41e1-899f-99b9614c84ad-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/36c8e780-ee5d-41e1-899f-99b9614c84ad-istio-envoy\") pod \"36c8e780-ee5d-41e1-899f-99b9614c84ad\" (UID: \"36c8e780-ee5d-41e1-899f-99b9614c84ad\") "
Oct 30 18:49:29.206337 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:49:29.206108    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36c8e780-ee5d-41e1-899f-99b9614c84ad-credential-socket" pod "36c8e780-ee5d-41e1-899f-99b9614c84ad" (UID: "36c8e780-ee5d-41e1-899f-99b9614c84ad"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:49:29.206337 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:49:29.206119    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/36c8e780-ee5d-41e1-899f-99b9614c84ad-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/36c8e780-ee5d-41e1-899f-99b9614c84ad-istio-data\") pod \"36c8e780-ee5d-41e1-899f-99b9614c84ad\" (UID: \"36c8e780-ee5d-41e1-899f-99b9614c84ad\") "
Oct 30 18:49:29.206337 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:49:29.206183    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/36c8e780-ee5d-41e1-899f-99b9614c84ad-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/36c8e780-ee5d-41e1-899f-99b9614c84ad-workload-certs\") pod \"36c8e780-ee5d-41e1-899f-99b9614c84ad\" (UID: \"36c8e780-ee5d-41e1-899f-99b9614c84ad\") "
Oct 30 18:49:29.206337 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:49:29.206235    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/36c8e780-ee5d-41e1-899f-99b9614c84ad-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/36c8e780-ee5d-41e1-899f-99b9614c84ad-workload-socket\") pod \"36c8e780-ee5d-41e1-899f-99b9614c84ad\" (UID: \"36c8e780-ee5d-41e1-899f-99b9614c84ad\") "
Oct 30 18:49:29.206337 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:49:29.206317    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/36c8e780-ee5d-41e1-899f-99b9614c84ad-kube-api-access-2kmwl\" (UniqueName: \"kubernetes.io/projected/36c8e780-ee5d-41e1-899f-99b9614c84ad-kube-api-access-2kmwl\") pod \"36c8e780-ee5d-41e1-899f-99b9614c84ad\" (UID: \"36c8e780-ee5d-41e1-899f-99b9614c84ad\") "
Oct 30 18:49:29.206630 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:49:29.206369    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/36c8e780-ee5d-41e1-899f-99b9614c84ad-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/36c8e780-ee5d-41e1-899f-99b9614c84ad-istio-podinfo\") pod \"36c8e780-ee5d-41e1-899f-99b9614c84ad\" (UID: \"36c8e780-ee5d-41e1-899f-99b9614c84ad\") "
Oct 30 18:49:29.206630 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:49:29.206391    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36c8e780-ee5d-41e1-899f-99b9614c84ad-istio-data" pod "36c8e780-ee5d-41e1-899f-99b9614c84ad" (UID: "36c8e780-ee5d-41e1-899f-99b9614c84ad"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:49:29.206630 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:49:29.206422    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/36c8e780-ee5d-41e1-899f-99b9614c84ad-istiod-ca-cert" pod "36c8e780-ee5d-41e1-899f-99b9614c84ad" (UID: "36c8e780-ee5d-41e1-899f-99b9614c84ad"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:49:29.206630 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:49:29.206616    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36c8e780-ee5d-41e1-899f-99b9614c84ad-workload-socket" pod "36c8e780-ee5d-41e1-899f-99b9614c84ad" (UID: "36c8e780-ee5d-41e1-899f-99b9614c84ad"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:49:29.207000 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:49:29.206781    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36c8e780-ee5d-41e1-899f-99b9614c84ad-workload-certs" pod "36c8e780-ee5d-41e1-899f-99b9614c84ad" (UID: "36c8e780-ee5d-41e1-899f-99b9614c84ad"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:49:29.207000 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:49:29.206799    4888 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/36c8e780-ee5d-41e1-899f-99b9614c84ad-credential-socket\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:49:29.207000 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:49:29.206817    4888 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/36c8e780-ee5d-41e1-899f-99b9614c84ad-istiod-ca-cert\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:49:29.207000 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:49:29.206826    4888 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/36c8e780-ee5d-41e1-899f-99b9614c84ad-istio-data\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:49:29.207000 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:49:29.206835    4888 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/36c8e780-ee5d-41e1-899f-99b9614c84ad-workload-socket\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:49:29.208406 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:49:29.208386    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36c8e780-ee5d-41e1-899f-99b9614c84ad-istio-envoy" pod "36c8e780-ee5d-41e1-899f-99b9614c84ad" (UID: "36c8e780-ee5d-41e1-899f-99b9614c84ad"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:49:29.208519 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:49:29.208483    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/36c8e780-ee5d-41e1-899f-99b9614c84ad-istio-token" pod "36c8e780-ee5d-41e1-899f-99b9614c84ad" (UID: "36c8e780-ee5d-41e1-899f-99b9614c84ad"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:49:29.209029 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:49:29.209002    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/36c8e780-ee5d-41e1-899f-99b9614c84ad-istio-podinfo" pod "36c8e780-ee5d-41e1-899f-99b9614c84ad" (UID: "36c8e780-ee5d-41e1-899f-99b9614c84ad"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 18:49:29.209029 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:49:29.209005    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/36c8e780-ee5d-41e1-899f-99b9614c84ad-kube-api-access-2kmwl" pod "36c8e780-ee5d-41e1-899f-99b9614c84ad" (UID: "36c8e780-ee5d-41e1-899f-99b9614c84ad"). InnerVolumeSpecName "kube-api-access-2kmwl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:49:29.308412 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:49:29.308381    4888 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/36c8e780-ee5d-41e1-899f-99b9614c84ad-workload-certs\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:49:29.308412 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:49:29.308412    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2kmwl\" (UniqueName: \"kubernetes.io/projected/36c8e780-ee5d-41e1-899f-99b9614c84ad-kube-api-access-2kmwl\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:49:29.308638 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:49:29.308426    4888 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/36c8e780-ee5d-41e1-899f-99b9614c84ad-istio-podinfo\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:49:29.308638 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:49:29.308439    4888 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/36c8e780-ee5d-41e1-899f-99b9614c84ad-istio-token\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:49:29.308638 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:49:29.308453    4888 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/36c8e780-ee5d-41e1-899f-99b9614c84ad-istio-envoy\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:49:29.665401 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:49:29.665369    4888 generic.go:356] "Generic (PLEG): container finished" podID="36c8e780-ee5d-41e1-899f-99b9614c84ad" containerID="dbad0b77cb998b7ee7ebd31f8462c53b27dd7ceae36ea003f51590f332358a1a" exitCode=0
Oct 30 18:49:29.665573 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:49:29.665417    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-wlrgd"
Oct 30 18:49:29.665573 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:49:29.665453    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-wlrgd" event={"ID":"36c8e780-ee5d-41e1-899f-99b9614c84ad","Type":"ContainerDied","Data":"dbad0b77cb998b7ee7ebd31f8462c53b27dd7ceae36ea003f51590f332358a1a"}
Oct 30 18:49:29.665573 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:49:29.665504    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-wlrgd" event={"ID":"36c8e780-ee5d-41e1-899f-99b9614c84ad","Type":"ContainerDied","Data":"5d5fe36e4064275e67dde7c3a43737ec570a4f928d03712f5298b8b1fb847577"}
Oct 30 18:49:29.665573 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:49:29.665524    4888 scope.go:122] "RemoveContainer" containerID="dbad0b77cb998b7ee7ebd31f8462c53b27dd7ceae36ea003f51590f332358a1a"
Oct 30 18:49:29.676365 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:49:29.676348    4888 scope.go:122] "RemoveContainer" containerID="dbad0b77cb998b7ee7ebd31f8462c53b27dd7ceae36ea003f51590f332358a1a"
Oct 30 18:49:29.676657 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:49:29.676640    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dbad0b77cb998b7ee7ebd31f8462c53b27dd7ceae36ea003f51590f332358a1a\": container with ID starting with dbad0b77cb998b7ee7ebd31f8462c53b27dd7ceae36ea003f51590f332358a1a not found: ID does not exist" containerID="dbad0b77cb998b7ee7ebd31f8462c53b27dd7ceae36ea003f51590f332358a1a"
Oct 30 18:49:29.676705 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:49:29.676665    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dbad0b77cb998b7ee7ebd31f8462c53b27dd7ceae36ea003f51590f332358a1a"} err="failed to get container status \"dbad0b77cb998b7ee7ebd31f8462c53b27dd7ceae36ea003f51590f332358a1a\": rpc error: code = NotFound desc = could not find container \"dbad0b77cb998b7ee7ebd31f8462c53b27dd7ceae36ea003f51590f332358a1a\": container with ID starting with dbad0b77cb998b7ee7ebd31f8462c53b27dd7ceae36ea003f51590f332358a1a not found: ID does not exist"
Oct 30 18:49:29.835663 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:49:29.835625    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-66db998887-wlrgd"]
Oct 30 18:49:29.841604 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:49:29.841581    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-66db998887-wlrgd"]
Oct 30 18:49:29.890000 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:49:29.889970    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="36c8e780-ee5d-41e1-899f-99b9614c84ad" path="/var/lib/kubelet/pods/36c8e780-ee5d-41e1-899f-99b9614c84ad/volumes"
Oct 30 18:49:35.617873 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:49:35.617782    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-jbsw6" containerName="extractor"
Oct 30 18:49:51.273298 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:49:51.273242    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-rjmdz"]
Oct 30 18:49:56.479478 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:49:56.479392    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["models-as-a-service/payload-processing-external-model-d8bf55b5b-xnpxw"]
Oct 30 18:49:56.479930 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:49:56.479913    4888 cpu_manager.go:414] "containerMap: removing container" podUID="36c8e780-ee5d-41e1-899f-99b9614c84ad" containerName="istio-proxy"
Oct 30 18:49:56.479978 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:49:56.479932    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="36c8e780-ee5d-41e1-899f-99b9614c84ad" containerName="istio-proxy"
Oct 30 18:49:56.480030 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:49:56.480021    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="36c8e780-ee5d-41e1-899f-99b9614c84ad" containerName="istio-proxy"
Oct 30 18:49:56.484814 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:49:56.484796    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-d8bf55b5b-xnpxw"
Oct 30 18:49:56.543167 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:49:56.543133    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["models-as-a-service/payload-processing-external-model-d8bf55b5b-xnpxw"]
Oct 30 18:49:56.590241 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:49:56.590210    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"models-as-a-service\"/\"agc-cred-0c229ede8440\""
Oct 30 18:49:56.590453 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:49:56.590299    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"models-as-a-service\"/\"openshift-service-ca.crt\""
Oct 30 18:49:56.590453 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:49:56.590307    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"models-as-a-service\"/\"kube-root-ca.crt\""
Oct 30 18:49:56.590453 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:49:56.590221    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"models-as-a-service\"/\"routing-overlay\""
Oct 30 18:49:56.590453 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:49:56.590368    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"models-as-a-service\"/\"payload-processing-external-model-dockercfg-sqhtd\""
Oct 30 18:49:56.590453 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:49:56.590440    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"models-as-a-service\"/\"payload-processing-external-model-plugins\""
Oct 30 18:49:56.590453 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:49:56.590441    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"models-as-a-service\"/\"agc-cred-01eb9baaf090\""
Oct 30 18:49:56.602932 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:49:56.602899    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xmfsw\" (UniqueName: \"kubernetes.io/projected/bb0e7476-6297-4d0d-a5c6-a15f7d5c63d2-kube-api-access-xmfsw\") pod \"payload-processing-external-model-d8bf55b5b-xnpxw\" (UID: \"bb0e7476-6297-4d0d-a5c6-a15f7d5c63d2\") " pod="models-as-a-service/payload-processing-external-model-d8bf55b5b-xnpxw"
Oct 30 18:49:56.602932 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:49:56.602931    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bb0e7476-6297-4d0d-a5c6-a15f7d5c63d2-plugins-config-volume\") pod \"payload-processing-external-model-d8bf55b5b-xnpxw\" (UID: \"bb0e7476-6297-4d0d-a5c6-a15f7d5c63d2\") " pod="models-as-a-service/payload-processing-external-model-d8bf55b5b-xnpxw"
Oct 30 18:49:56.603131 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:49:56.602955    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/bb0e7476-6297-4d0d-a5c6-a15f7d5c63d2-routing-overlay-volume\") pod \"payload-processing-external-model-d8bf55b5b-xnpxw\" (UID: \"bb0e7476-6297-4d0d-a5c6-a15f7d5c63d2\") " pod="models-as-a-service/payload-processing-external-model-d8bf55b5b-xnpxw"
Oct 30 18:49:56.603131 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:49:56.603053    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/bb0e7476-6297-4d0d-a5c6-a15f7d5c63d2-provider-credentials\") pod \"payload-processing-external-model-d8bf55b5b-xnpxw\" (UID: \"bb0e7476-6297-4d0d-a5c6-a15f7d5c63d2\") " pod="models-as-a-service/payload-processing-external-model-d8bf55b5b-xnpxw"
Oct 30 18:49:56.703787 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:49:56.703751    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xmfsw\" (UniqueName: \"kubernetes.io/projected/bb0e7476-6297-4d0d-a5c6-a15f7d5c63d2-kube-api-access-xmfsw\") pod \"payload-processing-external-model-d8bf55b5b-xnpxw\" (UID: \"bb0e7476-6297-4d0d-a5c6-a15f7d5c63d2\") " pod="models-as-a-service/payload-processing-external-model-d8bf55b5b-xnpxw"
Oct 30 18:49:56.703787 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:49:56.703793    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bb0e7476-6297-4d0d-a5c6-a15f7d5c63d2-plugins-config-volume\") pod \"payload-processing-external-model-d8bf55b5b-xnpxw\" (UID: \"bb0e7476-6297-4d0d-a5c6-a15f7d5c63d2\") " pod="models-as-a-service/payload-processing-external-model-d8bf55b5b-xnpxw"
Oct 30 18:49:56.704072 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:49:56.703817    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/bb0e7476-6297-4d0d-a5c6-a15f7d5c63d2-routing-overlay-volume\") pod \"payload-processing-external-model-d8bf55b5b-xnpxw\" (UID: \"bb0e7476-6297-4d0d-a5c6-a15f7d5c63d2\") " pod="models-as-a-service/payload-processing-external-model-d8bf55b5b-xnpxw"
Oct 30 18:49:56.704072 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:49:56.703856    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/bb0e7476-6297-4d0d-a5c6-a15f7d5c63d2-provider-credentials\") pod \"payload-processing-external-model-d8bf55b5b-xnpxw\" (UID: \"bb0e7476-6297-4d0d-a5c6-a15f7d5c63d2\") " pod="models-as-a-service/payload-processing-external-model-d8bf55b5b-xnpxw"
Oct 30 18:49:56.704605 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:49:56.704579    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/bb0e7476-6297-4d0d-a5c6-a15f7d5c63d2-routing-overlay-volume\") pod \"payload-processing-external-model-d8bf55b5b-xnpxw\" (UID: \"bb0e7476-6297-4d0d-a5c6-a15f7d5c63d2\") " pod="models-as-a-service/payload-processing-external-model-d8bf55b5b-xnpxw"
Oct 30 18:49:56.704894 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:49:56.704867    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bb0e7476-6297-4d0d-a5c6-a15f7d5c63d2-plugins-config-volume\") pod \"payload-processing-external-model-d8bf55b5b-xnpxw\" (UID: \"bb0e7476-6297-4d0d-a5c6-a15f7d5c63d2\") " pod="models-as-a-service/payload-processing-external-model-d8bf55b5b-xnpxw"
Oct 30 18:49:56.706622 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:49:56.706599    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/bb0e7476-6297-4d0d-a5c6-a15f7d5c63d2-provider-credentials\") pod \"payload-processing-external-model-d8bf55b5b-xnpxw\" (UID: \"bb0e7476-6297-4d0d-a5c6-a15f7d5c63d2\") " pod="models-as-a-service/payload-processing-external-model-d8bf55b5b-xnpxw"
Oct 30 18:49:56.766900 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:49:56.766829    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xmfsw\" (UniqueName: \"kubernetes.io/projected/bb0e7476-6297-4d0d-a5c6-a15f7d5c63d2-kube-api-access-xmfsw\") pod \"payload-processing-external-model-d8bf55b5b-xnpxw\" (UID: \"bb0e7476-6297-4d0d-a5c6-a15f7d5c63d2\") " pod="models-as-a-service/payload-processing-external-model-d8bf55b5b-xnpxw"
Oct 30 18:49:56.799732 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:49:56.799698    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-d8bf55b5b-xnpxw"
Oct 30 18:49:57.185955 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:49:57.185927    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["models-as-a-service/payload-processing-external-model-d8bf55b5b-xnpxw"]
Oct 30 18:49:57.188070 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:49:57.188040    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb0e7476_6297_4d0d_a5c6_a15f7d5c63d2.slice/crio-f31f838fb52e26609c30acc48abaf280a0b3ece6e2913b9e79f087bd67d60ab9 WatchSource:0}: Error finding container f31f838fb52e26609c30acc48abaf280a0b3ece6e2913b9e79f087bd67d60ab9: Status 404 returned error can't find the container with id f31f838fb52e26609c30acc48abaf280a0b3ece6e2913b9e79f087bd67d60ab9
Oct 30 18:49:57.904342 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:49:57.904305    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-d8bf55b5b-xnpxw" event={"ID":"bb0e7476-6297-4d0d-a5c6-a15f7d5c63d2","Type":"ContainerStarted","Data":"d870d23641019d3061f37433dbf085d0e133c5d3562e8c132c08766854cfbea2"}
Oct 30 18:49:57.904868 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:49:57.904363    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-d8bf55b5b-xnpxw" event={"ID":"bb0e7476-6297-4d0d-a5c6-a15f7d5c63d2","Type":"ContainerStarted","Data":"f31f838fb52e26609c30acc48abaf280a0b3ece6e2913b9e79f087bd67d60ab9"}
Oct 30 18:49:57.904868 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:49:57.904457    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="models-as-a-service/payload-processing-external-model-d8bf55b5b-xnpxw"
Oct 30 18:49:58.026563 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:49:58.026486    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="models-as-a-service/payload-processing-external-model-d8bf55b5b-xnpxw" podStartSLOduration=2.026466408 podStartE2EDuration="2.026466408s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:49:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:49:58.024969428 +0000 UTC m=+962.272375454" watchObservedRunningTime="2026-10-30 18:49:58.026466408 +0000 UTC m=+962.273872436"
Oct 30 18:50:05.639849 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:50:05.639798    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-jbsw6" containerName="extractor"
Oct 30 18:50:06.060328 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:50:06.060235    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-q8jkp"]
Oct 30 18:50:06.064456 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:50:06.064438    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-q8jkp"
Oct 30 18:50:06.117957 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:50:06.117923    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eb4ed8c8-8816-4b42-803e-3dfc9a504893-istio-podinfo\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-q8jkp\" (UID: \"eb4ed8c8-8816-4b42-803e-3dfc9a504893\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-q8jkp"
Oct 30 18:50:06.118132 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:50:06.117995    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eb4ed8c8-8816-4b42-803e-3dfc9a504893-istiod-ca-cert\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-q8jkp\" (UID: \"eb4ed8c8-8816-4b42-803e-3dfc9a504893\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-q8jkp"
Oct 30 18:50:06.118132 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:50:06.118032    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eb4ed8c8-8816-4b42-803e-3dfc9a504893-istio-envoy\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-q8jkp\" (UID: \"eb4ed8c8-8816-4b42-803e-3dfc9a504893\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-q8jkp"
Oct 30 18:50:06.118132 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:50:06.118061    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb4ed8c8-8816-4b42-803e-3dfc9a504893-workload-socket\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-q8jkp\" (UID: \"eb4ed8c8-8816-4b42-803e-3dfc9a504893\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-q8jkp"
Oct 30 18:50:06.118132 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:50:06.118109    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eb4ed8c8-8816-4b42-803e-3dfc9a504893-istio-data\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-q8jkp\" (UID: \"eb4ed8c8-8816-4b42-803e-3dfc9a504893\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-q8jkp"
Oct 30 18:50:06.118462 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:50:06.118149    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eb4ed8c8-8816-4b42-803e-3dfc9a504893-workload-certs\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-q8jkp\" (UID: \"eb4ed8c8-8816-4b42-803e-3dfc9a504893\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-q8jkp"
Oct 30 18:50:06.118462 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:50:06.118197    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eb4ed8c8-8816-4b42-803e-3dfc9a504893-istio-token\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-q8jkp\" (UID: \"eb4ed8c8-8816-4b42-803e-3dfc9a504893\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-q8jkp"
Oct 30 18:50:06.118462 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:50:06.118229    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb4ed8c8-8816-4b42-803e-3dfc9a504893-credential-socket\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-q8jkp\" (UID: \"eb4ed8c8-8816-4b42-803e-3dfc9a504893\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-q8jkp"
Oct 30 18:50:06.118462 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:50:06.118319    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j8rnj\" (UniqueName: \"kubernetes.io/projected/eb4ed8c8-8816-4b42-803e-3dfc9a504893-kube-api-access-j8rnj\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-q8jkp\" (UID: \"eb4ed8c8-8816-4b42-803e-3dfc9a504893\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-q8jkp"
Oct 30 18:50:06.118861 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:50:06.118836    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w3-3b6013-openshift-default-dockercfg-4sf9f\""
Oct 30 18:50:06.193626 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:50:06.193587    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-q8jkp"]
Oct 30 18:50:06.220326 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:50:06.220271    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eb4ed8c8-8816-4b42-803e-3dfc9a504893-istio-data\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-q8jkp\" (UID: \"eb4ed8c8-8816-4b42-803e-3dfc9a504893\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-q8jkp"
Oct 30 18:50:06.221257 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:50:06.221234    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eb4ed8c8-8816-4b42-803e-3dfc9a504893-workload-certs\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-q8jkp\" (UID: \"eb4ed8c8-8816-4b42-803e-3dfc9a504893\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-q8jkp"
Oct 30 18:50:06.221786 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:50:06.221767    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eb4ed8c8-8816-4b42-803e-3dfc9a504893-istio-token\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-q8jkp\" (UID: \"eb4ed8c8-8816-4b42-803e-3dfc9a504893\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-q8jkp"
Oct 30 18:50:06.222303 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:50:06.222262    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb4ed8c8-8816-4b42-803e-3dfc9a504893-credential-socket\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-q8jkp\" (UID: \"eb4ed8c8-8816-4b42-803e-3dfc9a504893\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-q8jkp"
Oct 30 18:50:06.226306 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:50:06.222791    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j8rnj\" (UniqueName: \"kubernetes.io/projected/eb4ed8c8-8816-4b42-803e-3dfc9a504893-kube-api-access-j8rnj\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-q8jkp\" (UID: \"eb4ed8c8-8816-4b42-803e-3dfc9a504893\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-q8jkp"
Oct 30 18:50:06.226796 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:50:06.226774    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eb4ed8c8-8816-4b42-803e-3dfc9a504893-istio-podinfo\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-q8jkp\" (UID: \"eb4ed8c8-8816-4b42-803e-3dfc9a504893\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-q8jkp"
Oct 30 18:50:06.227409 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:50:06.227387    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eb4ed8c8-8816-4b42-803e-3dfc9a504893-istiod-ca-cert\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-q8jkp\" (UID: \"eb4ed8c8-8816-4b42-803e-3dfc9a504893\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-q8jkp"
Oct 30 18:50:06.239637 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:50:06.221158    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eb4ed8c8-8816-4b42-803e-3dfc9a504893-istio-data\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-q8jkp\" (UID: \"eb4ed8c8-8816-4b42-803e-3dfc9a504893\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-q8jkp"
Oct 30 18:50:06.242008 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:50:06.221689    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eb4ed8c8-8816-4b42-803e-3dfc9a504893-workload-certs\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-q8jkp\" (UID: \"eb4ed8c8-8816-4b42-803e-3dfc9a504893\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-q8jkp"
Oct 30 18:50:06.242008 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:50:06.222695    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb4ed8c8-8816-4b42-803e-3dfc9a504893-credential-socket\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-q8jkp\" (UID: \"eb4ed8c8-8816-4b42-803e-3dfc9a504893\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-q8jkp"
Oct 30 18:50:06.242008 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:50:06.228174    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eb4ed8c8-8816-4b42-803e-3dfc9a504893-istiod-ca-cert\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-q8jkp\" (UID: \"eb4ed8c8-8816-4b42-803e-3dfc9a504893\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-q8jkp"
Oct 30 18:50:06.242008 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:50:06.228250    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eb4ed8c8-8816-4b42-803e-3dfc9a504893-istio-envoy\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-q8jkp\" (UID: \"eb4ed8c8-8816-4b42-803e-3dfc9a504893\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-q8jkp"
Oct 30 18:50:06.242008 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:50:06.240232    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb4ed8c8-8816-4b42-803e-3dfc9a504893-workload-socket\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-q8jkp\" (UID: \"eb4ed8c8-8816-4b42-803e-3dfc9a504893\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-q8jkp"
Oct 30 18:50:06.242008 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:50:06.240602    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb4ed8c8-8816-4b42-803e-3dfc9a504893-workload-socket\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-q8jkp\" (UID: \"eb4ed8c8-8816-4b42-803e-3dfc9a504893\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-q8jkp"
Oct 30 18:50:06.242008 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:50:06.240089    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eb4ed8c8-8816-4b42-803e-3dfc9a504893-istio-envoy\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-q8jkp\" (UID: \"eb4ed8c8-8816-4b42-803e-3dfc9a504893\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-q8jkp"
Oct 30 18:50:06.242008 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:50:06.241723    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eb4ed8c8-8816-4b42-803e-3dfc9a504893-istio-podinfo\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-q8jkp\" (UID: \"eb4ed8c8-8816-4b42-803e-3dfc9a504893\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-q8jkp"
Oct 30 18:50:06.337136 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:50:06.337106    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j8rnj\" (UniqueName: \"kubernetes.io/projected/eb4ed8c8-8816-4b42-803e-3dfc9a504893-kube-api-access-j8rnj\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-q8jkp\" (UID: \"eb4ed8c8-8816-4b42-803e-3dfc9a504893\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-q8jkp"
Oct 30 18:50:06.340389 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:50:06.340328    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eb4ed8c8-8816-4b42-803e-3dfc9a504893-istio-token\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-q8jkp\" (UID: \"eb4ed8c8-8816-4b42-803e-3dfc9a504893\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-q8jkp"
Oct 30 18:50:06.411637 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:50:06.411310    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-q8jkp"
Oct 30 18:50:06.897026 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:50:06.896990    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-q8jkp"]
Oct 30 18:50:06.935054 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:50:06.935019    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb4ed8c8_8816_4b42_803e_3dfc9a504893.slice/crio-ac919feb8d66f4360bc602358f72ab2abae196663ff6d61d7207f4601c86330d WatchSource:0}: Error finding container ac919feb8d66f4360bc602358f72ab2abae196663ff6d61d7207f4601c86330d: Status 404 returned error can't find the container with id ac919feb8d66f4360bc602358f72ab2abae196663ff6d61d7207f4601c86330d
Oct 30 18:50:06.937959 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:50:06.937918    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:50:06.938054 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:50:06.938011    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:50:06.938104 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:50:06.938056    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:50:06.979553 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:50:06.979507    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-q8jkp" event={"ID":"eb4ed8c8-8816-4b42-803e-3dfc9a504893","Type":"ContainerStarted","Data":"ac919feb8d66f4360bc602358f72ab2abae196663ff6d61d7207f4601c86330d"}
Oct 30 18:50:08.000829 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:50:08.000795    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-q8jkp" event={"ID":"eb4ed8c8-8816-4b42-803e-3dfc9a504893","Type":"ContainerStarted","Data":"b698322c35d5de0c2a1f0ad67eb051a2d1d880ec311800ba99961c38aa77a24f"}
Oct 30 18:50:08.240002 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:50:08.239946    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-q8jkp" podStartSLOduration=2.239928883 podStartE2EDuration="2.239928883s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:50:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:50:08.187831898 +0000 UTC m=+972.435237924" watchObservedRunningTime="2026-10-30 18:50:08.239928883 +0000 UTC m=+972.487334978"
Oct 30 18:50:08.412186 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:50:08.412151    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-q8jkp"
Oct 30 18:50:08.413540 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:50:08.413520    4888 patch_prober.go:29] interesting pod/e2e-worker-w3-3b6013-openshift-default-85cf856477-q8jkp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" start-of-body=
Oct 30 18:50:08.413594 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:50:08.413565    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-q8jkp" podUID="eb4ed8c8-8816-4b42-803e-3dfc9a504893" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused"
Oct 30 18:50:08.923406 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:50:08.923375    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="models-as-a-service/payload-processing-external-model-d8bf55b5b-xnpxw"
Oct 30 18:50:09.248807 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:50:09.248720    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-66db998887-jsv6r"]
Oct 30 18:50:09.253390 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:50:09.253368    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-jsv6r"
Oct 30 18:50:09.297558 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:50:09.297524    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/74852e36-70a6-48f9-8f69-61f180f3c772-credential-socket\") pod \"maas-default-gateway-openshift-default-66db998887-jsv6r\" (UID: \"74852e36-70a6-48f9-8f69-61f180f3c772\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-jsv6r"
Oct 30 18:50:09.297812 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:50:09.297792    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pzrqj\" (UniqueName: \"kubernetes.io/projected/74852e36-70a6-48f9-8f69-61f180f3c772-kube-api-access-pzrqj\") pod \"maas-default-gateway-openshift-default-66db998887-jsv6r\" (UID: \"74852e36-70a6-48f9-8f69-61f180f3c772\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-jsv6r"
Oct 30 18:50:09.297978 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:50:09.297962    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/74852e36-70a6-48f9-8f69-61f180f3c772-istio-data\") pod \"maas-default-gateway-openshift-default-66db998887-jsv6r\" (UID: \"74852e36-70a6-48f9-8f69-61f180f3c772\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-jsv6r"
Oct 30 18:50:09.298087 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:50:09.298072    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/74852e36-70a6-48f9-8f69-61f180f3c772-workload-socket\") pod \"maas-default-gateway-openshift-default-66db998887-jsv6r\" (UID: \"74852e36-70a6-48f9-8f69-61f180f3c772\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-jsv6r"
Oct 30 18:50:09.298196 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:50:09.298180    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/74852e36-70a6-48f9-8f69-61f180f3c772-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-66db998887-jsv6r\" (UID: \"74852e36-70a6-48f9-8f69-61f180f3c772\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-jsv6r"
Oct 30 18:50:09.298333 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:50:09.298315    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/74852e36-70a6-48f9-8f69-61f180f3c772-istio-envoy\") pod \"maas-default-gateway-openshift-default-66db998887-jsv6r\" (UID: \"74852e36-70a6-48f9-8f69-61f180f3c772\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-jsv6r"
Oct 30 18:50:09.298551 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:50:09.298522    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/74852e36-70a6-48f9-8f69-61f180f3c772-workload-certs\") pod \"maas-default-gateway-openshift-default-66db998887-jsv6r\" (UID: \"74852e36-70a6-48f9-8f69-61f180f3c772\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-jsv6r"
Oct 30 18:50:09.298767 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:50:09.298742    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/74852e36-70a6-48f9-8f69-61f180f3c772-istio-token\") pod \"maas-default-gateway-openshift-default-66db998887-jsv6r\" (UID: \"74852e36-70a6-48f9-8f69-61f180f3c772\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-jsv6r"
Oct 30 18:50:09.298869 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:50:09.298849    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/74852e36-70a6-48f9-8f69-61f180f3c772-istio-podinfo\") pod \"maas-default-gateway-openshift-default-66db998887-jsv6r\" (UID: \"74852e36-70a6-48f9-8f69-61f180f3c772\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-jsv6r"
Oct 30 18:50:09.341911 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:50:09.341864    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-66db998887-jsv6r"]
Oct 30 18:50:09.401728 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:50:09.401698    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/74852e36-70a6-48f9-8f69-61f180f3c772-istio-data\") pod \"maas-default-gateway-openshift-default-66db998887-jsv6r\" (UID: \"74852e36-70a6-48f9-8f69-61f180f3c772\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-jsv6r"
Oct 30 18:50:09.402015 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:50:09.401984    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/74852e36-70a6-48f9-8f69-61f180f3c772-workload-socket\") pod \"maas-default-gateway-openshift-default-66db998887-jsv6r\" (UID: \"74852e36-70a6-48f9-8f69-61f180f3c772\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-jsv6r"
Oct 30 18:50:09.402148 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:50:09.402132    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/74852e36-70a6-48f9-8f69-61f180f3c772-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-66db998887-jsv6r\" (UID: \"74852e36-70a6-48f9-8f69-61f180f3c772\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-jsv6r"
Oct 30 18:50:09.402326 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:50:09.402275    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/74852e36-70a6-48f9-8f69-61f180f3c772-istio-envoy\") pod \"maas-default-gateway-openshift-default-66db998887-jsv6r\" (UID: \"74852e36-70a6-48f9-8f69-61f180f3c772\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-jsv6r"
Oct 30 18:50:09.402433 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:50:09.402415    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/74852e36-70a6-48f9-8f69-61f180f3c772-workload-certs\") pod \"maas-default-gateway-openshift-default-66db998887-jsv6r\" (UID: \"74852e36-70a6-48f9-8f69-61f180f3c772\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-jsv6r"
Oct 30 18:50:09.402540 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:50:09.402515    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/74852e36-70a6-48f9-8f69-61f180f3c772-istio-token\") pod \"maas-default-gateway-openshift-default-66db998887-jsv6r\" (UID: \"74852e36-70a6-48f9-8f69-61f180f3c772\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-jsv6r"
Oct 30 18:50:09.402664 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:50:09.402650    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/74852e36-70a6-48f9-8f69-61f180f3c772-istio-podinfo\") pod \"maas-default-gateway-openshift-default-66db998887-jsv6r\" (UID: \"74852e36-70a6-48f9-8f69-61f180f3c772\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-jsv6r"
Oct 30 18:50:09.402742 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:50:09.402688    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/74852e36-70a6-48f9-8f69-61f180f3c772-credential-socket\") pod \"maas-default-gateway-openshift-default-66db998887-jsv6r\" (UID: \"74852e36-70a6-48f9-8f69-61f180f3c772\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-jsv6r"
Oct 30 18:50:09.402742 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:50:09.402714    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pzrqj\" (UniqueName: \"kubernetes.io/projected/74852e36-70a6-48f9-8f69-61f180f3c772-kube-api-access-pzrqj\") pod \"maas-default-gateway-openshift-default-66db998887-jsv6r\" (UID: \"74852e36-70a6-48f9-8f69-61f180f3c772\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-jsv6r"
Oct 30 18:50:09.403361 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:50:09.403340    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/74852e36-70a6-48f9-8f69-61f180f3c772-istio-data\") pod \"maas-default-gateway-openshift-default-66db998887-jsv6r\" (UID: \"74852e36-70a6-48f9-8f69-61f180f3c772\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-jsv6r"
Oct 30 18:50:09.403597 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:50:09.403578    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/74852e36-70a6-48f9-8f69-61f180f3c772-workload-socket\") pod \"maas-default-gateway-openshift-default-66db998887-jsv6r\" (UID: \"74852e36-70a6-48f9-8f69-61f180f3c772\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-jsv6r"
Oct 30 18:50:09.404146 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:50:09.404125    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/74852e36-70a6-48f9-8f69-61f180f3c772-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-66db998887-jsv6r\" (UID: \"74852e36-70a6-48f9-8f69-61f180f3c772\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-jsv6r"
Oct 30 18:50:09.404521 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:50:09.404501    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/74852e36-70a6-48f9-8f69-61f180f3c772-workload-certs\") pod \"maas-default-gateway-openshift-default-66db998887-jsv6r\" (UID: \"74852e36-70a6-48f9-8f69-61f180f3c772\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-jsv6r"
Oct 30 18:50:09.404609 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:50:09.404566    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/74852e36-70a6-48f9-8f69-61f180f3c772-credential-socket\") pod \"maas-default-gateway-openshift-default-66db998887-jsv6r\" (UID: \"74852e36-70a6-48f9-8f69-61f180f3c772\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-jsv6r"
Oct 30 18:50:09.406427 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:50:09.406409    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/74852e36-70a6-48f9-8f69-61f180f3c772-istio-envoy\") pod \"maas-default-gateway-openshift-default-66db998887-jsv6r\" (UID: \"74852e36-70a6-48f9-8f69-61f180f3c772\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-jsv6r"
Oct 30 18:50:09.407692 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:50:09.407672    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/74852e36-70a6-48f9-8f69-61f180f3c772-istio-podinfo\") pod \"maas-default-gateway-openshift-default-66db998887-jsv6r\" (UID: \"74852e36-70a6-48f9-8f69-61f180f3c772\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-jsv6r"
Oct 30 18:50:09.412689 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:50:09.412665    4888 patch_prober.go:29] interesting pod/e2e-worker-w3-3b6013-openshift-default-85cf856477-q8jkp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" start-of-body=
Oct 30 18:50:09.412748 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:50:09.412714    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-q8jkp" podUID="eb4ed8c8-8816-4b42-803e-3dfc9a504893" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused"
Oct 30 18:50:09.498365 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:50:09.491665    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pzrqj\" (UniqueName: \"kubernetes.io/projected/74852e36-70a6-48f9-8f69-61f180f3c772-kube-api-access-pzrqj\") pod \"maas-default-gateway-openshift-default-66db998887-jsv6r\" (UID: \"74852e36-70a6-48f9-8f69-61f180f3c772\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-jsv6r"
Oct 30 18:50:09.499161 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:50:09.499098    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/74852e36-70a6-48f9-8f69-61f180f3c772-istio-token\") pod \"maas-default-gateway-openshift-default-66db998887-jsv6r\" (UID: \"74852e36-70a6-48f9-8f69-61f180f3c772\") " pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-jsv6r"
Oct 30 18:50:09.593554 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:50:09.593521    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-jsv6r"
Oct 30 18:50:10.090266 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:50:10.089512    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:50:10.090266 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:50:10.089608    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:50:10.090266 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:50:10.089648    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:50:10.097473 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:50:10.096614    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-66db998887-jsv6r"]
Oct 30 18:50:10.412272 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:50:10.412236    4888 patch_prober.go:29] interesting pod/e2e-worker-w3-3b6013-openshift-default-85cf856477-q8jkp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" start-of-body=
Oct 30 18:50:10.412790 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:50:10.412310    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-q8jkp" podUID="eb4ed8c8-8816-4b42-803e-3dfc9a504893" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused"
Oct 30 18:50:11.031806 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:50:11.031776    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-jsv6r" event={"ID":"74852e36-70a6-48f9-8f69-61f180f3c772","Type":"ContainerStarted","Data":"80f094a9a59c2bca8846d63aeb73518ec2716f8f23a7cb84ebe1f4cff0f56518"}
Oct 30 18:50:11.032000 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:50:11.031829    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-jsv6r" event={"ID":"74852e36-70a6-48f9-8f69-61f180f3c772","Type":"ContainerStarted","Data":"94099d30ccf31481f785032f18f3152efed97399a26ac81276ccf3c339461322"}
Oct 30 18:50:11.163560 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:50:11.163493    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-jsv6r" podStartSLOduration=2.163474091 podStartE2EDuration="2.163474091s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:50:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:50:11.162854846 +0000 UTC m=+975.410260871" watchObservedRunningTime="2026-10-30 18:50:11.163474091 +0000 UTC m=+975.410880135"
Oct 30 18:50:11.414889 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:50:11.414849    4888 patch_prober.go:29] interesting pod/e2e-worker-w3-3b6013-openshift-default-85cf856477-q8jkp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" start-of-body=
Oct 30 18:50:11.415551 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:50:11.415522    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-q8jkp" podUID="eb4ed8c8-8816-4b42-803e-3dfc9a504893" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused"
Oct 30 18:50:11.597170 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:50:11.597138    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-jsv6r"
Oct 30 18:50:11.604310 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:50:11.601450    4888 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-66db998887-jsv6r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused" start-of-body=
Oct 30 18:50:11.604310 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:50:11.601541    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-jsv6r" podUID="74852e36-70a6-48f9-8f69-61f180f3c772" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused"
Oct 30 18:50:12.413309 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:50:12.412637    4888 patch_prober.go:29] interesting pod/e2e-worker-w3-3b6013-openshift-default-85cf856477-q8jkp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" start-of-body=
Oct 30 18:50:12.413309 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:50:12.412697    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-q8jkp" podUID="eb4ed8c8-8816-4b42-803e-3dfc9a504893" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused"
Oct 30 18:50:12.598305 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:50:12.597698    4888 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-66db998887-jsv6r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused" start-of-body=
Oct 30 18:50:12.598305 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:50:12.597759    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-jsv6r" podUID="74852e36-70a6-48f9-8f69-61f180f3c772" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused"
Oct 30 18:50:13.412810 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:50:13.412773    4888 patch_prober.go:29] interesting pod/e2e-worker-w3-3b6013-openshift-default-85cf856477-q8jkp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" start-of-body=
Oct 30 18:50:13.413076 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:50:13.413048    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-q8jkp" podUID="eb4ed8c8-8816-4b42-803e-3dfc9a504893" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused"
Oct 30 18:50:13.595712 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:50:13.595670    4888 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-66db998887-jsv6r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused" start-of-body=
Oct 30 18:50:13.595996 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:50:13.595967    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-jsv6r" podUID="74852e36-70a6-48f9-8f69-61f180f3c772" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused"
Oct 30 18:50:14.417488 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:50:14.417461    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-q8jkp"
Oct 30 18:50:14.418263 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:50:14.418236    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-q8jkp"
Oct 30 18:50:14.419030 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:50:14.419010    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-q8jkp"
Oct 30 18:50:14.598320 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:50:14.595637    4888 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-66db998887-jsv6r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused" start-of-body=
Oct 30 18:50:14.598320 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:50:14.595700    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-jsv6r" podUID="74852e36-70a6-48f9-8f69-61f180f3c772" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused"
Oct 30 18:50:15.397623 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:50:15.397592    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-rjmdz"]
Oct 30 18:50:15.600716 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:50:15.600680    4888 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-66db998887-jsv6r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused" start-of-body=
Oct 30 18:50:15.601334 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:50:15.601301    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-jsv6r" podUID="74852e36-70a6-48f9-8f69-61f180f3c772" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused"
Oct 30 18:50:16.594072 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:50:16.594035    4888 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-66db998887-jsv6r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused" start-of-body=
Oct 30 18:50:16.594255 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:50:16.594116    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-jsv6r" podUID="74852e36-70a6-48f9-8f69-61f180f3c772" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused"
Oct 30 18:50:16.860210 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:50:16.860129    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-2ade09/e2e-conflict-a-d331d324-kserve-64f575d657-9zzz5"]
Oct 30 18:50:16.865121 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:50:16.865099    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-2ade09/e2e-conflict-a-d331d324-kserve-64f575d657-9zzz5"
Oct 30 18:50:16.927009 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:50:16.926982    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/27762cc5-706e-4d70-a701-ab5f82c893d2-dshm\") pod \"e2e-conflict-a-d331d324-kserve-64f575d657-9zzz5\" (UID: \"27762cc5-706e-4d70-a701-ab5f82c893d2\") " pod="e2e-models-e2e-worker-w0-2ade09/e2e-conflict-a-d331d324-kserve-64f575d657-9zzz5"
Oct 30 18:50:16.927183 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:50:16.927039    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/27762cc5-706e-4d70-a701-ab5f82c893d2-tls-certs\") pod \"e2e-conflict-a-d331d324-kserve-64f575d657-9zzz5\" (UID: \"27762cc5-706e-4d70-a701-ab5f82c893d2\") " pod="e2e-models-e2e-worker-w0-2ade09/e2e-conflict-a-d331d324-kserve-64f575d657-9zzz5"
Oct 30 18:50:16.927244 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:50:16.927190    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9x66z\" (UniqueName: \"kubernetes.io/projected/27762cc5-706e-4d70-a701-ab5f82c893d2-kube-api-access-9x66z\") pod \"e2e-conflict-a-d331d324-kserve-64f575d657-9zzz5\" (UID: \"27762cc5-706e-4d70-a701-ab5f82c893d2\") " pod="e2e-models-e2e-worker-w0-2ade09/e2e-conflict-a-d331d324-kserve-64f575d657-9zzz5"
Oct 30 18:50:16.927345 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:50:16.927257    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/27762cc5-706e-4d70-a701-ab5f82c893d2-tmp-dir\") pod \"e2e-conflict-a-d331d324-kserve-64f575d657-9zzz5\" (UID: \"27762cc5-706e-4d70-a701-ab5f82c893d2\") " pod="e2e-models-e2e-worker-w0-2ade09/e2e-conflict-a-d331d324-kserve-64f575d657-9zzz5"
Oct 30 18:50:16.927544 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:50:16.927510    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/27762cc5-706e-4d70-a701-ab5f82c893d2-home\") pod \"e2e-conflict-a-d331d324-kserve-64f575d657-9zzz5\" (UID: \"27762cc5-706e-4d70-a701-ab5f82c893d2\") " pod="e2e-models-e2e-worker-w0-2ade09/e2e-conflict-a-d331d324-kserve-64f575d657-9zzz5"
Oct 30 18:50:16.927639 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:50:16.927550    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/27762cc5-706e-4d70-a701-ab5f82c893d2-model-cache\") pod \"e2e-conflict-a-d331d324-kserve-64f575d657-9zzz5\" (UID: \"27762cc5-706e-4d70-a701-ab5f82c893d2\") " pod="e2e-models-e2e-worker-w0-2ade09/e2e-conflict-a-d331d324-kserve-64f575d657-9zzz5"
Oct 30 18:50:16.932540 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:50:16.932508    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-2ade09/e2e-conflict-a-d331d324-kserve-64f575d657-9zzz5"]
Oct 30 18:50:16.980963 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:50:16.980931    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-2ade09\"/\"e2e-conflict-a-d331d324-kserve-self-signed-certs\""
Oct 30 18:50:16.980963 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:50:16.980954    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-2ade09\"/\"openshift-service-ca.crt\""
Oct 30 18:50:16.981164 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:50:16.980987    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-2ade09\"/\"default-dockercfg-mgqgf\""
Oct 30 18:50:16.981164 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:50:16.981041    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-2ade09\"/\"kube-root-ca.crt\""
Oct 30 18:50:17.031722 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:50:17.030855    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9x66z\" (UniqueName: \"kubernetes.io/projected/27762cc5-706e-4d70-a701-ab5f82c893d2-kube-api-access-9x66z\") pod \"e2e-conflict-a-d331d324-kserve-64f575d657-9zzz5\" (UID: \"27762cc5-706e-4d70-a701-ab5f82c893d2\") " pod="e2e-models-e2e-worker-w0-2ade09/e2e-conflict-a-d331d324-kserve-64f575d657-9zzz5"
Oct 30 18:50:17.031722 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:50:17.030927    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/27762cc5-706e-4d70-a701-ab5f82c893d2-tmp-dir\") pod \"e2e-conflict-a-d331d324-kserve-64f575d657-9zzz5\" (UID: \"27762cc5-706e-4d70-a701-ab5f82c893d2\") " pod="e2e-models-e2e-worker-w0-2ade09/e2e-conflict-a-d331d324-kserve-64f575d657-9zzz5"
Oct 30 18:50:17.031722 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:50:17.031100    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/27762cc5-706e-4d70-a701-ab5f82c893d2-home\") pod \"e2e-conflict-a-d331d324-kserve-64f575d657-9zzz5\" (UID: \"27762cc5-706e-4d70-a701-ab5f82c893d2\") " pod="e2e-models-e2e-worker-w0-2ade09/e2e-conflict-a-d331d324-kserve-64f575d657-9zzz5"
Oct 30 18:50:17.031722 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:50:17.031127    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/27762cc5-706e-4d70-a701-ab5f82c893d2-model-cache\") pod \"e2e-conflict-a-d331d324-kserve-64f575d657-9zzz5\" (UID: \"27762cc5-706e-4d70-a701-ab5f82c893d2\") " pod="e2e-models-e2e-worker-w0-2ade09/e2e-conflict-a-d331d324-kserve-64f575d657-9zzz5"
Oct 30 18:50:17.031722 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:50:17.031163    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/27762cc5-706e-4d70-a701-ab5f82c893d2-dshm\") pod \"e2e-conflict-a-d331d324-kserve-64f575d657-9zzz5\" (UID: \"27762cc5-706e-4d70-a701-ab5f82c893d2\") " pod="e2e-models-e2e-worker-w0-2ade09/e2e-conflict-a-d331d324-kserve-64f575d657-9zzz5"
Oct 30 18:50:17.031722 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:50:17.031213    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/27762cc5-706e-4d70-a701-ab5f82c893d2-tls-certs\") pod \"e2e-conflict-a-d331d324-kserve-64f575d657-9zzz5\" (UID: \"27762cc5-706e-4d70-a701-ab5f82c893d2\") " pod="e2e-models-e2e-worker-w0-2ade09/e2e-conflict-a-d331d324-kserve-64f575d657-9zzz5"
Oct 30 18:50:17.033384 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:50:17.033364    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/27762cc5-706e-4d70-a701-ab5f82c893d2-model-cache\") pod \"e2e-conflict-a-d331d324-kserve-64f575d657-9zzz5\" (UID: \"27762cc5-706e-4d70-a701-ab5f82c893d2\") " pod="e2e-models-e2e-worker-w0-2ade09/e2e-conflict-a-d331d324-kserve-64f575d657-9zzz5"
Oct 30 18:50:17.037187 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:50:17.037123    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/27762cc5-706e-4d70-a701-ab5f82c893d2-tmp-dir\") pod \"e2e-conflict-a-d331d324-kserve-64f575d657-9zzz5\" (UID: \"27762cc5-706e-4d70-a701-ab5f82c893d2\") " pod="e2e-models-e2e-worker-w0-2ade09/e2e-conflict-a-d331d324-kserve-64f575d657-9zzz5"
Oct 30 18:50:17.040007 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:50:17.039896    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/27762cc5-706e-4d70-a701-ab5f82c893d2-dshm\") pod \"e2e-conflict-a-d331d324-kserve-64f575d657-9zzz5\" (UID: \"27762cc5-706e-4d70-a701-ab5f82c893d2\") " pod="e2e-models-e2e-worker-w0-2ade09/e2e-conflict-a-d331d324-kserve-64f575d657-9zzz5"
Oct 30 18:50:17.040007 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:50:17.039958    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/27762cc5-706e-4d70-a701-ab5f82c893d2-tls-certs\") pod \"e2e-conflict-a-d331d324-kserve-64f575d657-9zzz5\" (UID: \"27762cc5-706e-4d70-a701-ab5f82c893d2\") " pod="e2e-models-e2e-worker-w0-2ade09/e2e-conflict-a-d331d324-kserve-64f575d657-9zzz5"
Oct 30 18:50:17.040336 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:50:17.040316    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/27762cc5-706e-4d70-a701-ab5f82c893d2-home\") pod \"e2e-conflict-a-d331d324-kserve-64f575d657-9zzz5\" (UID: \"27762cc5-706e-4d70-a701-ab5f82c893d2\") " pod="e2e-models-e2e-worker-w0-2ade09/e2e-conflict-a-d331d324-kserve-64f575d657-9zzz5"
Oct 30 18:50:17.106971 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:50:17.106940    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9x66z\" (UniqueName: \"kubernetes.io/projected/27762cc5-706e-4d70-a701-ab5f82c893d2-kube-api-access-9x66z\") pod \"e2e-conflict-a-d331d324-kserve-64f575d657-9zzz5\" (UID: \"27762cc5-706e-4d70-a701-ab5f82c893d2\") " pod="e2e-models-e2e-worker-w0-2ade09/e2e-conflict-a-d331d324-kserve-64f575d657-9zzz5"
Oct 30 18:50:17.181435 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:50:17.181344    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-2ade09/e2e-conflict-a-d331d324-kserve-64f575d657-9zzz5"
Oct 30 18:50:17.612310 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:50:17.611633    4888 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-66db998887-jsv6r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused" start-of-body=
Oct 30 18:50:17.612310 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:50:17.611691    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-jsv6r" podUID="74852e36-70a6-48f9-8f69-61f180f3c772" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused"
Oct 30 18:50:17.635137 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:50:17.635072    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-2ade09/e2e-conflict-a-d331d324-kserve-64f575d657-9zzz5"]
Oct 30 18:50:17.641590 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:50:17.641461    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod27762cc5_706e_4d70_a701_ab5f82c893d2.slice/crio-0b29f868e4137d126bb229808bfc29b34553c081f9c0c2380575f6004b9e8b15 WatchSource:0}: Error finding container 0b29f868e4137d126bb229808bfc29b34553c081f9c0c2380575f6004b9e8b15: Status 404 returned error can't find the container with id 0b29f868e4137d126bb229808bfc29b34553c081f9c0c2380575f6004b9e8b15
Oct 30 18:50:18.147300 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:50:18.147255    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-2ade09/e2e-conflict-a-d331d324-kserve-64f575d657-9zzz5" event={"ID":"27762cc5-706e-4d70-a701-ab5f82c893d2","Type":"ContainerStarted","Data":"2236c97a36263fcd0d91f1d504acbe0ea7719143a7d47b8d57dbdc02b8c167a5"}
Oct 30 18:50:18.147778 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:50:18.147315    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-2ade09/e2e-conflict-a-d331d324-kserve-64f575d657-9zzz5" event={"ID":"27762cc5-706e-4d70-a701-ab5f82c893d2","Type":"ContainerStarted","Data":"0b29f868e4137d126bb229808bfc29b34553c081f9c0c2380575f6004b9e8b15"}
Oct 30 18:50:18.184315 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:50:18.183375    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-2ade09/e2e-conflict-a-d331d324-kserve-64f575d657-9zzz5"
Oct 30 18:50:18.322327 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:50:18.317502    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-2ade09/e2e-conflict-a-d331d324-kserve-64f575d657-9zzz5" podStartSLOduration=2.317482243 podStartE2EDuration="2.317482243s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:50:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:50:18.317019743 +0000 UTC m=+982.564425769" watchObservedRunningTime="2026-10-30 18:50:18.317482243 +0000 UTC m=+982.564888269"
Oct 30 18:50:19.500545 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:50:19.500515    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-jsv6r"
Oct 30 18:50:19.501192 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:50:19.500981    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-jsv6r"
Oct 30 18:50:19.501791 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:50:19.501775    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-jsv6r"
Oct 30 18:50:27.182700 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:50:27.181869    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-2ade09/e2e-conflict-a-d331d324-kserve-64f575d657-9zzz5"
Oct 30 18:50:27.201383 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:50:27.201356    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-2ade09/e2e-conflict-a-d331d324-kserve-64f575d657-9zzz5"
Oct 30 18:50:27.236458 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:50:27.236422    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-2ade09/e2e-conflict-a-d331d324-kserve-64f575d657-9zzz5"
Oct 30 18:50:35.617436 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:50:35.617403    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-jbsw6" containerName="extractor"
Oct 30 18:50:41.136101 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:50:41.136060    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-rjmdz"]
Oct 30 18:51:05.618563 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:51:05.618465    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-jbsw6" containerName="extractor"
Oct 30 18:51:05.799141 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:51:05.799107    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-e7929d/distinct-w4-e7929d-kserve-5bc9ff5bf5-kmrwv"]
Oct 30 18:51:05.804489 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:51:05.804468    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-e7929d/distinct-w4-e7929d-kserve-5bc9ff5bf5-kmrwv"
Oct 30 18:51:05.849352 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:51:05.844442    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fgk4z\" (UniqueName: \"kubernetes.io/projected/abfd315d-a01a-42dd-9c06-2a80d3f8579a-kube-api-access-fgk4z\") pod \"distinct-w4-e7929d-kserve-5bc9ff5bf5-kmrwv\" (UID: \"abfd315d-a01a-42dd-9c06-2a80d3f8579a\") " pod="e2e-models-e2e-worker-w4-e7929d/distinct-w4-e7929d-kserve-5bc9ff5bf5-kmrwv"
Oct 30 18:51:05.849352 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:51:05.844548    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/abfd315d-a01a-42dd-9c06-2a80d3f8579a-dshm\") pod \"distinct-w4-e7929d-kserve-5bc9ff5bf5-kmrwv\" (UID: \"abfd315d-a01a-42dd-9c06-2a80d3f8579a\") " pod="e2e-models-e2e-worker-w4-e7929d/distinct-w4-e7929d-kserve-5bc9ff5bf5-kmrwv"
Oct 30 18:51:05.849352 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:51:05.844604    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/abfd315d-a01a-42dd-9c06-2a80d3f8579a-tmp-dir\") pod \"distinct-w4-e7929d-kserve-5bc9ff5bf5-kmrwv\" (UID: \"abfd315d-a01a-42dd-9c06-2a80d3f8579a\") " pod="e2e-models-e2e-worker-w4-e7929d/distinct-w4-e7929d-kserve-5bc9ff5bf5-kmrwv"
Oct 30 18:51:05.849352 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:51:05.844699    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/abfd315d-a01a-42dd-9c06-2a80d3f8579a-tls-certs\") pod \"distinct-w4-e7929d-kserve-5bc9ff5bf5-kmrwv\" (UID: \"abfd315d-a01a-42dd-9c06-2a80d3f8579a\") " pod="e2e-models-e2e-worker-w4-e7929d/distinct-w4-e7929d-kserve-5bc9ff5bf5-kmrwv"
Oct 30 18:51:05.849352 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:51:05.844758    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/abfd315d-a01a-42dd-9c06-2a80d3f8579a-home\") pod \"distinct-w4-e7929d-kserve-5bc9ff5bf5-kmrwv\" (UID: \"abfd315d-a01a-42dd-9c06-2a80d3f8579a\") " pod="e2e-models-e2e-worker-w4-e7929d/distinct-w4-e7929d-kserve-5bc9ff5bf5-kmrwv"
Oct 30 18:51:05.849352 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:51:05.844792    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/abfd315d-a01a-42dd-9c06-2a80d3f8579a-model-cache\") pod \"distinct-w4-e7929d-kserve-5bc9ff5bf5-kmrwv\" (UID: \"abfd315d-a01a-42dd-9c06-2a80d3f8579a\") " pod="e2e-models-e2e-worker-w4-e7929d/distinct-w4-e7929d-kserve-5bc9ff5bf5-kmrwv"
Oct 30 18:51:05.911354 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:51:05.902672    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-e7929d/distinct-w4-e7929d-kserve-5bc9ff5bf5-kmrwv"]
Oct 30 18:51:05.916356 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:51:05.909060    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-e7929d\"/\"openshift-service-ca.crt\""
Oct 30 18:51:05.916549 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:51:05.909691    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-e7929d\"/\"kube-root-ca.crt\""
Oct 30 18:51:05.916549 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:51:05.909787    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-e7929d\"/\"distinct-w4-e7929d-kserve-self-signed-certs\""
Oct 30 18:51:05.916549 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:51:05.909881    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-e7929d\"/\"default-dockercfg-h7dfc\""
Oct 30 18:51:05.945773 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:51:05.945743    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fgk4z\" (UniqueName: \"kubernetes.io/projected/abfd315d-a01a-42dd-9c06-2a80d3f8579a-kube-api-access-fgk4z\") pod \"distinct-w4-e7929d-kserve-5bc9ff5bf5-kmrwv\" (UID: \"abfd315d-a01a-42dd-9c06-2a80d3f8579a\") " pod="e2e-models-e2e-worker-w4-e7929d/distinct-w4-e7929d-kserve-5bc9ff5bf5-kmrwv"
Oct 30 18:51:05.945982 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:51:05.945808    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/abfd315d-a01a-42dd-9c06-2a80d3f8579a-dshm\") pod \"distinct-w4-e7929d-kserve-5bc9ff5bf5-kmrwv\" (UID: \"abfd315d-a01a-42dd-9c06-2a80d3f8579a\") " pod="e2e-models-e2e-worker-w4-e7929d/distinct-w4-e7929d-kserve-5bc9ff5bf5-kmrwv"
Oct 30 18:51:05.945982 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:51:05.945845    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/abfd315d-a01a-42dd-9c06-2a80d3f8579a-tmp-dir\") pod \"distinct-w4-e7929d-kserve-5bc9ff5bf5-kmrwv\" (UID: \"abfd315d-a01a-42dd-9c06-2a80d3f8579a\") " pod="e2e-models-e2e-worker-w4-e7929d/distinct-w4-e7929d-kserve-5bc9ff5bf5-kmrwv"
Oct 30 18:51:05.945982 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:51:05.945880    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/abfd315d-a01a-42dd-9c06-2a80d3f8579a-tls-certs\") pod \"distinct-w4-e7929d-kserve-5bc9ff5bf5-kmrwv\" (UID: \"abfd315d-a01a-42dd-9c06-2a80d3f8579a\") " pod="e2e-models-e2e-worker-w4-e7929d/distinct-w4-e7929d-kserve-5bc9ff5bf5-kmrwv"
Oct 30 18:51:05.945982 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:51:05.945924    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/abfd315d-a01a-42dd-9c06-2a80d3f8579a-home\") pod \"distinct-w4-e7929d-kserve-5bc9ff5bf5-kmrwv\" (UID: \"abfd315d-a01a-42dd-9c06-2a80d3f8579a\") " pod="e2e-models-e2e-worker-w4-e7929d/distinct-w4-e7929d-kserve-5bc9ff5bf5-kmrwv"
Oct 30 18:51:05.945982 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:51:05.945951    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/abfd315d-a01a-42dd-9c06-2a80d3f8579a-model-cache\") pod \"distinct-w4-e7929d-kserve-5bc9ff5bf5-kmrwv\" (UID: \"abfd315d-a01a-42dd-9c06-2a80d3f8579a\") " pod="e2e-models-e2e-worker-w4-e7929d/distinct-w4-e7929d-kserve-5bc9ff5bf5-kmrwv"
Oct 30 18:51:05.946407 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:51:05.946385    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/abfd315d-a01a-42dd-9c06-2a80d3f8579a-model-cache\") pod \"distinct-w4-e7929d-kserve-5bc9ff5bf5-kmrwv\" (UID: \"abfd315d-a01a-42dd-9c06-2a80d3f8579a\") " pod="e2e-models-e2e-worker-w4-e7929d/distinct-w4-e7929d-kserve-5bc9ff5bf5-kmrwv"
Oct 30 18:51:05.946631 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:51:05.946603    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/abfd315d-a01a-42dd-9c06-2a80d3f8579a-tmp-dir\") pod \"distinct-w4-e7929d-kserve-5bc9ff5bf5-kmrwv\" (UID: \"abfd315d-a01a-42dd-9c06-2a80d3f8579a\") " pod="e2e-models-e2e-worker-w4-e7929d/distinct-w4-e7929d-kserve-5bc9ff5bf5-kmrwv"
Oct 30 18:51:05.946710 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:51:05.946647    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/abfd315d-a01a-42dd-9c06-2a80d3f8579a-home\") pod \"distinct-w4-e7929d-kserve-5bc9ff5bf5-kmrwv\" (UID: \"abfd315d-a01a-42dd-9c06-2a80d3f8579a\") " pod="e2e-models-e2e-worker-w4-e7929d/distinct-w4-e7929d-kserve-5bc9ff5bf5-kmrwv"
Oct 30 18:51:05.949258 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:51:05.949231    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/abfd315d-a01a-42dd-9c06-2a80d3f8579a-tls-certs\") pod \"distinct-w4-e7929d-kserve-5bc9ff5bf5-kmrwv\" (UID: \"abfd315d-a01a-42dd-9c06-2a80d3f8579a\") " pod="e2e-models-e2e-worker-w4-e7929d/distinct-w4-e7929d-kserve-5bc9ff5bf5-kmrwv"
Oct 30 18:51:05.949462 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:51:05.949443    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/abfd315d-a01a-42dd-9c06-2a80d3f8579a-dshm\") pod \"distinct-w4-e7929d-kserve-5bc9ff5bf5-kmrwv\" (UID: \"abfd315d-a01a-42dd-9c06-2a80d3f8579a\") " pod="e2e-models-e2e-worker-w4-e7929d/distinct-w4-e7929d-kserve-5bc9ff5bf5-kmrwv"
Oct 30 18:51:06.008235 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:51:06.008196    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fgk4z\" (UniqueName: \"kubernetes.io/projected/abfd315d-a01a-42dd-9c06-2a80d3f8579a-kube-api-access-fgk4z\") pod \"distinct-w4-e7929d-kserve-5bc9ff5bf5-kmrwv\" (UID: \"abfd315d-a01a-42dd-9c06-2a80d3f8579a\") " pod="e2e-models-e2e-worker-w4-e7929d/distinct-w4-e7929d-kserve-5bc9ff5bf5-kmrwv"
Oct 30 18:51:06.119462 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:51:06.119436    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-e7929d/distinct-w4-e7929d-kserve-5bc9ff5bf5-kmrwv"
Oct 30 18:51:06.508191 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:51:06.508164    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-e7929d/distinct-w4-e7929d-kserve-5bc9ff5bf5-kmrwv"]
Oct 30 18:51:06.509102 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:51:06.509078    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podabfd315d_a01a_42dd_9c06_2a80d3f8579a.slice/crio-b7da3838a1af55f6dd6a7746ba83162bab5472ed3c2c785826529fd9261d72d1 WatchSource:0}: Error finding container b7da3838a1af55f6dd6a7746ba83162bab5472ed3c2c785826529fd9261d72d1: Status 404 returned error can't find the container with id b7da3838a1af55f6dd6a7746ba83162bab5472ed3c2c785826529fd9261d72d1
Oct 30 18:51:06.536009 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:51:06.535986    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-e7929d/distinct-w4-e7929d-kserve-5bc9ff5bf5-kmrwv" event={"ID":"abfd315d-a01a-42dd-9c06-2a80d3f8579a","Type":"ContainerStarted","Data":"b7da3838a1af55f6dd6a7746ba83162bab5472ed3c2c785826529fd9261d72d1"}
Oct 30 18:51:07.543236 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:51:07.543200    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-e7929d/distinct-w4-e7929d-kserve-5bc9ff5bf5-kmrwv" event={"ID":"abfd315d-a01a-42dd-9c06-2a80d3f8579a","Type":"ContainerStarted","Data":"ee4d9f5543c3dc362899c92e8ef587b30633ccffdad1ab23ee1c2dc2ee84eee4"}
Oct 30 18:51:07.664996 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:51:07.664940    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-e7929d/distinct-w4-e7929d-kserve-5bc9ff5bf5-kmrwv" podStartSLOduration=2.664925202 podStartE2EDuration="2.664925202s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:51:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:51:07.662679627 +0000 UTC m=+1031.910085662" watchObservedRunningTime="2026-10-30 18:51:07.664925202 +0000 UTC m=+1031.912331227"
Oct 30 18:51:08.120581 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:51:08.120546    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-e7929d/distinct-w4-e7929d-kserve-5bc9ff5bf5-kmrwv"
Oct 30 18:51:11.303756 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:51:11.303723    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-rjmdz"]
Oct 30 18:51:14.974936 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:51:14.974899    4888 projected.go:266] Couldn't get secret models-as-a-service/agc-cred-0c229ede8440: secret "agc-cred-0c229ede8440" not found
Oct 30 18:51:14.977193 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:51:14.977174    4888 projected.go:196] Error preparing data for projected volume provider-credentials for pod models-as-a-service/payload-processing-external-model-d8bf55b5b-xnpxw: secret "agc-cred-0c229ede8440" not found
Oct 30 18:51:14.977420 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:51:14.977408    4888 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bb0e7476-6297-4d0d-a5c6-a15f7d5c63d2-provider-credentials podName:bb0e7476-6297-4d0d-a5c6-a15f7d5c63d2 nodeName:}" failed. No retries permitted until 2026-10-30 18:51:15.477388689 +0000 UTC m=+1039.724794706 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "provider-credentials" (UniqueName: "kubernetes.io/projected/bb0e7476-6297-4d0d-a5c6-a15f7d5c63d2-provider-credentials") pod "payload-processing-external-model-d8bf55b5b-xnpxw" (UID: "bb0e7476-6297-4d0d-a5c6-a15f7d5c63d2") : secret "agc-cred-0c229ede8440" not found
Oct 30 18:51:15.481843 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:51:15.481807    4888 projected.go:266] Couldn't get secret models-as-a-service/agc-cred-0c229ede8440: secret "agc-cred-0c229ede8440" not found
Oct 30 18:51:15.481843 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:51:15.481836    4888 projected.go:196] Error preparing data for projected volume provider-credentials for pod models-as-a-service/payload-processing-external-model-d8bf55b5b-xnpxw: secret "agc-cred-0c229ede8440" not found
Oct 30 18:51:15.482061 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:51:15.481896    4888 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bb0e7476-6297-4d0d-a5c6-a15f7d5c63d2-provider-credentials podName:bb0e7476-6297-4d0d-a5c6-a15f7d5c63d2 nodeName:}" failed. No retries permitted until 2026-10-30 18:51:16.481882096 +0000 UTC m=+1040.729288112 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "provider-credentials" (UniqueName: "kubernetes.io/projected/bb0e7476-6297-4d0d-a5c6-a15f7d5c63d2-provider-credentials") pod "payload-processing-external-model-d8bf55b5b-xnpxw" (UID: "bb0e7476-6297-4d0d-a5c6-a15f7d5c63d2") : secret "agc-cred-0c229ede8440" not found
Oct 30 18:51:16.119880 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:51:16.119842    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-e7929d/distinct-w4-e7929d-kserve-5bc9ff5bf5-kmrwv"
Oct 30 18:51:16.132206 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:51:16.132181    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-e7929d/distinct-w4-e7929d-kserve-5bc9ff5bf5-kmrwv"
Oct 30 18:51:16.493179 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:51:16.493105    4888 projected.go:266] Couldn't get secret models-as-a-service/agc-cred-0c229ede8440: secret "agc-cred-0c229ede8440" not found
Oct 30 18:51:16.493179 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:51:16.493130    4888 projected.go:196] Error preparing data for projected volume provider-credentials for pod models-as-a-service/payload-processing-external-model-d8bf55b5b-xnpxw: secret "agc-cred-0c229ede8440" not found
Oct 30 18:51:16.493388 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:51:16.493186    4888 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bb0e7476-6297-4d0d-a5c6-a15f7d5c63d2-provider-credentials podName:bb0e7476-6297-4d0d-a5c6-a15f7d5c63d2 nodeName:}" failed. No retries permitted until 2026-10-30 18:51:18.493171716 +0000 UTC m=+1042.740577720 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "provider-credentials" (UniqueName: "kubernetes.io/projected/bb0e7476-6297-4d0d-a5c6-a15f7d5c63d2-provider-credentials") pod "payload-processing-external-model-d8bf55b5b-xnpxw" (UID: "bb0e7476-6297-4d0d-a5c6-a15f7d5c63d2") : secret "agc-cred-0c229ede8440" not found
Oct 30 18:51:16.660971 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:51:16.660945    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-e7929d/distinct-w4-e7929d-kserve-5bc9ff5bf5-kmrwv"
Oct 30 18:51:18.515968 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:51:18.515942    4888 projected.go:266] Couldn't get secret models-as-a-service/agc-cred-0c229ede8440: secret "agc-cred-0c229ede8440" not found
Oct 30 18:51:18.515968 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:51:18.515963    4888 projected.go:196] Error preparing data for projected volume provider-credentials for pod models-as-a-service/payload-processing-external-model-d8bf55b5b-xnpxw: secret "agc-cred-0c229ede8440" not found
Oct 30 18:51:18.516426 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:51:18.516026    4888 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bb0e7476-6297-4d0d-a5c6-a15f7d5c63d2-provider-credentials podName:bb0e7476-6297-4d0d-a5c6-a15f7d5c63d2 nodeName:}" failed. No retries permitted until 2026-10-30 18:51:22.516012265 +0000 UTC m=+1046.763418282 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "provider-credentials" (UniqueName: "kubernetes.io/projected/bb0e7476-6297-4d0d-a5c6-a15f7d5c63d2-provider-credentials") pod "payload-processing-external-model-d8bf55b5b-xnpxw" (UID: "bb0e7476-6297-4d0d-a5c6-a15f7d5c63d2") : secret "agc-cred-0c229ede8440" not found
Oct 30 18:51:22.495471 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:51:22.495431    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["models-as-a-service/payload-processing-external-model-d8bf55b5b-xnpxw"]
Oct 30 18:51:22.556777 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:51:22.556743    4888 projected.go:266] Couldn't get secret models-as-a-service/agc-cred-0c229ede8440: secret "agc-cred-0c229ede8440" not found
Oct 30 18:51:22.556777 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:51:22.556775    4888 projected.go:196] Error preparing data for projected volume provider-credentials for pod models-as-a-service/payload-processing-external-model-d8bf55b5b-xnpxw: secret "agc-cred-0c229ede8440" not found
Oct 30 18:51:22.556974 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:51:22.556852    4888 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bb0e7476-6297-4d0d-a5c6-a15f7d5c63d2-provider-credentials podName:bb0e7476-6297-4d0d-a5c6-a15f7d5c63d2 nodeName:}" failed. No retries permitted until 2026-10-30 18:51:30.556832204 +0000 UTC m=+1054.804238209 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "provider-credentials" (UniqueName: "kubernetes.io/projected/bb0e7476-6297-4d0d-a5c6-a15f7d5c63d2-provider-credentials") pod "payload-processing-external-model-d8bf55b5b-xnpxw" (UID: "bb0e7476-6297-4d0d-a5c6-a15f7d5c63d2") : secret "agc-cred-0c229ede8440" not found
Oct 30 18:51:27.496273 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:51:27.496208    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="models-as-a-service/payload-processing-external-model-d8bf55b5b-xnpxw" podUID="bb0e7476-6297-4d0d-a5c6-a15f7d5c63d2" containerName="payload-processing" containerID="cri-o://d870d23641019d3061f37433dbf085d0e133c5d3562e8c132c08766854cfbea2" gracePeriod=25
Oct 30 18:51:27.722721 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:51:27.722689    4888 generic.go:356] "Generic (PLEG): container finished" podID="bb0e7476-6297-4d0d-a5c6-a15f7d5c63d2" containerID="d870d23641019d3061f37433dbf085d0e133c5d3562e8c132c08766854cfbea2" exitCode=0
Oct 30 18:51:27.722897 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:51:27.722766    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-d8bf55b5b-xnpxw" event={"ID":"bb0e7476-6297-4d0d-a5c6-a15f7d5c63d2","Type":"ContainerDied","Data":"d870d23641019d3061f37433dbf085d0e133c5d3562e8c132c08766854cfbea2"}
Oct 30 18:51:27.754864 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:51:27.754805    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-d8bf55b5b-xnpxw"
Oct 30 18:51:27.917882 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:51:27.917850    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bb0e7476-6297-4d0d-a5c6-a15f7d5c63d2-provider-credentials\" (UniqueName: \"kubernetes.io/projected/bb0e7476-6297-4d0d-a5c6-a15f7d5c63d2-provider-credentials\") pod \"bb0e7476-6297-4d0d-a5c6-a15f7d5c63d2\" (UID: \"bb0e7476-6297-4d0d-a5c6-a15f7d5c63d2\") "
Oct 30 18:51:27.918072 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:51:27.917999    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/bb0e7476-6297-4d0d-a5c6-a15f7d5c63d2-routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/bb0e7476-6297-4d0d-a5c6-a15f7d5c63d2-routing-overlay-volume\") pod \"bb0e7476-6297-4d0d-a5c6-a15f7d5c63d2\" (UID: \"bb0e7476-6297-4d0d-a5c6-a15f7d5c63d2\") "
Oct 30 18:51:27.918072 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:51:27.918039    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/bb0e7476-6297-4d0d-a5c6-a15f7d5c63d2-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bb0e7476-6297-4d0d-a5c6-a15f7d5c63d2-plugins-config-volume\") pod \"bb0e7476-6297-4d0d-a5c6-a15f7d5c63d2\" (UID: \"bb0e7476-6297-4d0d-a5c6-a15f7d5c63d2\") "
Oct 30 18:51:27.918211 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:51:27.918141    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bb0e7476-6297-4d0d-a5c6-a15f7d5c63d2-kube-api-access-xmfsw\" (UniqueName: \"kubernetes.io/projected/bb0e7476-6297-4d0d-a5c6-a15f7d5c63d2-kube-api-access-xmfsw\") pod \"bb0e7476-6297-4d0d-a5c6-a15f7d5c63d2\" (UID: \"bb0e7476-6297-4d0d-a5c6-a15f7d5c63d2\") "
Oct 30 18:51:27.918434 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:51:27.918358    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bb0e7476-6297-4d0d-a5c6-a15f7d5c63d2-routing-overlay-volume" pod "bb0e7476-6297-4d0d-a5c6-a15f7d5c63d2" (UID: "bb0e7476-6297-4d0d-a5c6-a15f7d5c63d2"). InnerVolumeSpecName "routing-overlay-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:51:27.918615 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:51:27.918588    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bb0e7476-6297-4d0d-a5c6-a15f7d5c63d2-plugins-config-volume" pod "bb0e7476-6297-4d0d-a5c6-a15f7d5c63d2" (UID: "bb0e7476-6297-4d0d-a5c6-a15f7d5c63d2"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:51:27.918688 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:51:27.918605    4888 reconciler_common.go:299] "Volume detached for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/bb0e7476-6297-4d0d-a5c6-a15f7d5c63d2-routing-overlay-volume\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:51:27.920460 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:51:27.920435    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bb0e7476-6297-4d0d-a5c6-a15f7d5c63d2-provider-credentials" pod "bb0e7476-6297-4d0d-a5c6-a15f7d5c63d2" (UID: "bb0e7476-6297-4d0d-a5c6-a15f7d5c63d2"). InnerVolumeSpecName "provider-credentials". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:51:27.920584 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:51:27.920562    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bb0e7476-6297-4d0d-a5c6-a15f7d5c63d2-kube-api-access-xmfsw" pod "bb0e7476-6297-4d0d-a5c6-a15f7d5c63d2" (UID: "bb0e7476-6297-4d0d-a5c6-a15f7d5c63d2"). InnerVolumeSpecName "kube-api-access-xmfsw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:51:28.019491 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:51:28.019414    4888 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bb0e7476-6297-4d0d-a5c6-a15f7d5c63d2-plugins-config-volume\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:51:28.019491 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:51:28.019438    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xmfsw\" (UniqueName: \"kubernetes.io/projected/bb0e7476-6297-4d0d-a5c6-a15f7d5c63d2-kube-api-access-xmfsw\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:51:28.019491 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:51:28.019448    4888 reconciler_common.go:299] "Volume detached for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/bb0e7476-6297-4d0d-a5c6-a15f7d5c63d2-provider-credentials\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:51:28.676932 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:51:28.676894    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-hnndm"]
Oct 30 18:51:28.677741 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:51:28.677720    4888 cpu_manager.go:414] "containerMap: removing container" podUID="bb0e7476-6297-4d0d-a5c6-a15f7d5c63d2" containerName="payload-processing"
Oct 30 18:51:28.677841 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:51:28.677747    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bb0e7476-6297-4d0d-a5c6-a15f7d5c63d2" containerName="payload-processing"
Oct 30 18:51:28.677916 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:51:28.677904    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="bb0e7476-6297-4d0d-a5c6-a15f7d5c63d2" containerName="payload-processing"
Oct 30 18:51:28.681936 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:51:28.681917    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-hnndm"
Oct 30 18:51:28.730049 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:51:28.730024    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-d8bf55b5b-xnpxw"
Oct 30 18:51:28.730220 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:51:28.730063    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-d8bf55b5b-xnpxw" event={"ID":"bb0e7476-6297-4d0d-a5c6-a15f7d5c63d2","Type":"ContainerDied","Data":"f31f838fb52e26609c30acc48abaf280a0b3ece6e2913b9e79f087bd67d60ab9"}
Oct 30 18:51:28.730220 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:51:28.730132    4888 scope.go:122] "RemoveContainer" containerID="d870d23641019d3061f37433dbf085d0e133c5d3562e8c132c08766854cfbea2"
Oct 30 18:51:28.735776 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:51:28.735756    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w4-e7929d-openshift-default-dockercfg-v4s8t\""
Oct 30 18:51:28.790578 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:51:28.790553    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-hnndm"]
Oct 30 18:51:28.828493 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:51:28.828460    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6294bd8-9418-4c1a-99c6-cf52ba98edaa-workload-socket\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-hnndm\" (UID: \"d6294bd8-9418-4c1a-99c6-cf52ba98edaa\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-hnndm"
Oct 30 18:51:28.828604 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:51:28.828500    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d6294bd8-9418-4c1a-99c6-cf52ba98edaa-workload-certs\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-hnndm\" (UID: \"d6294bd8-9418-4c1a-99c6-cf52ba98edaa\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-hnndm"
Oct 30 18:51:28.828644 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:51:28.828603    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d6294bd8-9418-4c1a-99c6-cf52ba98edaa-istio-envoy\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-hnndm\" (UID: \"d6294bd8-9418-4c1a-99c6-cf52ba98edaa\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-hnndm"
Oct 30 18:51:28.828644 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:51:28.828636    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6294bd8-9418-4c1a-99c6-cf52ba98edaa-credential-socket\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-hnndm\" (UID: \"d6294bd8-9418-4c1a-99c6-cf52ba98edaa\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-hnndm"
Oct 30 18:51:28.828736 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:51:28.828718    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d6294bd8-9418-4c1a-99c6-cf52ba98edaa-istio-podinfo\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-hnndm\" (UID: \"d6294bd8-9418-4c1a-99c6-cf52ba98edaa\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-hnndm"
Oct 30 18:51:28.828769 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:51:28.828757    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d6294bd8-9418-4c1a-99c6-cf52ba98edaa-istio-token\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-hnndm\" (UID: \"d6294bd8-9418-4c1a-99c6-cf52ba98edaa\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-hnndm"
Oct 30 18:51:28.828803 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:51:28.828781    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d6294bd8-9418-4c1a-99c6-cf52ba98edaa-istio-data\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-hnndm\" (UID: \"d6294bd8-9418-4c1a-99c6-cf52ba98edaa\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-hnndm"
Oct 30 18:51:28.828855 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:51:28.828843    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-52g5c\" (UniqueName: \"kubernetes.io/projected/d6294bd8-9418-4c1a-99c6-cf52ba98edaa-kube-api-access-52g5c\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-hnndm\" (UID: \"d6294bd8-9418-4c1a-99c6-cf52ba98edaa\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-hnndm"
Oct 30 18:51:28.828938 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:51:28.828920    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d6294bd8-9418-4c1a-99c6-cf52ba98edaa-istiod-ca-cert\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-hnndm\" (UID: \"d6294bd8-9418-4c1a-99c6-cf52ba98edaa\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-hnndm"
Oct 30 18:51:28.929704 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:51:28.929633    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d6294bd8-9418-4c1a-99c6-cf52ba98edaa-istio-token\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-hnndm\" (UID: \"d6294bd8-9418-4c1a-99c6-cf52ba98edaa\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-hnndm"
Oct 30 18:51:28.929704 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:51:28.929668    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d6294bd8-9418-4c1a-99c6-cf52ba98edaa-istio-data\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-hnndm\" (UID: \"d6294bd8-9418-4c1a-99c6-cf52ba98edaa\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-hnndm"
Oct 30 18:51:28.929915 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:51:28.929819    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-52g5c\" (UniqueName: \"kubernetes.io/projected/d6294bd8-9418-4c1a-99c6-cf52ba98edaa-kube-api-access-52g5c\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-hnndm\" (UID: \"d6294bd8-9418-4c1a-99c6-cf52ba98edaa\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-hnndm"
Oct 30 18:51:28.929915 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:51:28.929892    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d6294bd8-9418-4c1a-99c6-cf52ba98edaa-istiod-ca-cert\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-hnndm\" (UID: \"d6294bd8-9418-4c1a-99c6-cf52ba98edaa\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-hnndm"
Oct 30 18:51:28.930029 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:51:28.929940    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6294bd8-9418-4c1a-99c6-cf52ba98edaa-workload-socket\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-hnndm\" (UID: \"d6294bd8-9418-4c1a-99c6-cf52ba98edaa\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-hnndm"
Oct 30 18:51:28.930029 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:51:28.929978    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d6294bd8-9418-4c1a-99c6-cf52ba98edaa-workload-certs\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-hnndm\" (UID: \"d6294bd8-9418-4c1a-99c6-cf52ba98edaa\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-hnndm"
Oct 30 18:51:28.930121 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:51:28.930031    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d6294bd8-9418-4c1a-99c6-cf52ba98edaa-istio-data\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-hnndm\" (UID: \"d6294bd8-9418-4c1a-99c6-cf52ba98edaa\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-hnndm"
Oct 30 18:51:28.930121 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:51:28.930064    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d6294bd8-9418-4c1a-99c6-cf52ba98edaa-istio-envoy\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-hnndm\" (UID: \"d6294bd8-9418-4c1a-99c6-cf52ba98edaa\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-hnndm"
Oct 30 18:51:28.930121 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:51:28.930107    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6294bd8-9418-4c1a-99c6-cf52ba98edaa-credential-socket\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-hnndm\" (UID: \"d6294bd8-9418-4c1a-99c6-cf52ba98edaa\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-hnndm"
Oct 30 18:51:28.930301 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:51:28.930144    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d6294bd8-9418-4c1a-99c6-cf52ba98edaa-istio-podinfo\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-hnndm\" (UID: \"d6294bd8-9418-4c1a-99c6-cf52ba98edaa\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-hnndm"
Oct 30 18:51:28.930301 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:51:28.930261    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6294bd8-9418-4c1a-99c6-cf52ba98edaa-workload-socket\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-hnndm\" (UID: \"d6294bd8-9418-4c1a-99c6-cf52ba98edaa\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-hnndm"
Oct 30 18:51:28.930414 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:51:28.930400    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d6294bd8-9418-4c1a-99c6-cf52ba98edaa-workload-certs\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-hnndm\" (UID: \"d6294bd8-9418-4c1a-99c6-cf52ba98edaa\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-hnndm"
Oct 30 18:51:28.930469 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:51:28.930453    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6294bd8-9418-4c1a-99c6-cf52ba98edaa-credential-socket\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-hnndm\" (UID: \"d6294bd8-9418-4c1a-99c6-cf52ba98edaa\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-hnndm"
Oct 30 18:51:28.930661 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:51:28.930637    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d6294bd8-9418-4c1a-99c6-cf52ba98edaa-istiod-ca-cert\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-hnndm\" (UID: \"d6294bd8-9418-4c1a-99c6-cf52ba98edaa\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-hnndm"
Oct 30 18:51:28.932507 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:51:28.932484    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d6294bd8-9418-4c1a-99c6-cf52ba98edaa-istio-envoy\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-hnndm\" (UID: \"d6294bd8-9418-4c1a-99c6-cf52ba98edaa\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-hnndm"
Oct 30 18:51:28.932688 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:51:28.932671    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d6294bd8-9418-4c1a-99c6-cf52ba98edaa-istio-podinfo\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-hnndm\" (UID: \"d6294bd8-9418-4c1a-99c6-cf52ba98edaa\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-hnndm"
Oct 30 18:51:28.938913 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:51:28.938892    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["models-as-a-service/payload-processing-external-model-d8bf55b5b-xnpxw"]
Oct 30 18:51:28.959383 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:51:28.959361    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["models-as-a-service/payload-processing-external-model-d8bf55b5b-xnpxw"]
Oct 30 18:51:28.992186 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:51:28.992148    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d6294bd8-9418-4c1a-99c6-cf52ba98edaa-istio-token\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-hnndm\" (UID: \"d6294bd8-9418-4c1a-99c6-cf52ba98edaa\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-hnndm"
Oct 30 18:51:28.998344 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:51:28.998321    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-52g5c\" (UniqueName: \"kubernetes.io/projected/d6294bd8-9418-4c1a-99c6-cf52ba98edaa-kube-api-access-52g5c\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-hnndm\" (UID: \"d6294bd8-9418-4c1a-99c6-cf52ba98edaa\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-hnndm"
Oct 30 18:51:29.000161 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:51:29.000141    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-hnndm"
Oct 30 18:51:29.436721 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:51:29.436693    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-hnndm"]
Oct 30 18:51:29.438810 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:51:29.438786    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6294bd8_9418_4c1a_99c6_cf52ba98edaa.slice/crio-61a09bcdd207ce8f295a456a5643432a0122b42fef3cd07d04189539dcde3632 WatchSource:0}: Error finding container 61a09bcdd207ce8f295a456a5643432a0122b42fef3cd07d04189539dcde3632: Status 404 returned error can't find the container with id 61a09bcdd207ce8f295a456a5643432a0122b42fef3cd07d04189539dcde3632
Oct 30 18:51:29.441777 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:51:29.441738    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:51:29.441857 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:51:29.441826    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:51:29.441897 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:51:29.441872    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:51:29.740486 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:51:29.740390    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-hnndm" event={"ID":"d6294bd8-9418-4c1a-99c6-cf52ba98edaa","Type":"ContainerStarted","Data":"461c7337f2cbaa04e696b602018356667011c43f993fb2076a7082372dff2f93"}
Oct 30 18:51:29.740486 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:51:29.740451    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-hnndm" event={"ID":"d6294bd8-9418-4c1a-99c6-cf52ba98edaa","Type":"ContainerStarted","Data":"61a09bcdd207ce8f295a456a5643432a0122b42fef3cd07d04189539dcde3632"}
Oct 30 18:51:29.867176 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:51:29.867129    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-hnndm" podStartSLOduration=1.8671157919999999 podStartE2EDuration="1.867115792s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:51:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:51:29.864054182 +0000 UTC m=+1054.111460208" watchObservedRunningTime="2026-10-30 18:51:29.867115792 +0000 UTC m=+1054.114521817"
Oct 30 18:51:29.892032 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:51:29.892003    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="bb0e7476-6297-4d0d-a5c6-a15f7d5c63d2" path="/var/lib/kubelet/pods/bb0e7476-6297-4d0d-a5c6-a15f7d5c63d2/volumes"
Oct 30 18:51:30.000434 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:51:30.000341    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-hnndm"
Oct 30 18:51:30.002231 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:51:30.002199    4888 patch_prober.go:29] interesting pod/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-hnndm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused" start-of-body=
Oct 30 18:51:30.002375 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:51:30.002244    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-hnndm" podUID="d6294bd8-9418-4c1a-99c6-cf52ba98edaa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused"
Oct 30 18:51:31.001835 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:51:31.001375    4888 patch_prober.go:29] interesting pod/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-hnndm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused" start-of-body=
Oct 30 18:51:31.001835 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:51:31.001467    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-hnndm" podUID="d6294bd8-9418-4c1a-99c6-cf52ba98edaa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused"
Oct 30 18:51:32.002807 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:51:32.002770    4888 patch_prober.go:29] interesting pod/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-hnndm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused" start-of-body=
Oct 30 18:51:32.003542 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:51:32.003479    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-hnndm" podUID="d6294bd8-9418-4c1a-99c6-cf52ba98edaa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused"
Oct 30 18:51:32.507475 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:51:32.507433    4888 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-66db998887-mhzc5 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.32:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 18:51:32.507770 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:51:32.507738    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-mhzc5" podUID="bf0f21cf-8198-4fe4-8983-db083c82b34a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.32:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:51:33.005309 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:51:33.004593    4888 patch_prober.go:29] interesting pod/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-hnndm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused" start-of-body=
Oct 30 18:51:33.005309 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:51:33.004665    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-hnndm" podUID="d6294bd8-9418-4c1a-99c6-cf52ba98edaa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused"
Oct 30 18:51:34.000830 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:51:34.000790    4888 patch_prober.go:29] interesting pod/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-hnndm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused" start-of-body=
Oct 30 18:51:34.001014 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:51:34.000855    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-hnndm" podUID="d6294bd8-9418-4c1a-99c6-cf52ba98edaa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused"
Oct 30 18:51:35.001020 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:51:35.000972    4888 patch_prober.go:29] interesting pod/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-hnndm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused" start-of-body=
Oct 30 18:51:35.001429 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:51:35.001039    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-hnndm" podUID="d6294bd8-9418-4c1a-99c6-cf52ba98edaa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused"
Oct 30 18:51:35.622515 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:51:35.622472    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-jbsw6" containerName="extractor"
Oct 30 18:51:36.001458 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:51:36.001368    4888 patch_prober.go:29] interesting pod/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-hnndm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused" start-of-body=
Oct 30 18:51:36.001458 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:51:36.001428    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-hnndm" podUID="d6294bd8-9418-4c1a-99c6-cf52ba98edaa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused"
Oct 30 18:51:37.005027 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:51:37.004998    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-hnndm"
Oct 30 18:51:37.005672 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:51:37.005333    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-hnndm"
Oct 30 18:51:37.006176 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:51:37.006157    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-hnndm"
Oct 30 18:51:41.321300 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:51:41.321258    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-rjmdz"]
Oct 30 18:51:56.246252 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:51:56.246224    4888 scope.go:122] "RemoveContainer" containerID="4f145df9c33b492e96215dc4eee656f62eb551a3713a5f9aefdd2774a2767123"
Oct 30 18:51:56.255985 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:51:56.255966    4888 scope.go:122] "RemoveContainer" containerID="118ed213b2333e41de22c2637eb89e67f7cc74ce5580bd5f03837b5939a0ced9"
Oct 30 18:52:05.627862 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:52:05.627819    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-jbsw6" containerName="extractor"
Oct 30 18:52:09.230770 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:52:09.230734    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-rjmdz"]
Oct 30 18:52:14.689080 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:52:14.689039    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["models-as-a-service/payload-processing-external-model-59c7d6dc54-pvvbg"]
Oct 30 18:52:14.693195 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:52:14.693172    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-59c7d6dc54-pvvbg"
Oct 30 18:52:14.732209 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:52:14.732178    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nzhc8\" (UniqueName: \"kubernetes.io/projected/9482318f-75fe-41a1-8835-23e6fe632ac5-kube-api-access-nzhc8\") pod \"payload-processing-external-model-59c7d6dc54-pvvbg\" (UID: \"9482318f-75fe-41a1-8835-23e6fe632ac5\") " pod="models-as-a-service/payload-processing-external-model-59c7d6dc54-pvvbg"
Oct 30 18:52:14.732373 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:52:14.732229    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/9482318f-75fe-41a1-8835-23e6fe632ac5-routing-overlay-volume\") pod \"payload-processing-external-model-59c7d6dc54-pvvbg\" (UID: \"9482318f-75fe-41a1-8835-23e6fe632ac5\") " pod="models-as-a-service/payload-processing-external-model-59c7d6dc54-pvvbg"
Oct 30 18:52:14.732373 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:52:14.732361    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9482318f-75fe-41a1-8835-23e6fe632ac5-plugins-config-volume\") pod \"payload-processing-external-model-59c7d6dc54-pvvbg\" (UID: \"9482318f-75fe-41a1-8835-23e6fe632ac5\") " pod="models-as-a-service/payload-processing-external-model-59c7d6dc54-pvvbg"
Oct 30 18:52:14.732465 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:52:14.732443    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/9482318f-75fe-41a1-8835-23e6fe632ac5-provider-credentials\") pod \"payload-processing-external-model-59c7d6dc54-pvvbg\" (UID: \"9482318f-75fe-41a1-8835-23e6fe632ac5\") " pod="models-as-a-service/payload-processing-external-model-59c7d6dc54-pvvbg"
Oct 30 18:52:14.798779 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:52:14.798731    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"models-as-a-service\"/\"payload-processing-external-model-plugins\""
Oct 30 18:52:14.798779 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:52:14.798756    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"models-as-a-service\"/\"routing-overlay\""
Oct 30 18:52:14.798779 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:52:14.798732    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"models-as-a-service\"/\"agc-cred-01eb9baaf090\""
Oct 30 18:52:14.799091 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:52:14.798809    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"models-as-a-service\"/\"payload-processing-external-model-dockercfg-sqhtd\""
Oct 30 18:52:14.799091 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:52:14.799045    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"models-as-a-service\"/\"kube-root-ca.crt\""
Oct 30 18:52:14.799202 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:52:14.799107    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"models-as-a-service\"/\"openshift-service-ca.crt\""
Oct 30 18:52:14.802028 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:52:14.802001    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["models-as-a-service/payload-processing-external-model-59c7d6dc54-pvvbg"]
Oct 30 18:52:14.833327 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:52:14.833275    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/9482318f-75fe-41a1-8835-23e6fe632ac5-routing-overlay-volume\") pod \"payload-processing-external-model-59c7d6dc54-pvvbg\" (UID: \"9482318f-75fe-41a1-8835-23e6fe632ac5\") " pod="models-as-a-service/payload-processing-external-model-59c7d6dc54-pvvbg"
Oct 30 18:52:14.833497 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:52:14.833392    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9482318f-75fe-41a1-8835-23e6fe632ac5-plugins-config-volume\") pod \"payload-processing-external-model-59c7d6dc54-pvvbg\" (UID: \"9482318f-75fe-41a1-8835-23e6fe632ac5\") " pod="models-as-a-service/payload-processing-external-model-59c7d6dc54-pvvbg"
Oct 30 18:52:14.833497 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:52:14.833444    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/9482318f-75fe-41a1-8835-23e6fe632ac5-provider-credentials\") pod \"payload-processing-external-model-59c7d6dc54-pvvbg\" (UID: \"9482318f-75fe-41a1-8835-23e6fe632ac5\") " pod="models-as-a-service/payload-processing-external-model-59c7d6dc54-pvvbg"
Oct 30 18:52:14.833627 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:52:14.833579    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nzhc8\" (UniqueName: \"kubernetes.io/projected/9482318f-75fe-41a1-8835-23e6fe632ac5-kube-api-access-nzhc8\") pod \"payload-processing-external-model-59c7d6dc54-pvvbg\" (UID: \"9482318f-75fe-41a1-8835-23e6fe632ac5\") " pod="models-as-a-service/payload-processing-external-model-59c7d6dc54-pvvbg"
Oct 30 18:52:14.834059 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:52:14.834034    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/9482318f-75fe-41a1-8835-23e6fe632ac5-routing-overlay-volume\") pod \"payload-processing-external-model-59c7d6dc54-pvvbg\" (UID: \"9482318f-75fe-41a1-8835-23e6fe632ac5\") " pod="models-as-a-service/payload-processing-external-model-59c7d6dc54-pvvbg"
Oct 30 18:52:14.834171 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:52:14.834153    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9482318f-75fe-41a1-8835-23e6fe632ac5-plugins-config-volume\") pod \"payload-processing-external-model-59c7d6dc54-pvvbg\" (UID: \"9482318f-75fe-41a1-8835-23e6fe632ac5\") " pod="models-as-a-service/payload-processing-external-model-59c7d6dc54-pvvbg"
Oct 30 18:52:14.836363 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:52:14.836339    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/9482318f-75fe-41a1-8835-23e6fe632ac5-provider-credentials\") pod \"payload-processing-external-model-59c7d6dc54-pvvbg\" (UID: \"9482318f-75fe-41a1-8835-23e6fe632ac5\") " pod="models-as-a-service/payload-processing-external-model-59c7d6dc54-pvvbg"
Oct 30 18:52:14.881703 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:52:14.881674    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-e7929d/distinct-2-w4-e7929d-kserve-5b479cc87b-htv9p"]
Oct 30 18:52:14.886272 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:52:14.886242    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-e7929d/distinct-2-w4-e7929d-kserve-5b479cc87b-htv9p"
Oct 30 18:52:14.899250 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:52:14.899223    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nzhc8\" (UniqueName: \"kubernetes.io/projected/9482318f-75fe-41a1-8835-23e6fe632ac5-kube-api-access-nzhc8\") pod \"payload-processing-external-model-59c7d6dc54-pvvbg\" (UID: \"9482318f-75fe-41a1-8835-23e6fe632ac5\") " pod="models-as-a-service/payload-processing-external-model-59c7d6dc54-pvvbg"
Oct 30 18:52:14.934388 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:52:14.934355    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f6df655d-86e8-44ff-ace8-27440f56294f-tls-certs\") pod \"distinct-2-w4-e7929d-kserve-5b479cc87b-htv9p\" (UID: \"f6df655d-86e8-44ff-ace8-27440f56294f\") " pod="e2e-models-e2e-worker-w4-e7929d/distinct-2-w4-e7929d-kserve-5b479cc87b-htv9p"
Oct 30 18:52:14.934712 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:52:14.934681    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f6df655d-86e8-44ff-ace8-27440f56294f-tmp-dir\") pod \"distinct-2-w4-e7929d-kserve-5b479cc87b-htv9p\" (UID: \"f6df655d-86e8-44ff-ace8-27440f56294f\") " pod="e2e-models-e2e-worker-w4-e7929d/distinct-2-w4-e7929d-kserve-5b479cc87b-htv9p"
Oct 30 18:52:14.934832 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:52:14.934758    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f6df655d-86e8-44ff-ace8-27440f56294f-dshm\") pod \"distinct-2-w4-e7929d-kserve-5b479cc87b-htv9p\" (UID: \"f6df655d-86e8-44ff-ace8-27440f56294f\") " pod="e2e-models-e2e-worker-w4-e7929d/distinct-2-w4-e7929d-kserve-5b479cc87b-htv9p"
Oct 30 18:52:14.934985 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:52:14.934963    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fhfmt\" (UniqueName: \"kubernetes.io/projected/f6df655d-86e8-44ff-ace8-27440f56294f-kube-api-access-fhfmt\") pod \"distinct-2-w4-e7929d-kserve-5b479cc87b-htv9p\" (UID: \"f6df655d-86e8-44ff-ace8-27440f56294f\") " pod="e2e-models-e2e-worker-w4-e7929d/distinct-2-w4-e7929d-kserve-5b479cc87b-htv9p"
Oct 30 18:52:14.935057 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:52:14.935004    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f6df655d-86e8-44ff-ace8-27440f56294f-home\") pod \"distinct-2-w4-e7929d-kserve-5b479cc87b-htv9p\" (UID: \"f6df655d-86e8-44ff-ace8-27440f56294f\") " pod="e2e-models-e2e-worker-w4-e7929d/distinct-2-w4-e7929d-kserve-5b479cc87b-htv9p"
Oct 30 18:52:14.935118 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:52:14.935101    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f6df655d-86e8-44ff-ace8-27440f56294f-model-cache\") pod \"distinct-2-w4-e7929d-kserve-5b479cc87b-htv9p\" (UID: \"f6df655d-86e8-44ff-ace8-27440f56294f\") " pod="e2e-models-e2e-worker-w4-e7929d/distinct-2-w4-e7929d-kserve-5b479cc87b-htv9p"
Oct 30 18:52:14.941360 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:52:14.941276    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-e7929d\"/\"distinct-2-w4-e7929d-kserve-self-signed-certs\""
Oct 30 18:52:14.945728 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:52:14.945705    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-e7929d/distinct-2-w4-e7929d-kserve-5b479cc87b-htv9p"]
Oct 30 18:52:15.010742 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:52:15.010711    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-59c7d6dc54-pvvbg"
Oct 30 18:52:15.036517 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:52:15.036477    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f6df655d-86e8-44ff-ace8-27440f56294f-model-cache\") pod \"distinct-2-w4-e7929d-kserve-5b479cc87b-htv9p\" (UID: \"f6df655d-86e8-44ff-ace8-27440f56294f\") " pod="e2e-models-e2e-worker-w4-e7929d/distinct-2-w4-e7929d-kserve-5b479cc87b-htv9p"
Oct 30 18:52:15.036688 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:52:15.036559    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f6df655d-86e8-44ff-ace8-27440f56294f-tls-certs\") pod \"distinct-2-w4-e7929d-kserve-5b479cc87b-htv9p\" (UID: \"f6df655d-86e8-44ff-ace8-27440f56294f\") " pod="e2e-models-e2e-worker-w4-e7929d/distinct-2-w4-e7929d-kserve-5b479cc87b-htv9p"
Oct 30 18:52:15.036688 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:52:15.036636    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f6df655d-86e8-44ff-ace8-27440f56294f-tmp-dir\") pod \"distinct-2-w4-e7929d-kserve-5b479cc87b-htv9p\" (UID: \"f6df655d-86e8-44ff-ace8-27440f56294f\") " pod="e2e-models-e2e-worker-w4-e7929d/distinct-2-w4-e7929d-kserve-5b479cc87b-htv9p"
Oct 30 18:52:15.036688 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:52:15.036666    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f6df655d-86e8-44ff-ace8-27440f56294f-dshm\") pod \"distinct-2-w4-e7929d-kserve-5b479cc87b-htv9p\" (UID: \"f6df655d-86e8-44ff-ace8-27440f56294f\") " pod="e2e-models-e2e-worker-w4-e7929d/distinct-2-w4-e7929d-kserve-5b479cc87b-htv9p"
Oct 30 18:52:15.036914 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:52:15.036790    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fhfmt\" (UniqueName: \"kubernetes.io/projected/f6df655d-86e8-44ff-ace8-27440f56294f-kube-api-access-fhfmt\") pod \"distinct-2-w4-e7929d-kserve-5b479cc87b-htv9p\" (UID: \"f6df655d-86e8-44ff-ace8-27440f56294f\") " pod="e2e-models-e2e-worker-w4-e7929d/distinct-2-w4-e7929d-kserve-5b479cc87b-htv9p"
Oct 30 18:52:15.036914 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:52:15.036824    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f6df655d-86e8-44ff-ace8-27440f56294f-home\") pod \"distinct-2-w4-e7929d-kserve-5b479cc87b-htv9p\" (UID: \"f6df655d-86e8-44ff-ace8-27440f56294f\") " pod="e2e-models-e2e-worker-w4-e7929d/distinct-2-w4-e7929d-kserve-5b479cc87b-htv9p"
Oct 30 18:52:15.037540 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:52:15.037514    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f6df655d-86e8-44ff-ace8-27440f56294f-home\") pod \"distinct-2-w4-e7929d-kserve-5b479cc87b-htv9p\" (UID: \"f6df655d-86e8-44ff-ace8-27440f56294f\") " pod="e2e-models-e2e-worker-w4-e7929d/distinct-2-w4-e7929d-kserve-5b479cc87b-htv9p"
Oct 30 18:52:15.038022 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:52:15.037947    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f6df655d-86e8-44ff-ace8-27440f56294f-model-cache\") pod \"distinct-2-w4-e7929d-kserve-5b479cc87b-htv9p\" (UID: \"f6df655d-86e8-44ff-ace8-27440f56294f\") " pod="e2e-models-e2e-worker-w4-e7929d/distinct-2-w4-e7929d-kserve-5b479cc87b-htv9p"
Oct 30 18:52:15.038404 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:52:15.038379    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f6df655d-86e8-44ff-ace8-27440f56294f-tmp-dir\") pod \"distinct-2-w4-e7929d-kserve-5b479cc87b-htv9p\" (UID: \"f6df655d-86e8-44ff-ace8-27440f56294f\") " pod="e2e-models-e2e-worker-w4-e7929d/distinct-2-w4-e7929d-kserve-5b479cc87b-htv9p"
Oct 30 18:52:15.040109 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:52:15.040076    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f6df655d-86e8-44ff-ace8-27440f56294f-dshm\") pod \"distinct-2-w4-e7929d-kserve-5b479cc87b-htv9p\" (UID: \"f6df655d-86e8-44ff-ace8-27440f56294f\") " pod="e2e-models-e2e-worker-w4-e7929d/distinct-2-w4-e7929d-kserve-5b479cc87b-htv9p"
Oct 30 18:52:15.040260 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:52:15.040240    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f6df655d-86e8-44ff-ace8-27440f56294f-tls-certs\") pod \"distinct-2-w4-e7929d-kserve-5b479cc87b-htv9p\" (UID: \"f6df655d-86e8-44ff-ace8-27440f56294f\") " pod="e2e-models-e2e-worker-w4-e7929d/distinct-2-w4-e7929d-kserve-5b479cc87b-htv9p"
Oct 30 18:52:15.112573 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:52:15.112528    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fhfmt\" (UniqueName: \"kubernetes.io/projected/f6df655d-86e8-44ff-ace8-27440f56294f-kube-api-access-fhfmt\") pod \"distinct-2-w4-e7929d-kserve-5b479cc87b-htv9p\" (UID: \"f6df655d-86e8-44ff-ace8-27440f56294f\") " pod="e2e-models-e2e-worker-w4-e7929d/distinct-2-w4-e7929d-kserve-5b479cc87b-htv9p"
Oct 30 18:52:15.199156 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:52:15.199078    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-e7929d/distinct-2-w4-e7929d-kserve-5b479cc87b-htv9p"
Oct 30 18:52:15.407494 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:52:15.407457    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9482318f_75fe_41a1_8835_23e6fe632ac5.slice/crio-088eb68896b96a32a2aa5e02dd31e2ff784ca121b9b7ab08e5e3c6ca8f3071b4 WatchSource:0}: Error finding container 088eb68896b96a32a2aa5e02dd31e2ff784ca121b9b7ab08e5e3c6ca8f3071b4: Status 404 returned error can't find the container with id 088eb68896b96a32a2aa5e02dd31e2ff784ca121b9b7ab08e5e3c6ca8f3071b4
Oct 30 18:52:15.407885 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:52:15.407861    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["models-as-a-service/payload-processing-external-model-59c7d6dc54-pvvbg"]
Oct 30 18:52:15.588502 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:52:15.588480    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-e7929d/distinct-2-w4-e7929d-kserve-5b479cc87b-htv9p"]
Oct 30 18:52:15.589021 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:52:15.588995    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf6df655d_86e8_44ff_ace8_27440f56294f.slice/crio-392f86fa58d68f94c737ef20cb468cdaf5ad3291120b92c80b32924e6dd9c2a1 WatchSource:0}: Error finding container 392f86fa58d68f94c737ef20cb468cdaf5ad3291120b92c80b32924e6dd9c2a1: Status 404 returned error can't find the container with id 392f86fa58d68f94c737ef20cb468cdaf5ad3291120b92c80b32924e6dd9c2a1
Oct 30 18:52:16.153930 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:52:16.153894    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-59c7d6dc54-pvvbg" event={"ID":"9482318f-75fe-41a1-8835-23e6fe632ac5","Type":"ContainerStarted","Data":"7fd6c9642e4902cced43ea59de35ee2a5ff5001f22c428a2d80a3258b3b161e7"}
Oct 30 18:52:16.154482 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:52:16.153959    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-59c7d6dc54-pvvbg" event={"ID":"9482318f-75fe-41a1-8835-23e6fe632ac5","Type":"ContainerStarted","Data":"088eb68896b96a32a2aa5e02dd31e2ff784ca121b9b7ab08e5e3c6ca8f3071b4"}
Oct 30 18:52:16.154482 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:52:16.154109    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="models-as-a-service/payload-processing-external-model-59c7d6dc54-pvvbg"
Oct 30 18:52:16.155733 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:52:16.155707    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-e7929d/distinct-2-w4-e7929d-kserve-5b479cc87b-htv9p" event={"ID":"f6df655d-86e8-44ff-ace8-27440f56294f","Type":"ContainerStarted","Data":"2b97fd93ad52f3a83c6c5e51a3ea4b48f8890b63980c41432818aebe19fb8bb4"}
Oct 30 18:52:16.155864 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:52:16.155744    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-e7929d/distinct-2-w4-e7929d-kserve-5b479cc87b-htv9p" event={"ID":"f6df655d-86e8-44ff-ace8-27440f56294f","Type":"ContainerStarted","Data":"392f86fa58d68f94c737ef20cb468cdaf5ad3291120b92c80b32924e6dd9c2a1"}
Oct 30 18:52:16.200149 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:52:16.200123    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-e7929d/distinct-2-w4-e7929d-kserve-5b479cc87b-htv9p"
Oct 30 18:52:16.276055 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:52:16.276008    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="models-as-a-service/payload-processing-external-model-59c7d6dc54-pvvbg" podStartSLOduration=2.275996114 podStartE2EDuration="2.275996114s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:52:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:52:16.274603737 +0000 UTC m=+1100.522009762" watchObservedRunningTime="2026-10-30 18:52:16.275996114 +0000 UTC m=+1100.523402139"
Oct 30 18:52:16.395253 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:52:16.395206    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-e7929d/distinct-2-w4-e7929d-kserve-5b479cc87b-htv9p" podStartSLOduration=2.395193754 podStartE2EDuration="2.395193754s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:52:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:52:16.393573419 +0000 UTC m=+1100.640979446" watchObservedRunningTime="2026-10-30 18:52:16.395193754 +0000 UTC m=+1100.642599840"
Oct 30 18:52:25.199430 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:52:25.199383    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-e7929d/distinct-2-w4-e7929d-kserve-5b479cc87b-htv9p"
Oct 30 18:52:25.213234 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:52:25.213205    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-e7929d/distinct-2-w4-e7929d-kserve-5b479cc87b-htv9p"
Oct 30 18:52:25.229967 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:52:25.229940    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-e7929d/distinct-2-w4-e7929d-kserve-5b479cc87b-htv9p"
Oct 30 18:52:27.165562 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:52:27.165525    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-59c7d6dc54-pvvbg" podUID="9482318f-75fe-41a1-8835-23e6fe632ac5" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 18:52:27.751113 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:52:27.751040    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["models-as-a-service/payload-processing-external-model-59c7d6dc54-pvvbg"]
Oct 30 18:52:27.752768 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:52:27.752742    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="models-as-a-service/payload-processing-external-model-59c7d6dc54-pvvbg" podUID="9482318f-75fe-41a1-8835-23e6fe632ac5" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 18:52:28.605708 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:52:28.605661    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w3-3b6013/distinct-w3-3b6013-kserve-857f55d885-8fcrq"]
Oct 30 18:52:28.611520 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:52:28.611498    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-3b6013/distinct-w3-3b6013-kserve-857f55d885-8fcrq"
Oct 30 18:52:28.691308 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:52:28.688606    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-3b6013\"/\"distinct-w3-3b6013-kserve-self-signed-certs\""
Oct 30 18:52:28.842312 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:52:28.807981    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/51927877-4d99-4797-82f7-1684f6ecda6b-home\") pod \"distinct-w3-3b6013-kserve-857f55d885-8fcrq\" (UID: \"51927877-4d99-4797-82f7-1684f6ecda6b\") " pod="e2e-models-e2e-worker-w3-3b6013/distinct-w3-3b6013-kserve-857f55d885-8fcrq"
Oct 30 18:52:28.842312 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:52:28.808043    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/51927877-4d99-4797-82f7-1684f6ecda6b-tmp-dir\") pod \"distinct-w3-3b6013-kserve-857f55d885-8fcrq\" (UID: \"51927877-4d99-4797-82f7-1684f6ecda6b\") " pod="e2e-models-e2e-worker-w3-3b6013/distinct-w3-3b6013-kserve-857f55d885-8fcrq"
Oct 30 18:52:28.842312 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:52:28.808166    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/51927877-4d99-4797-82f7-1684f6ecda6b-model-cache\") pod \"distinct-w3-3b6013-kserve-857f55d885-8fcrq\" (UID: \"51927877-4d99-4797-82f7-1684f6ecda6b\") " pod="e2e-models-e2e-worker-w3-3b6013/distinct-w3-3b6013-kserve-857f55d885-8fcrq"
Oct 30 18:52:28.842312 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:52:28.808195    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6nd5j\" (UniqueName: \"kubernetes.io/projected/51927877-4d99-4797-82f7-1684f6ecda6b-kube-api-access-6nd5j\") pod \"distinct-w3-3b6013-kserve-857f55d885-8fcrq\" (UID: \"51927877-4d99-4797-82f7-1684f6ecda6b\") " pod="e2e-models-e2e-worker-w3-3b6013/distinct-w3-3b6013-kserve-857f55d885-8fcrq"
Oct 30 18:52:28.842312 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:52:28.808239    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/51927877-4d99-4797-82f7-1684f6ecda6b-tls-certs\") pod \"distinct-w3-3b6013-kserve-857f55d885-8fcrq\" (UID: \"51927877-4d99-4797-82f7-1684f6ecda6b\") " pod="e2e-models-e2e-worker-w3-3b6013/distinct-w3-3b6013-kserve-857f55d885-8fcrq"
Oct 30 18:52:28.842312 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:52:28.808303    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/51927877-4d99-4797-82f7-1684f6ecda6b-dshm\") pod \"distinct-w3-3b6013-kserve-857f55d885-8fcrq\" (UID: \"51927877-4d99-4797-82f7-1684f6ecda6b\") " pod="e2e-models-e2e-worker-w3-3b6013/distinct-w3-3b6013-kserve-857f55d885-8fcrq"
Oct 30 18:52:28.866359 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:52:28.842886    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-3b6013/distinct-w3-3b6013-kserve-857f55d885-8fcrq"]
Oct 30 18:52:28.915311 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:52:28.909426    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/51927877-4d99-4797-82f7-1684f6ecda6b-tmp-dir\") pod \"distinct-w3-3b6013-kserve-857f55d885-8fcrq\" (UID: \"51927877-4d99-4797-82f7-1684f6ecda6b\") " pod="e2e-models-e2e-worker-w3-3b6013/distinct-w3-3b6013-kserve-857f55d885-8fcrq"
Oct 30 18:52:28.915311 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:52:28.909564    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/51927877-4d99-4797-82f7-1684f6ecda6b-model-cache\") pod \"distinct-w3-3b6013-kserve-857f55d885-8fcrq\" (UID: \"51927877-4d99-4797-82f7-1684f6ecda6b\") " pod="e2e-models-e2e-worker-w3-3b6013/distinct-w3-3b6013-kserve-857f55d885-8fcrq"
Oct 30 18:52:28.915311 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:52:28.909593    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6nd5j\" (UniqueName: \"kubernetes.io/projected/51927877-4d99-4797-82f7-1684f6ecda6b-kube-api-access-6nd5j\") pod \"distinct-w3-3b6013-kserve-857f55d885-8fcrq\" (UID: \"51927877-4d99-4797-82f7-1684f6ecda6b\") " pod="e2e-models-e2e-worker-w3-3b6013/distinct-w3-3b6013-kserve-857f55d885-8fcrq"
Oct 30 18:52:28.915311 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:52:28.909635    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/51927877-4d99-4797-82f7-1684f6ecda6b-tls-certs\") pod \"distinct-w3-3b6013-kserve-857f55d885-8fcrq\" (UID: \"51927877-4d99-4797-82f7-1684f6ecda6b\") " pod="e2e-models-e2e-worker-w3-3b6013/distinct-w3-3b6013-kserve-857f55d885-8fcrq"
Oct 30 18:52:28.915311 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:52:28.909676    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/51927877-4d99-4797-82f7-1684f6ecda6b-dshm\") pod \"distinct-w3-3b6013-kserve-857f55d885-8fcrq\" (UID: \"51927877-4d99-4797-82f7-1684f6ecda6b\") " pod="e2e-models-e2e-worker-w3-3b6013/distinct-w3-3b6013-kserve-857f55d885-8fcrq"
Oct 30 18:52:28.915311 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:52:28.909762    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/51927877-4d99-4797-82f7-1684f6ecda6b-home\") pod \"distinct-w3-3b6013-kserve-857f55d885-8fcrq\" (UID: \"51927877-4d99-4797-82f7-1684f6ecda6b\") " pod="e2e-models-e2e-worker-w3-3b6013/distinct-w3-3b6013-kserve-857f55d885-8fcrq"
Oct 30 18:52:28.915311 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:52:28.910219    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/51927877-4d99-4797-82f7-1684f6ecda6b-home\") pod \"distinct-w3-3b6013-kserve-857f55d885-8fcrq\" (UID: \"51927877-4d99-4797-82f7-1684f6ecda6b\") " pod="e2e-models-e2e-worker-w3-3b6013/distinct-w3-3b6013-kserve-857f55d885-8fcrq"
Oct 30 18:52:28.915311 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:52:28.910847    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/51927877-4d99-4797-82f7-1684f6ecda6b-tmp-dir\") pod \"distinct-w3-3b6013-kserve-857f55d885-8fcrq\" (UID: \"51927877-4d99-4797-82f7-1684f6ecda6b\") " pod="e2e-models-e2e-worker-w3-3b6013/distinct-w3-3b6013-kserve-857f55d885-8fcrq"
Oct 30 18:52:28.915311 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:52:28.911088    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/51927877-4d99-4797-82f7-1684f6ecda6b-model-cache\") pod \"distinct-w3-3b6013-kserve-857f55d885-8fcrq\" (UID: \"51927877-4d99-4797-82f7-1684f6ecda6b\") " pod="e2e-models-e2e-worker-w3-3b6013/distinct-w3-3b6013-kserve-857f55d885-8fcrq"
Oct 30 18:52:29.004350 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:52:28.994351    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6nd5j\" (UniqueName: \"kubernetes.io/projected/51927877-4d99-4797-82f7-1684f6ecda6b-kube-api-access-6nd5j\") pod \"distinct-w3-3b6013-kserve-857f55d885-8fcrq\" (UID: \"51927877-4d99-4797-82f7-1684f6ecda6b\") " pod="e2e-models-e2e-worker-w3-3b6013/distinct-w3-3b6013-kserve-857f55d885-8fcrq"
Oct 30 18:52:29.004350 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:52:28.994569    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/51927877-4d99-4797-82f7-1684f6ecda6b-dshm\") pod \"distinct-w3-3b6013-kserve-857f55d885-8fcrq\" (UID: \"51927877-4d99-4797-82f7-1684f6ecda6b\") " pod="e2e-models-e2e-worker-w3-3b6013/distinct-w3-3b6013-kserve-857f55d885-8fcrq"
Oct 30 18:52:29.022321 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:52:29.018228    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/51927877-4d99-4797-82f7-1684f6ecda6b-tls-certs\") pod \"distinct-w3-3b6013-kserve-857f55d885-8fcrq\" (UID: \"51927877-4d99-4797-82f7-1684f6ecda6b\") " pod="e2e-models-e2e-worker-w3-3b6013/distinct-w3-3b6013-kserve-857f55d885-8fcrq"
Oct 30 18:52:29.189373 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:52:29.184515    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-3b6013/distinct-w3-3b6013-kserve-857f55d885-8fcrq"
Oct 30 18:52:30.171716 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:52:30.171678    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-3b6013/distinct-w3-3b6013-kserve-857f55d885-8fcrq"]
Oct 30 18:52:30.172426 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:52:30.172399    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod51927877_4d99_4797_82f7_1684f6ecda6b.slice/crio-070a4da7eb6bd941325e341b3d2f5c0fef8a171d095f7c8a649c10d6fcf96c59 WatchSource:0}: Error finding container 070a4da7eb6bd941325e341b3d2f5c0fef8a171d095f7c8a649c10d6fcf96c59: Status 404 returned error can't find the container with id 070a4da7eb6bd941325e341b3d2f5c0fef8a171d095f7c8a649c10d6fcf96c59
Oct 30 18:52:30.318216 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:52:30.318188    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-3b6013/distinct-w3-3b6013-kserve-857f55d885-8fcrq" event={"ID":"51927877-4d99-4797-82f7-1684f6ecda6b","Type":"ContainerStarted","Data":"03797d706f04895778b42aca80253000864743c3106c238d298f3f547c4b2755"}
Oct 30 18:52:30.318391 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:52:30.318227    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-3b6013/distinct-w3-3b6013-kserve-857f55d885-8fcrq" event={"ID":"51927877-4d99-4797-82f7-1684f6ecda6b","Type":"ContainerStarted","Data":"070a4da7eb6bd941325e341b3d2f5c0fef8a171d095f7c8a649c10d6fcf96c59"}
Oct 30 18:52:30.438407 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:52:30.438263    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w3-3b6013/distinct-w3-3b6013-kserve-857f55d885-8fcrq" podStartSLOduration=2.438238326 podStartE2EDuration="2.438238326s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:52:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:52:30.43641897 +0000 UTC m=+1114.683824995" watchObservedRunningTime="2026-10-30 18:52:30.438238326 +0000 UTC m=+1114.685644351"
Oct 30 18:52:31.185240 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:52:31.185210    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-3b6013/distinct-w3-3b6013-kserve-857f55d885-8fcrq"
Oct 30 18:52:32.752456 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:52:32.752401    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="models-as-a-service/payload-processing-external-model-59c7d6dc54-pvvbg" podUID="9482318f-75fe-41a1-8835-23e6fe632ac5" containerName="payload-processing" containerID="cri-o://7fd6c9642e4902cced43ea59de35ee2a5ff5001f22c428a2d80a3258b3b161e7" gracePeriod=25
Oct 30 18:52:33.021644 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:52:33.021583    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-59c7d6dc54-pvvbg"
Oct 30 18:52:33.086976 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:52:33.086954    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9482318f-75fe-41a1-8835-23e6fe632ac5-provider-credentials\" (UniqueName: \"kubernetes.io/projected/9482318f-75fe-41a1-8835-23e6fe632ac5-provider-credentials\") pod \"9482318f-75fe-41a1-8835-23e6fe632ac5\" (UID: \"9482318f-75fe-41a1-8835-23e6fe632ac5\") "
Oct 30 18:52:33.087110 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:52:33.087057    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9482318f-75fe-41a1-8835-23e6fe632ac5-routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/9482318f-75fe-41a1-8835-23e6fe632ac5-routing-overlay-volume\") pod \"9482318f-75fe-41a1-8835-23e6fe632ac5\" (UID: \"9482318f-75fe-41a1-8835-23e6fe632ac5\") "
Oct 30 18:52:33.087177 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:52:33.087161    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9482318f-75fe-41a1-8835-23e6fe632ac5-kube-api-access-nzhc8\" (UniqueName: \"kubernetes.io/projected/9482318f-75fe-41a1-8835-23e6fe632ac5-kube-api-access-nzhc8\") pod \"9482318f-75fe-41a1-8835-23e6fe632ac5\" (UID: \"9482318f-75fe-41a1-8835-23e6fe632ac5\") "
Oct 30 18:52:33.087254 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:52:33.087238    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9482318f-75fe-41a1-8835-23e6fe632ac5-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9482318f-75fe-41a1-8835-23e6fe632ac5-plugins-config-volume\") pod \"9482318f-75fe-41a1-8835-23e6fe632ac5\" (UID: \"9482318f-75fe-41a1-8835-23e6fe632ac5\") "
Oct 30 18:52:33.087487 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:52:33.087457    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9482318f-75fe-41a1-8835-23e6fe632ac5-routing-overlay-volume" pod "9482318f-75fe-41a1-8835-23e6fe632ac5" (UID: "9482318f-75fe-41a1-8835-23e6fe632ac5"). InnerVolumeSpecName "routing-overlay-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:52:33.087719 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:52:33.087691    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9482318f-75fe-41a1-8835-23e6fe632ac5-plugins-config-volume" pod "9482318f-75fe-41a1-8835-23e6fe632ac5" (UID: "9482318f-75fe-41a1-8835-23e6fe632ac5"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:52:33.087801 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:52:33.087724    4888 reconciler_common.go:299] "Volume detached for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/9482318f-75fe-41a1-8835-23e6fe632ac5-routing-overlay-volume\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:52:33.089412 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:52:33.089383    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9482318f-75fe-41a1-8835-23e6fe632ac5-kube-api-access-nzhc8" pod "9482318f-75fe-41a1-8835-23e6fe632ac5" (UID: "9482318f-75fe-41a1-8835-23e6fe632ac5"). InnerVolumeSpecName "kube-api-access-nzhc8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:52:33.089412 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:52:33.089387    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9482318f-75fe-41a1-8835-23e6fe632ac5-provider-credentials" pod "9482318f-75fe-41a1-8835-23e6fe632ac5" (UID: "9482318f-75fe-41a1-8835-23e6fe632ac5"). InnerVolumeSpecName "provider-credentials". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:52:33.189419 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:52:33.189382    4888 reconciler_common.go:299] "Volume detached for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/9482318f-75fe-41a1-8835-23e6fe632ac5-provider-credentials\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:52:33.189419 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:52:33.189420    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nzhc8\" (UniqueName: \"kubernetes.io/projected/9482318f-75fe-41a1-8835-23e6fe632ac5-kube-api-access-nzhc8\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:52:33.189661 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:52:33.189437    4888 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9482318f-75fe-41a1-8835-23e6fe632ac5-plugins-config-volume\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:52:33.341565 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:52:33.341534    4888 generic.go:356] "Generic (PLEG): container finished" podID="9482318f-75fe-41a1-8835-23e6fe632ac5" containerID="7fd6c9642e4902cced43ea59de35ee2a5ff5001f22c428a2d80a3258b3b161e7" exitCode=0
Oct 30 18:52:33.341731 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:52:33.341581    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-59c7d6dc54-pvvbg"
Oct 30 18:52:33.341731 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:52:33.341627    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-59c7d6dc54-pvvbg" event={"ID":"9482318f-75fe-41a1-8835-23e6fe632ac5","Type":"ContainerDied","Data":"7fd6c9642e4902cced43ea59de35ee2a5ff5001f22c428a2d80a3258b3b161e7"}
Oct 30 18:52:33.341731 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:52:33.341697    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-59c7d6dc54-pvvbg" event={"ID":"9482318f-75fe-41a1-8835-23e6fe632ac5","Type":"ContainerDied","Data":"088eb68896b96a32a2aa5e02dd31e2ff784ca121b9b7ab08e5e3c6ca8f3071b4"}
Oct 30 18:52:33.341731 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:52:33.341729    4888 scope.go:122] "RemoveContainer" containerID="7fd6c9642e4902cced43ea59de35ee2a5ff5001f22c428a2d80a3258b3b161e7"
Oct 30 18:52:33.353322 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:52:33.353298    4888 scope.go:122] "RemoveContainer" containerID="7fd6c9642e4902cced43ea59de35ee2a5ff5001f22c428a2d80a3258b3b161e7"
Oct 30 18:52:33.353598 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:52:33.353580    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7fd6c9642e4902cced43ea59de35ee2a5ff5001f22c428a2d80a3258b3b161e7\": container with ID starting with 7fd6c9642e4902cced43ea59de35ee2a5ff5001f22c428a2d80a3258b3b161e7 not found: ID does not exist" containerID="7fd6c9642e4902cced43ea59de35ee2a5ff5001f22c428a2d80a3258b3b161e7"
Oct 30 18:52:33.353678 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:52:33.353606    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7fd6c9642e4902cced43ea59de35ee2a5ff5001f22c428a2d80a3258b3b161e7"} err="failed to get container status \"7fd6c9642e4902cced43ea59de35ee2a5ff5001f22c428a2d80a3258b3b161e7\": rpc error: code = NotFound desc = could not find container \"7fd6c9642e4902cced43ea59de35ee2a5ff5001f22c428a2d80a3258b3b161e7\": container with ID starting with 7fd6c9642e4902cced43ea59de35ee2a5ff5001f22c428a2d80a3258b3b161e7 not found: ID does not exist"
Oct 30 18:52:33.512172 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:52:33.512141    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["models-as-a-service/payload-processing-external-model-59c7d6dc54-pvvbg"]
Oct 30 18:52:33.521131 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:52:33.521101    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["models-as-a-service/payload-processing-external-model-59c7d6dc54-pvvbg"]
Oct 30 18:52:33.890435 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:52:33.890407    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9482318f-75fe-41a1-8835-23e6fe632ac5" path="/var/lib/kubelet/pods/9482318f-75fe-41a1-8835-23e6fe632ac5/volumes"
Oct 30 18:52:35.618212 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:52:35.618141    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-jbsw6" containerName="extractor"
Oct 30 18:52:39.185931 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:52:39.185902    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w3-3b6013/distinct-w3-3b6013-kserve-857f55d885-8fcrq"
Oct 30 18:52:39.199364 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:52:39.199334    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w3-3b6013/distinct-w3-3b6013-kserve-857f55d885-8fcrq"
Oct 30 18:52:39.393250 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:52:39.393213    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-3b6013/distinct-w3-3b6013-kserve-857f55d885-8fcrq"
Oct 30 18:52:39.511057 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:52:39.510969    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-rjmdz"]
Oct 30 18:53:05.620578 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:53:05.620538    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-jbsw6" containerName="extractor"
Oct 30 18:53:07.262436 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:07.262403    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-rjmdz"]
Oct 30 18:53:23.846841 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:23.846802    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-2ade09/e2e-conflict-a-d331d324-kserve-64f575d657-9zzz5"]
Oct 30 18:53:24.239118 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:24.239043    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-c682f"]
Oct 30 18:53:24.239614 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:24.239601    4888 cpu_manager.go:414] "containerMap: removing container" podUID="9482318f-75fe-41a1-8835-23e6fe632ac5" containerName="payload-processing"
Oct 30 18:53:24.239668 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:24.239615    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9482318f-75fe-41a1-8835-23e6fe632ac5" containerName="payload-processing"
Oct 30 18:53:24.239717 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:24.239709    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="9482318f-75fe-41a1-8835-23e6fe632ac5" containerName="payload-processing"
Oct 30 18:53:24.243127 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:24.243110    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-c682f"
Oct 30 18:53:24.249762 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:24.249734    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6462e226-4e52-40da-8fbd-656be33ed133-istio-data\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-c682f\" (UID: \"6462e226-4e52-40da-8fbd-656be33ed133\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-c682f"
Oct 30 18:53:24.249904 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:24.249768    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6462e226-4e52-40da-8fbd-656be33ed133-istiod-ca-cert\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-c682f\" (UID: \"6462e226-4e52-40da-8fbd-656be33ed133\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-c682f"
Oct 30 18:53:24.249904 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:24.249840    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6462e226-4e52-40da-8fbd-656be33ed133-workload-certs\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-c682f\" (UID: \"6462e226-4e52-40da-8fbd-656be33ed133\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-c682f"
Oct 30 18:53:24.250349 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:24.249922    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6462e226-4e52-40da-8fbd-656be33ed133-istio-envoy\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-c682f\" (UID: \"6462e226-4e52-40da-8fbd-656be33ed133\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-c682f"
Oct 30 18:53:24.250349 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:24.249992    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6462e226-4e52-40da-8fbd-656be33ed133-istio-token\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-c682f\" (UID: \"6462e226-4e52-40da-8fbd-656be33ed133\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-c682f"
Oct 30 18:53:24.250349 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:24.250063    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6462e226-4e52-40da-8fbd-656be33ed133-istio-podinfo\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-c682f\" (UID: \"6462e226-4e52-40da-8fbd-656be33ed133\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-c682f"
Oct 30 18:53:24.250349 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:24.250130    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6462e226-4e52-40da-8fbd-656be33ed133-workload-socket\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-c682f\" (UID: \"6462e226-4e52-40da-8fbd-656be33ed133\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-c682f"
Oct 30 18:53:24.250349 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:24.250184    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wxnvm\" (UniqueName: \"kubernetes.io/projected/6462e226-4e52-40da-8fbd-656be33ed133-kube-api-access-wxnvm\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-c682f\" (UID: \"6462e226-4e52-40da-8fbd-656be33ed133\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-c682f"
Oct 30 18:53:24.250349 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:24.250227    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6462e226-4e52-40da-8fbd-656be33ed133-credential-socket\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-c682f\" (UID: \"6462e226-4e52-40da-8fbd-656be33ed133\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-c682f"
Oct 30 18:53:24.296862 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:24.296836    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w0-2ade09-openshift-default-dockercfg-8v89k\""
Oct 30 18:53:24.351224 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:24.351191    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6462e226-4e52-40da-8fbd-656be33ed133-istio-data\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-c682f\" (UID: \"6462e226-4e52-40da-8fbd-656be33ed133\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-c682f"
Oct 30 18:53:24.351224 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:24.351225    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6462e226-4e52-40da-8fbd-656be33ed133-istiod-ca-cert\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-c682f\" (UID: \"6462e226-4e52-40da-8fbd-656be33ed133\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-c682f"
Oct 30 18:53:24.351479 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:24.351242    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-c682f"]
Oct 30 18:53:24.351479 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:24.351296    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6462e226-4e52-40da-8fbd-656be33ed133-workload-certs\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-c682f\" (UID: \"6462e226-4e52-40da-8fbd-656be33ed133\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-c682f"
Oct 30 18:53:24.351479 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:24.351328    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6462e226-4e52-40da-8fbd-656be33ed133-istio-envoy\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-c682f\" (UID: \"6462e226-4e52-40da-8fbd-656be33ed133\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-c682f"
Oct 30 18:53:24.351479 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:24.351386    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6462e226-4e52-40da-8fbd-656be33ed133-istio-token\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-c682f\" (UID: \"6462e226-4e52-40da-8fbd-656be33ed133\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-c682f"
Oct 30 18:53:24.351479 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:24.351451    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6462e226-4e52-40da-8fbd-656be33ed133-istio-podinfo\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-c682f\" (UID: \"6462e226-4e52-40da-8fbd-656be33ed133\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-c682f"
Oct 30 18:53:24.351742 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:24.351498    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6462e226-4e52-40da-8fbd-656be33ed133-workload-socket\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-c682f\" (UID: \"6462e226-4e52-40da-8fbd-656be33ed133\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-c682f"
Oct 30 18:53:24.351742 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:24.351563    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wxnvm\" (UniqueName: \"kubernetes.io/projected/6462e226-4e52-40da-8fbd-656be33ed133-kube-api-access-wxnvm\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-c682f\" (UID: \"6462e226-4e52-40da-8fbd-656be33ed133\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-c682f"
Oct 30 18:53:24.351742 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:24.351612    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6462e226-4e52-40da-8fbd-656be33ed133-credential-socket\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-c682f\" (UID: \"6462e226-4e52-40da-8fbd-656be33ed133\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-c682f"
Oct 30 18:53:24.351742 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:24.351679    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6462e226-4e52-40da-8fbd-656be33ed133-istio-data\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-c682f\" (UID: \"6462e226-4e52-40da-8fbd-656be33ed133\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-c682f"
Oct 30 18:53:24.351742 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:24.351729    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6462e226-4e52-40da-8fbd-656be33ed133-workload-certs\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-c682f\" (UID: \"6462e226-4e52-40da-8fbd-656be33ed133\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-c682f"
Oct 30 18:53:24.352000 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:24.351974    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6462e226-4e52-40da-8fbd-656be33ed133-workload-socket\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-c682f\" (UID: \"6462e226-4e52-40da-8fbd-656be33ed133\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-c682f"
Oct 30 18:53:24.352205 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:24.352186    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6462e226-4e52-40da-8fbd-656be33ed133-credential-socket\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-c682f\" (UID: \"6462e226-4e52-40da-8fbd-656be33ed133\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-c682f"
Oct 30 18:53:24.352298 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:24.352259    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6462e226-4e52-40da-8fbd-656be33ed133-istiod-ca-cert\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-c682f\" (UID: \"6462e226-4e52-40da-8fbd-656be33ed133\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-c682f"
Oct 30 18:53:24.353824 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:24.353800    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6462e226-4e52-40da-8fbd-656be33ed133-istio-envoy\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-c682f\" (UID: \"6462e226-4e52-40da-8fbd-656be33ed133\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-c682f"
Oct 30 18:53:24.354236 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:24.354214    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6462e226-4e52-40da-8fbd-656be33ed133-istio-podinfo\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-c682f\" (UID: \"6462e226-4e52-40da-8fbd-656be33ed133\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-c682f"
Oct 30 18:53:24.414322 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:24.414274    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6462e226-4e52-40da-8fbd-656be33ed133-istio-token\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-c682f\" (UID: \"6462e226-4e52-40da-8fbd-656be33ed133\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-c682f"
Oct 30 18:53:24.414843 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:24.414822    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wxnvm\" (UniqueName: \"kubernetes.io/projected/6462e226-4e52-40da-8fbd-656be33ed133-kube-api-access-wxnvm\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-c682f\" (UID: \"6462e226-4e52-40da-8fbd-656be33ed133\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-c682f"
Oct 30 18:53:24.556924 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:24.556850    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-c682f"
Oct 30 18:53:24.995200 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:24.995173    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-c682f"]
Oct 30 18:53:24.996428 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:53:24.996404    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6462e226_4e52_40da_8fbd_656be33ed133.slice/crio-6ebe969f8442d517dafc6a675fc322e4767a9a2cbb28cf023ebc06a24fc81960 WatchSource:0}: Error finding container 6ebe969f8442d517dafc6a675fc322e4767a9a2cbb28cf023ebc06a24fc81960: Status 404 returned error can't find the container with id 6ebe969f8442d517dafc6a675fc322e4767a9a2cbb28cf023ebc06a24fc81960
Oct 30 18:53:24.999414 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:24.999379    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:53:24.999532 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:24.999447    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:53:24.999532 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:24.999475    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:53:25.737504 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:25.737468    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-c682f" event={"ID":"6462e226-4e52-40da-8fbd-656be33ed133","Type":"ContainerStarted","Data":"c8ce1ce99d508fe76b4ba8ba847e8033ecbbb0ee6be74e03754ba147696d2d84"}
Oct 30 18:53:25.737684 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:25.737525    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-c682f" event={"ID":"6462e226-4e52-40da-8fbd-656be33ed133","Type":"ContainerStarted","Data":"6ebe969f8442d517dafc6a675fc322e4767a9a2cbb28cf023ebc06a24fc81960"}
Oct 30 18:53:25.911928 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:25.911878    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-c682f" podStartSLOduration=1.911864623 podStartE2EDuration="1.911864623s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:53:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:53:25.910226538 +0000 UTC m=+1170.157632574" watchObservedRunningTime="2026-10-30 18:53:25.911864623 +0000 UTC m=+1170.159270648"
Oct 30 18:53:26.557900 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:26.557865    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-c682f"
Oct 30 18:53:26.558998 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:26.558978    4888 patch_prober.go:29] interesting pod/e2e-worker-w0-2ade09-openshift-default-785b7cf675-c682f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.93:15021/healthz/ready\": dial tcp 10.132.0.93:15021: connect: connection refused" start-of-body=
Oct 30 18:53:26.559060 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:26.559039    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-c682f" podUID="6462e226-4e52-40da-8fbd-656be33ed133" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.93:15021/healthz/ready\": dial tcp 10.132.0.93:15021: connect: connection refused"
Oct 30 18:53:27.558067 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:27.558033    4888 patch_prober.go:29] interesting pod/e2e-worker-w0-2ade09-openshift-default-785b7cf675-c682f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.93:15021/healthz/ready\": dial tcp 10.132.0.93:15021: connect: connection refused" start-of-body=
Oct 30 18:53:27.558490 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:27.558097    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-c682f" podUID="6462e226-4e52-40da-8fbd-656be33ed133" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.93:15021/healthz/ready\": dial tcp 10.132.0.93:15021: connect: connection refused"
Oct 30 18:53:28.274884 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:28.274836    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-5h9tz"]
Oct 30 18:53:28.291219 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:28.291193    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-5h9tz"
Oct 30 18:53:28.322718 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:28.322686    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9rstj"]
Oct 30 18:53:28.327723 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:28.327701    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9rstj"
Oct 30 18:53:28.330382 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:28.330362    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5ffda80-881d-406d-8ef1-3664bf010a67-credential-socket\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-5h9tz\" (UID: \"f5ffda80-881d-406d-8ef1-3664bf010a67\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-5h9tz"
Oct 30 18:53:28.330472 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:28.330416    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5ffda80-881d-406d-8ef1-3664bf010a67-workload-socket\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-5h9tz\" (UID: \"f5ffda80-881d-406d-8ef1-3664bf010a67\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-5h9tz"
Oct 30 18:53:28.330548 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:28.330527    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f5ffda80-881d-406d-8ef1-3664bf010a67-istiod-ca-cert\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-5h9tz\" (UID: \"f5ffda80-881d-406d-8ef1-3664bf010a67\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-5h9tz"
Oct 30 18:53:28.330607 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:28.330568    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f5ffda80-881d-406d-8ef1-3664bf010a67-istio-envoy\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-5h9tz\" (UID: \"f5ffda80-881d-406d-8ef1-3664bf010a67\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-5h9tz"
Oct 30 18:53:28.330684 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:28.330672    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f5ffda80-881d-406d-8ef1-3664bf010a67-istio-podinfo\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-5h9tz\" (UID: \"f5ffda80-881d-406d-8ef1-3664bf010a67\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-5h9tz"
Oct 30 18:53:28.330725 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:28.330715    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dtjzr\" (UniqueName: \"kubernetes.io/projected/f5ffda80-881d-406d-8ef1-3664bf010a67-kube-api-access-dtjzr\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-5h9tz\" (UID: \"f5ffda80-881d-406d-8ef1-3664bf010a67\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-5h9tz"
Oct 30 18:53:28.330843 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:28.330820    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f5ffda80-881d-406d-8ef1-3664bf010a67-workload-certs\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-5h9tz\" (UID: \"f5ffda80-881d-406d-8ef1-3664bf010a67\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-5h9tz"
Oct 30 18:53:28.330896 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:28.330859    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f5ffda80-881d-406d-8ef1-3664bf010a67-istio-data\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-5h9tz\" (UID: \"f5ffda80-881d-406d-8ef1-3664bf010a67\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-5h9tz"
Oct 30 18:53:28.330942 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:28.330930    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f5ffda80-881d-406d-8ef1-3664bf010a67-istio-token\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-5h9tz\" (UID: \"f5ffda80-881d-406d-8ef1-3664bf010a67\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-5h9tz"
Oct 30 18:53:28.398329 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:28.398273    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-5h9tz"]
Oct 30 18:53:28.431607 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:28.431578    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f5ffda80-881d-406d-8ef1-3664bf010a67-workload-certs\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-5h9tz\" (UID: \"f5ffda80-881d-406d-8ef1-3664bf010a67\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-5h9tz"
Oct 30 18:53:28.431782 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:28.431617    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d5a7945-c6aa-453a-ac8c-1b23cddc1642-workload-socket\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9rstj\" (UID: \"0d5a7945-c6aa-453a-ac8c-1b23cddc1642\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9rstj"
Oct 30 18:53:28.431782 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:28.431647    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f5ffda80-881d-406d-8ef1-3664bf010a67-istio-data\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-5h9tz\" (UID: \"f5ffda80-881d-406d-8ef1-3664bf010a67\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-5h9tz"
Oct 30 18:53:28.431782 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:28.431672    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qfxlx\" (UniqueName: \"kubernetes.io/projected/0d5a7945-c6aa-453a-ac8c-1b23cddc1642-kube-api-access-qfxlx\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9rstj\" (UID: \"0d5a7945-c6aa-453a-ac8c-1b23cddc1642\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9rstj"
Oct 30 18:53:28.431782 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:28.431763    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f5ffda80-881d-406d-8ef1-3664bf010a67-istio-token\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-5h9tz\" (UID: \"f5ffda80-881d-406d-8ef1-3664bf010a67\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-5h9tz"
Oct 30 18:53:28.432008 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:28.431810    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5ffda80-881d-406d-8ef1-3664bf010a67-credential-socket\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-5h9tz\" (UID: \"f5ffda80-881d-406d-8ef1-3664bf010a67\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-5h9tz"
Oct 30 18:53:28.432008 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:28.431831    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0d5a7945-c6aa-453a-ac8c-1b23cddc1642-istio-data\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9rstj\" (UID: \"0d5a7945-c6aa-453a-ac8c-1b23cddc1642\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9rstj"
Oct 30 18:53:28.432008 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:28.431878    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5ffda80-881d-406d-8ef1-3664bf010a67-workload-socket\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-5h9tz\" (UID: \"f5ffda80-881d-406d-8ef1-3664bf010a67\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-5h9tz"
Oct 30 18:53:28.432008 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:28.431919    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0d5a7945-c6aa-453a-ac8c-1b23cddc1642-workload-certs\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9rstj\" (UID: \"0d5a7945-c6aa-453a-ac8c-1b23cddc1642\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9rstj"
Oct 30 18:53:28.432008 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:28.431976    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f5ffda80-881d-406d-8ef1-3664bf010a67-istiod-ca-cert\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-5h9tz\" (UID: \"f5ffda80-881d-406d-8ef1-3664bf010a67\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-5h9tz"
Oct 30 18:53:28.432008 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:28.431977    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f5ffda80-881d-406d-8ef1-3664bf010a67-workload-certs\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-5h9tz\" (UID: \"f5ffda80-881d-406d-8ef1-3664bf010a67\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-5h9tz"
Oct 30 18:53:28.432377 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:28.432024    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f5ffda80-881d-406d-8ef1-3664bf010a67-istio-envoy\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-5h9tz\" (UID: \"f5ffda80-881d-406d-8ef1-3664bf010a67\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-5h9tz"
Oct 30 18:53:28.432377 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:28.432034    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f5ffda80-881d-406d-8ef1-3664bf010a67-istio-data\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-5h9tz\" (UID: \"f5ffda80-881d-406d-8ef1-3664bf010a67\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-5h9tz"
Oct 30 18:53:28.432377 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:28.432052    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0d5a7945-c6aa-453a-ac8c-1b23cddc1642-istio-podinfo\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9rstj\" (UID: \"0d5a7945-c6aa-453a-ac8c-1b23cddc1642\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9rstj"
Oct 30 18:53:28.432377 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:28.432163    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5ffda80-881d-406d-8ef1-3664bf010a67-credential-socket\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-5h9tz\" (UID: \"f5ffda80-881d-406d-8ef1-3664bf010a67\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-5h9tz"
Oct 30 18:53:28.432377 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:28.432255    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0d5a7945-c6aa-453a-ac8c-1b23cddc1642-istio-token\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9rstj\" (UID: \"0d5a7945-c6aa-453a-ac8c-1b23cddc1642\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9rstj"
Oct 30 18:53:28.432377 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:28.432314    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5ffda80-881d-406d-8ef1-3664bf010a67-workload-socket\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-5h9tz\" (UID: \"f5ffda80-881d-406d-8ef1-3664bf010a67\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-5h9tz"
Oct 30 18:53:28.432377 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:28.432330    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0d5a7945-c6aa-453a-ac8c-1b23cddc1642-istiod-ca-cert\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9rstj\" (UID: \"0d5a7945-c6aa-453a-ac8c-1b23cddc1642\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9rstj"
Oct 30 18:53:28.432377 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:28.432380    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d5a7945-c6aa-453a-ac8c-1b23cddc1642-credential-socket\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9rstj\" (UID: \"0d5a7945-c6aa-453a-ac8c-1b23cddc1642\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9rstj"
Oct 30 18:53:28.432801 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:28.432437    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f5ffda80-881d-406d-8ef1-3664bf010a67-istio-podinfo\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-5h9tz\" (UID: \"f5ffda80-881d-406d-8ef1-3664bf010a67\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-5h9tz"
Oct 30 18:53:28.432801 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:28.432548    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dtjzr\" (UniqueName: \"kubernetes.io/projected/f5ffda80-881d-406d-8ef1-3664bf010a67-kube-api-access-dtjzr\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-5h9tz\" (UID: \"f5ffda80-881d-406d-8ef1-3664bf010a67\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-5h9tz"
Oct 30 18:53:28.432801 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:28.432603    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0d5a7945-c6aa-453a-ac8c-1b23cddc1642-istio-envoy\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9rstj\" (UID: \"0d5a7945-c6aa-453a-ac8c-1b23cddc1642\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9rstj"
Oct 30 18:53:28.432801 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:28.432740    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f5ffda80-881d-406d-8ef1-3664bf010a67-istiod-ca-cert\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-5h9tz\" (UID: \"f5ffda80-881d-406d-8ef1-3664bf010a67\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-5h9tz"
Oct 30 18:53:28.434889 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:28.434857    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f5ffda80-881d-406d-8ef1-3664bf010a67-istio-envoy\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-5h9tz\" (UID: \"f5ffda80-881d-406d-8ef1-3664bf010a67\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-5h9tz"
Oct 30 18:53:28.435495 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:28.435471    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f5ffda80-881d-406d-8ef1-3664bf010a67-istio-podinfo\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-5h9tz\" (UID: \"f5ffda80-881d-406d-8ef1-3664bf010a67\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-5h9tz"
Oct 30 18:53:28.451952 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:28.451925    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9rstj"]
Oct 30 18:53:28.500121 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:28.500045    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f5ffda80-881d-406d-8ef1-3664bf010a67-istio-token\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-5h9tz\" (UID: \"f5ffda80-881d-406d-8ef1-3664bf010a67\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-5h9tz"
Oct 30 18:53:28.503716 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:28.503659    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dtjzr\" (UniqueName: \"kubernetes.io/projected/f5ffda80-881d-406d-8ef1-3664bf010a67-kube-api-access-dtjzr\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-5h9tz\" (UID: \"f5ffda80-881d-406d-8ef1-3664bf010a67\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-5h9tz"
Oct 30 18:53:28.534311 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:28.534198    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0d5a7945-c6aa-453a-ac8c-1b23cddc1642-istio-data\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9rstj\" (UID: \"0d5a7945-c6aa-453a-ac8c-1b23cddc1642\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9rstj"
Oct 30 18:53:28.534469 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:28.534322    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0d5a7945-c6aa-453a-ac8c-1b23cddc1642-workload-certs\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9rstj\" (UID: \"0d5a7945-c6aa-453a-ac8c-1b23cddc1642\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9rstj"
Oct 30 18:53:28.534469 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:28.534398    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0d5a7945-c6aa-453a-ac8c-1b23cddc1642-istio-podinfo\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9rstj\" (UID: \"0d5a7945-c6aa-453a-ac8c-1b23cddc1642\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9rstj"
Oct 30 18:53:28.534469 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:28.534428    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0d5a7945-c6aa-453a-ac8c-1b23cddc1642-istio-token\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9rstj\" (UID: \"0d5a7945-c6aa-453a-ac8c-1b23cddc1642\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9rstj"
Oct 30 18:53:28.534469 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:28.534459    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0d5a7945-c6aa-453a-ac8c-1b23cddc1642-istiod-ca-cert\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9rstj\" (UID: \"0d5a7945-c6aa-453a-ac8c-1b23cddc1642\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9rstj"
Oct 30 18:53:28.534751 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:28.534498    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d5a7945-c6aa-453a-ac8c-1b23cddc1642-credential-socket\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9rstj\" (UID: \"0d5a7945-c6aa-453a-ac8c-1b23cddc1642\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9rstj"
Oct 30 18:53:28.534751 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:28.534606    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0d5a7945-c6aa-453a-ac8c-1b23cddc1642-istio-envoy\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9rstj\" (UID: \"0d5a7945-c6aa-453a-ac8c-1b23cddc1642\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9rstj"
Oct 30 18:53:28.534751 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:28.534659    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d5a7945-c6aa-453a-ac8c-1b23cddc1642-workload-socket\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9rstj\" (UID: \"0d5a7945-c6aa-453a-ac8c-1b23cddc1642\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9rstj"
Oct 30 18:53:28.534751 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:28.534702    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qfxlx\" (UniqueName: \"kubernetes.io/projected/0d5a7945-c6aa-453a-ac8c-1b23cddc1642-kube-api-access-qfxlx\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9rstj\" (UID: \"0d5a7945-c6aa-453a-ac8c-1b23cddc1642\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9rstj"
Oct 30 18:53:28.534967 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:28.534924    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d5a7945-c6aa-453a-ac8c-1b23cddc1642-credential-socket\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9rstj\" (UID: \"0d5a7945-c6aa-453a-ac8c-1b23cddc1642\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9rstj"
Oct 30 18:53:28.535228 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:28.535203    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d5a7945-c6aa-453a-ac8c-1b23cddc1642-workload-socket\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9rstj\" (UID: \"0d5a7945-c6aa-453a-ac8c-1b23cddc1642\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9rstj"
Oct 30 18:53:28.535355 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:28.535336    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0d5a7945-c6aa-453a-ac8c-1b23cddc1642-istiod-ca-cert\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9rstj\" (UID: \"0d5a7945-c6aa-453a-ac8c-1b23cddc1642\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9rstj"
Oct 30 18:53:28.535397 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:28.535377    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0d5a7945-c6aa-453a-ac8c-1b23cddc1642-workload-certs\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9rstj\" (UID: \"0d5a7945-c6aa-453a-ac8c-1b23cddc1642\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9rstj"
Oct 30 18:53:28.535603 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:28.535582    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0d5a7945-c6aa-453a-ac8c-1b23cddc1642-istio-data\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9rstj\" (UID: \"0d5a7945-c6aa-453a-ac8c-1b23cddc1642\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9rstj"
Oct 30 18:53:28.537823 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:28.537795    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0d5a7945-c6aa-453a-ac8c-1b23cddc1642-istio-envoy\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9rstj\" (UID: \"0d5a7945-c6aa-453a-ac8c-1b23cddc1642\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9rstj"
Oct 30 18:53:28.538787 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:28.538762    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0d5a7945-c6aa-453a-ac8c-1b23cddc1642-istio-podinfo\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9rstj\" (UID: \"0d5a7945-c6aa-453a-ac8c-1b23cddc1642\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9rstj"
Oct 30 18:53:28.558027 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:28.558002    4888 patch_prober.go:29] interesting pod/e2e-worker-w0-2ade09-openshift-default-785b7cf675-c682f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.93:15021/healthz/ready\": dial tcp 10.132.0.93:15021: connect: connection refused" start-of-body=
Oct 30 18:53:28.558109 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:28.558051    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-c682f" podUID="6462e226-4e52-40da-8fbd-656be33ed133" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.93:15021/healthz/ready\": dial tcp 10.132.0.93:15021: connect: connection refused"
Oct 30 18:53:28.597679 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:28.597645    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0d5a7945-c6aa-453a-ac8c-1b23cddc1642-istio-token\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9rstj\" (UID: \"0d5a7945-c6aa-453a-ac8c-1b23cddc1642\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9rstj"
Oct 30 18:53:28.598081 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:28.598055    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qfxlx\" (UniqueName: \"kubernetes.io/projected/0d5a7945-c6aa-453a-ac8c-1b23cddc1642-kube-api-access-qfxlx\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9rstj\" (UID: \"0d5a7945-c6aa-453a-ac8c-1b23cddc1642\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9rstj"
Oct 30 18:53:28.609766 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:28.609744    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-5h9tz"
Oct 30 18:53:28.641333 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:28.641308    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9rstj"
Oct 30 18:53:29.063234 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:29.063202    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-5h9tz"]
Oct 30 18:53:29.063452 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:53:29.063434    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf5ffda80_881d_406d_8ef1_3664bf010a67.slice/crio-d2abb2363e118338bb9cf40bd776a36fd868a4119963f72fc1ff687cbc41142a WatchSource:0}: Error finding container d2abb2363e118338bb9cf40bd776a36fd868a4119963f72fc1ff687cbc41142a: Status 404 returned error can't find the container with id d2abb2363e118338bb9cf40bd776a36fd868a4119963f72fc1ff687cbc41142a
Oct 30 18:53:29.065916 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:29.065886    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:53:29.065982 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:29.065948    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:53:29.065982 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:29.065975    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:53:29.066833 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:29.066815    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9rstj"]
Oct 30 18:53:29.103393 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:53:29.103360    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d5a7945_c6aa_453a_ac8c_1b23cddc1642.slice/crio-01f81bcb2ae7b40ff48843868e0e28a2f1b1b90d25c435acd825d8852b967415 WatchSource:0}: Error finding container 01f81bcb2ae7b40ff48843868e0e28a2f1b1b90d25c435acd825d8852b967415: Status 404 returned error can't find the container with id 01f81bcb2ae7b40ff48843868e0e28a2f1b1b90d25c435acd825d8852b967415
Oct 30 18:53:29.106635 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:29.106597    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:53:29.106759 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:29.106691    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:53:29.106823 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:29.106788    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:53:29.557329 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:29.557274    4888 patch_prober.go:29] interesting pod/e2e-worker-w0-2ade09-openshift-default-785b7cf675-c682f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.93:15021/healthz/ready\": dial tcp 10.132.0.93:15021: connect: connection refused" start-of-body=
Oct 30 18:53:29.557523 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:29.557353    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-c682f" podUID="6462e226-4e52-40da-8fbd-656be33ed133" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.93:15021/healthz/ready\": dial tcp 10.132.0.93:15021: connect: connection refused"
Oct 30 18:53:29.739750 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:29.739717    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4p2zp"]
Oct 30 18:53:29.751521 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:29.745165    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4p2zp"
Oct 30 18:53:29.883315 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:29.881026    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2ba277ac-44b2-4d12-b873-6b584085f0d1-workload-certs\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4p2zp\" (UID: \"2ba277ac-44b2-4d12-b873-6b584085f0d1\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4p2zp"
Oct 30 18:53:29.883315 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:29.881087    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2ba277ac-44b2-4d12-b873-6b584085f0d1-istio-token\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4p2zp\" (UID: \"2ba277ac-44b2-4d12-b873-6b584085f0d1\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4p2zp"
Oct 30 18:53:29.883315 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:29.881137    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ba277ac-44b2-4d12-b873-6b584085f0d1-workload-socket\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4p2zp\" (UID: \"2ba277ac-44b2-4d12-b873-6b584085f0d1\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4p2zp"
Oct 30 18:53:29.883315 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:29.881230    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ba277ac-44b2-4d12-b873-6b584085f0d1-credential-socket\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4p2zp\" (UID: \"2ba277ac-44b2-4d12-b873-6b584085f0d1\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4p2zp"
Oct 30 18:53:29.883315 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:29.881258    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-stnfd\" (UniqueName: \"kubernetes.io/projected/2ba277ac-44b2-4d12-b873-6b584085f0d1-kube-api-access-stnfd\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4p2zp\" (UID: \"2ba277ac-44b2-4d12-b873-6b584085f0d1\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4p2zp"
Oct 30 18:53:29.883315 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:29.881343    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2ba277ac-44b2-4d12-b873-6b584085f0d1-istio-podinfo\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4p2zp\" (UID: \"2ba277ac-44b2-4d12-b873-6b584085f0d1\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4p2zp"
Oct 30 18:53:29.883315 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:29.881403    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2ba277ac-44b2-4d12-b873-6b584085f0d1-istiod-ca-cert\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4p2zp\" (UID: \"2ba277ac-44b2-4d12-b873-6b584085f0d1\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4p2zp"
Oct 30 18:53:29.883315 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:29.881544    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2ba277ac-44b2-4d12-b873-6b584085f0d1-istio-envoy\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4p2zp\" (UID: \"2ba277ac-44b2-4d12-b873-6b584085f0d1\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4p2zp"
Oct 30 18:53:29.883315 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:29.881583    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2ba277ac-44b2-4d12-b873-6b584085f0d1-istio-data\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4p2zp\" (UID: \"2ba277ac-44b2-4d12-b873-6b584085f0d1\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4p2zp"
Oct 30 18:53:30.052496 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:30.052450    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2ba277ac-44b2-4d12-b873-6b584085f0d1-istio-envoy\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4p2zp\" (UID: \"2ba277ac-44b2-4d12-b873-6b584085f0d1\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4p2zp"
Oct 30 18:53:30.052708 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:30.052550    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2ba277ac-44b2-4d12-b873-6b584085f0d1-istio-data\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4p2zp\" (UID: \"2ba277ac-44b2-4d12-b873-6b584085f0d1\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4p2zp"
Oct 30 18:53:30.052708 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:30.052598    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2ba277ac-44b2-4d12-b873-6b584085f0d1-workload-certs\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4p2zp\" (UID: \"2ba277ac-44b2-4d12-b873-6b584085f0d1\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4p2zp"
Oct 30 18:53:30.052708 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:30.052634    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2ba277ac-44b2-4d12-b873-6b584085f0d1-istio-token\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4p2zp\" (UID: \"2ba277ac-44b2-4d12-b873-6b584085f0d1\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4p2zp"
Oct 30 18:53:30.052708 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:30.052669    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ba277ac-44b2-4d12-b873-6b584085f0d1-workload-socket\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4p2zp\" (UID: \"2ba277ac-44b2-4d12-b873-6b584085f0d1\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4p2zp"
Oct 30 18:53:30.052926 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:30.052816    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ba277ac-44b2-4d12-b873-6b584085f0d1-credential-socket\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4p2zp\" (UID: \"2ba277ac-44b2-4d12-b873-6b584085f0d1\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4p2zp"
Oct 30 18:53:30.052926 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:30.052846    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-stnfd\" (UniqueName: \"kubernetes.io/projected/2ba277ac-44b2-4d12-b873-6b584085f0d1-kube-api-access-stnfd\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4p2zp\" (UID: \"2ba277ac-44b2-4d12-b873-6b584085f0d1\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4p2zp"
Oct 30 18:53:30.053025 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:30.052933    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2ba277ac-44b2-4d12-b873-6b584085f0d1-istio-podinfo\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4p2zp\" (UID: \"2ba277ac-44b2-4d12-b873-6b584085f0d1\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4p2zp"
Oct 30 18:53:30.053077 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:30.053034    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2ba277ac-44b2-4d12-b873-6b584085f0d1-istiod-ca-cert\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4p2zp\" (UID: \"2ba277ac-44b2-4d12-b873-6b584085f0d1\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4p2zp"
Oct 30 18:53:30.063322 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:30.061469    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2ba277ac-44b2-4d12-b873-6b584085f0d1-istio-data\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4p2zp\" (UID: \"2ba277ac-44b2-4d12-b873-6b584085f0d1\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4p2zp"
Oct 30 18:53:30.063322 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:30.061741    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2ba277ac-44b2-4d12-b873-6b584085f0d1-workload-certs\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4p2zp\" (UID: \"2ba277ac-44b2-4d12-b873-6b584085f0d1\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4p2zp"
Oct 30 18:53:30.063322 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:30.062346    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ba277ac-44b2-4d12-b873-6b584085f0d1-workload-socket\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4p2zp\" (UID: \"2ba277ac-44b2-4d12-b873-6b584085f0d1\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4p2zp"
Oct 30 18:53:30.063322 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:30.062926    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ba277ac-44b2-4d12-b873-6b584085f0d1-credential-socket\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4p2zp\" (UID: \"2ba277ac-44b2-4d12-b873-6b584085f0d1\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4p2zp"
Oct 30 18:53:30.064422 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:30.064399    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2ba277ac-44b2-4d12-b873-6b584085f0d1-istiod-ca-cert\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4p2zp\" (UID: \"2ba277ac-44b2-4d12-b873-6b584085f0d1\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4p2zp"
Oct 30 18:53:30.090977 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:30.068557    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2ba277ac-44b2-4d12-b873-6b584085f0d1-istio-envoy\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4p2zp\" (UID: \"2ba277ac-44b2-4d12-b873-6b584085f0d1\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4p2zp"
Oct 30 18:53:30.130334 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:30.118991    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2ba277ac-44b2-4d12-b873-6b584085f0d1-istio-podinfo\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4p2zp\" (UID: \"2ba277ac-44b2-4d12-b873-6b584085f0d1\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4p2zp"
Oct 30 18:53:30.130334 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:30.127571    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-5h9tz" event={"ID":"f5ffda80-881d-406d-8ef1-3664bf010a67","Type":"ContainerStarted","Data":"57b68653ad3ea8bc9ed04b8eb1be80126b104daa84ab371dfa97f97a8b90d314"}
Oct 30 18:53:30.130334 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:30.127635    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4p2zp"]
Oct 30 18:53:30.130334 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:30.127674    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-5h9tz" event={"ID":"f5ffda80-881d-406d-8ef1-3664bf010a67","Type":"ContainerStarted","Data":"d2abb2363e118338bb9cf40bd776a36fd868a4119963f72fc1ff687cbc41142a"}
Oct 30 18:53:30.130334 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:30.127695    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9rstj" event={"ID":"0d5a7945-c6aa-453a-ac8c-1b23cddc1642","Type":"ContainerStarted","Data":"f0bc676121e67f034883592e084979545dbb256182eae9e11347cd66b32d7caa"}
Oct 30 18:53:30.130334 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:30.127721    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9rstj" event={"ID":"0d5a7945-c6aa-453a-ac8c-1b23cddc1642","Type":"ContainerStarted","Data":"01f81bcb2ae7b40ff48843868e0e28a2f1b1b90d25c435acd825d8852b967415"}
Oct 30 18:53:30.257795 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:30.249384    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2ba277ac-44b2-4d12-b873-6b584085f0d1-istio-token\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4p2zp\" (UID: \"2ba277ac-44b2-4d12-b873-6b584085f0d1\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4p2zp"
Oct 30 18:53:30.257795 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:30.257148    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-5h9tz" podStartSLOduration=2.2571286 podStartE2EDuration="2.2571286s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:53:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:53:30.249180953 +0000 UTC m=+1174.496586989" watchObservedRunningTime="2026-10-30 18:53:30.2571286 +0000 UTC m=+1174.504534628"
Oct 30 18:53:30.257795 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:30.257548    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-stnfd\" (UniqueName: \"kubernetes.io/projected/2ba277ac-44b2-4d12-b873-6b584085f0d1-kube-api-access-stnfd\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4p2zp\" (UID: \"2ba277ac-44b2-4d12-b873-6b584085f0d1\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4p2zp"
Oct 30 18:53:30.402311 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:30.401531    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4p2zp"
Oct 30 18:53:30.463022 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:30.462946    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9rstj" podStartSLOduration=2.462926382 podStartE2EDuration="2.462926382s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:53:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:53:30.460630592 +0000 UTC m=+1174.708036620" watchObservedRunningTime="2026-10-30 18:53:30.462926382 +0000 UTC m=+1174.710332409"
Oct 30 18:53:30.557666 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:30.557583    4888 patch_prober.go:29] interesting pod/e2e-worker-w0-2ade09-openshift-default-785b7cf675-c682f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.93:15021/healthz/ready\": dial tcp 10.132.0.93:15021: connect: connection refused" start-of-body=
Oct 30 18:53:30.557812 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:30.557657    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-c682f" podUID="6462e226-4e52-40da-8fbd-656be33ed133" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.93:15021/healthz/ready\": dial tcp 10.132.0.93:15021: connect: connection refused"
Oct 30 18:53:30.618326 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:30.618250    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-5h9tz"
Oct 30 18:53:30.626318 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:30.622394    4888 patch_prober.go:29] interesting pod/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-5h9tz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused" start-of-body=
Oct 30 18:53:30.626318 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:30.622450    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-5h9tz" podUID="f5ffda80-881d-406d-8ef1-3664bf010a67" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused"
Oct 30 18:53:30.646308 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:30.645669    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9rstj"
Oct 30 18:53:30.650393 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:30.649397    4888 patch_prober.go:29] interesting pod/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9rstj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused" start-of-body=
Oct 30 18:53:30.650393 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:30.649458    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9rstj" podUID="0d5a7945-c6aa-453a-ac8c-1b23cddc1642" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused"
Oct 30 18:53:31.262421 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:31.262384    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4p2zp"]
Oct 30 18:53:31.354947 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:31.354904    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:53:31.355043 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:31.354999    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:53:31.355088 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:31.355040    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:53:31.561315 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:31.557978    4888 patch_prober.go:29] interesting pod/e2e-worker-w0-2ade09-openshift-default-785b7cf675-c682f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.93:15021/healthz/ready\": dial tcp 10.132.0.93:15021: connect: connection refused" start-of-body=
Oct 30 18:53:31.561315 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:31.558054    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-c682f" podUID="6462e226-4e52-40da-8fbd-656be33ed133" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.93:15021/healthz/ready\": dial tcp 10.132.0.93:15021: connect: connection refused"
Oct 30 18:53:31.619883 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:31.619855    4888 patch_prober.go:29] interesting pod/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-5h9tz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused" start-of-body=
Oct 30 18:53:31.620165 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:31.620124    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-5h9tz" podUID="f5ffda80-881d-406d-8ef1-3664bf010a67" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused"
Oct 30 18:53:31.641892 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:31.641855    4888 patch_prober.go:29] interesting pod/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9rstj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused" start-of-body=
Oct 30 18:53:31.642057 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:31.641911    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9rstj" podUID="0d5a7945-c6aa-453a-ac8c-1b23cddc1642" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused"
Oct 30 18:53:32.159519 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:32.159482    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4p2zp" event={"ID":"2ba277ac-44b2-4d12-b873-6b584085f0d1","Type":"ContainerStarted","Data":"5432460e6fed255408e2c642cd55b7ce9c4d906e7636c508abe5d5f10ea2df2b"}
Oct 30 18:53:32.159714 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:32.159538    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4p2zp" event={"ID":"2ba277ac-44b2-4d12-b873-6b584085f0d1","Type":"ContainerStarted","Data":"429dec9d2859989e15abb5415eea6dcf7cea0396957b7e3516500c3ab23fb664"}
Oct 30 18:53:32.296530 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:32.293664    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4p2zp" podStartSLOduration=3.293643998 podStartE2EDuration="3.293643998s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:53:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:53:32.289721463 +0000 UTC m=+1176.537127489" watchObservedRunningTime="2026-10-30 18:53:32.293643998 +0000 UTC m=+1176.541050024"
Oct 30 18:53:32.412441 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:32.412350    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4p2zp"
Oct 30 18:53:32.412693 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:32.412429    4888 patch_prober.go:29] interesting pod/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4p2zp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused" start-of-body=
Oct 30 18:53:32.412844 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:32.412820    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4p2zp" podUID="2ba277ac-44b2-4d12-b873-6b584085f0d1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused"
Oct 30 18:53:32.610820 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:32.610784    4888 patch_prober.go:29] interesting pod/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-5h9tz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused" start-of-body=
Oct 30 18:53:32.611138 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:32.611092    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-5h9tz" podUID="f5ffda80-881d-406d-8ef1-3664bf010a67" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused"
Oct 30 18:53:32.643782 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:32.643747    4888 patch_prober.go:29] interesting pod/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9rstj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused" start-of-body=
Oct 30 18:53:32.644103 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:32.644058    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9rstj" podUID="0d5a7945-c6aa-453a-ac8c-1b23cddc1642" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused"
Oct 30 18:53:33.087687 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:33.087651    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-c682f"
Oct 30 18:53:33.091875 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:33.091851    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-c682f"
Oct 30 18:53:33.099545 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:33.099523    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-c682f"
Oct 30 18:53:33.559363 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:33.554741    4888 patch_prober.go:29] interesting pod/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4p2zp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused" start-of-body=
Oct 30 18:53:33.559363 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:33.554859    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4p2zp" podUID="2ba277ac-44b2-4d12-b873-6b584085f0d1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused"
Oct 30 18:53:33.618874 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:33.618837    4888 patch_prober.go:29] interesting pod/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-5h9tz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused" start-of-body=
Oct 30 18:53:33.619130 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:33.619099    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-5h9tz" podUID="f5ffda80-881d-406d-8ef1-3664bf010a67" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused"
Oct 30 18:53:33.654218 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:33.653952    4888 patch_prober.go:29] interesting pod/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9rstj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused" start-of-body=
Oct 30 18:53:33.654218 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:33.654005    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9rstj" podUID="0d5a7945-c6aa-453a-ac8c-1b23cddc1642" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused"
Oct 30 18:53:34.405678 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:34.405641    4888 patch_prober.go:29] interesting pod/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4p2zp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused" start-of-body=
Oct 30 18:53:34.405993 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:34.405960    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4p2zp" podUID="2ba277ac-44b2-4d12-b873-6b584085f0d1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused"
Oct 30 18:53:34.612815 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:34.612780    4888 patch_prober.go:29] interesting pod/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-5h9tz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused" start-of-body=
Oct 30 18:53:34.613485 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:34.613440    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-5h9tz" podUID="f5ffda80-881d-406d-8ef1-3664bf010a67" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused"
Oct 30 18:53:34.645731 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:34.645676    4888 patch_prober.go:29] interesting pod/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9rstj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused" start-of-body=
Oct 30 18:53:34.645943 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:34.645744    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9rstj" podUID="0d5a7945-c6aa-453a-ac8c-1b23cddc1642" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused"
Oct 30 18:53:35.410707 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:35.410668    4888 patch_prober.go:29] interesting pod/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4p2zp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused" start-of-body=
Oct 30 18:53:35.411007 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:35.410981    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4p2zp" podUID="2ba277ac-44b2-4d12-b873-6b584085f0d1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused"
Oct 30 18:53:35.611382 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:35.610917    4888 patch_prober.go:29] interesting pod/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-5h9tz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused" start-of-body=
Oct 30 18:53:35.611382 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:35.610977    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-5h9tz" podUID="f5ffda80-881d-406d-8ef1-3664bf010a67" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused"
Oct 30 18:53:35.637891 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:53:35.637847    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-jbsw6" containerName="extractor"
Oct 30 18:53:35.642377 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:35.642348    4888 patch_prober.go:29] interesting pod/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9rstj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused" start-of-body=
Oct 30 18:53:35.642506 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:35.642423    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9rstj" podUID="0d5a7945-c6aa-453a-ac8c-1b23cddc1642" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused"
Oct 30 18:53:36.410712 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:36.410675    4888 patch_prober.go:29] interesting pod/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4p2zp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused" start-of-body=
Oct 30 18:53:36.410909 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:36.410738    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4p2zp" podUID="2ba277ac-44b2-4d12-b873-6b584085f0d1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused"
Oct 30 18:53:37.223797 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:37.223763    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-5h9tz"
Oct 30 18:53:37.225149 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:37.225124    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-5h9tz"
Oct 30 18:53:37.225789 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:37.225766    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-5h9tz"
Oct 30 18:53:37.247173 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:37.247147    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9rstj"
Oct 30 18:53:37.247710 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:37.247692    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9rstj"
Oct 30 18:53:37.249517 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:37.249500    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9rstj"
Oct 30 18:53:37.403715 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:37.403680    4888 patch_prober.go:29] interesting pod/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4p2zp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused" start-of-body=
Oct 30 18:53:37.403931 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:37.403724    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4p2zp" podUID="2ba277ac-44b2-4d12-b873-6b584085f0d1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused"
Oct 30 18:53:38.403646 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:38.403611    4888 patch_prober.go:29] interesting pod/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4p2zp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused" start-of-body=
Oct 30 18:53:38.404094 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:38.403671    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4p2zp" podUID="2ba277ac-44b2-4d12-b873-6b584085f0d1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused"
Oct 30 18:53:38.874201 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:38.874160    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-2ade09/e2e-conflict-a-d331d324-kserve-64f575d657-9zzz5" podUID="27762cc5-706e-4d70-a701-ab5f82c893d2" containerName="main" containerID="cri-o://2236c97a36263fcd0d91f1d504acbe0ea7719143a7d47b8d57dbdc02b8c167a5" gracePeriod=45
Oct 30 18:53:39.165980 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:39.165950    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-2ade09/e2e-conflict-a-d331d324-kserve-64f575d657-9zzz5"
Oct 30 18:53:39.209253 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:39.209214    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/27762cc5-706e-4d70-a701-ab5f82c893d2-dshm\" (UniqueName: \"kubernetes.io/empty-dir/27762cc5-706e-4d70-a701-ab5f82c893d2-dshm\") pod \"27762cc5-706e-4d70-a701-ab5f82c893d2\" (UID: \"27762cc5-706e-4d70-a701-ab5f82c893d2\") "
Oct 30 18:53:39.209468 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:39.209307    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/27762cc5-706e-4d70-a701-ab5f82c893d2-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/27762cc5-706e-4d70-a701-ab5f82c893d2-model-cache\") pod \"27762cc5-706e-4d70-a701-ab5f82c893d2\" (UID: \"27762cc5-706e-4d70-a701-ab5f82c893d2\") "
Oct 30 18:53:39.209468 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:39.209376    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/27762cc5-706e-4d70-a701-ab5f82c893d2-home\" (UniqueName: \"kubernetes.io/empty-dir/27762cc5-706e-4d70-a701-ab5f82c893d2-home\") pod \"27762cc5-706e-4d70-a701-ab5f82c893d2\" (UID: \"27762cc5-706e-4d70-a701-ab5f82c893d2\") "
Oct 30 18:53:39.209468 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:39.209413    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/27762cc5-706e-4d70-a701-ab5f82c893d2-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/27762cc5-706e-4d70-a701-ab5f82c893d2-tmp-dir\") pod \"27762cc5-706e-4d70-a701-ab5f82c893d2\" (UID: \"27762cc5-706e-4d70-a701-ab5f82c893d2\") "
Oct 30 18:53:39.209468 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:39.209447    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/27762cc5-706e-4d70-a701-ab5f82c893d2-tls-certs\" (UniqueName: \"kubernetes.io/secret/27762cc5-706e-4d70-a701-ab5f82c893d2-tls-certs\") pod \"27762cc5-706e-4d70-a701-ab5f82c893d2\" (UID: \"27762cc5-706e-4d70-a701-ab5f82c893d2\") "
Oct 30 18:53:39.209640 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:39.209530    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/27762cc5-706e-4d70-a701-ab5f82c893d2-kube-api-access-9x66z\" (UniqueName: \"kubernetes.io/projected/27762cc5-706e-4d70-a701-ab5f82c893d2-kube-api-access-9x66z\") pod \"27762cc5-706e-4d70-a701-ab5f82c893d2\" (UID: \"27762cc5-706e-4d70-a701-ab5f82c893d2\") "
Oct 30 18:53:39.209693 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:39.209630    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/27762cc5-706e-4d70-a701-ab5f82c893d2-model-cache" pod "27762cc5-706e-4d70-a701-ab5f82c893d2" (UID: "27762cc5-706e-4d70-a701-ab5f82c893d2"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:53:39.210133 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:39.210096    4888 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/27762cc5-706e-4d70-a701-ab5f82c893d2-model-cache\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:53:39.210133 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:39.210115    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/27762cc5-706e-4d70-a701-ab5f82c893d2-tmp-dir" pod "27762cc5-706e-4d70-a701-ab5f82c893d2" (UID: "27762cc5-706e-4d70-a701-ab5f82c893d2"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:53:39.210813 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:39.210785    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/27762cc5-706e-4d70-a701-ab5f82c893d2-home" pod "27762cc5-706e-4d70-a701-ab5f82c893d2" (UID: "27762cc5-706e-4d70-a701-ab5f82c893d2"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:53:39.212667 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:39.212639    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/27762cc5-706e-4d70-a701-ab5f82c893d2-dshm" pod "27762cc5-706e-4d70-a701-ab5f82c893d2" (UID: "27762cc5-706e-4d70-a701-ab5f82c893d2"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:53:39.212667 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:39.212651    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/27762cc5-706e-4d70-a701-ab5f82c893d2-kube-api-access-9x66z" pod "27762cc5-706e-4d70-a701-ab5f82c893d2" (UID: "27762cc5-706e-4d70-a701-ab5f82c893d2"). InnerVolumeSpecName "kube-api-access-9x66z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:53:39.212982 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:39.212959    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/27762cc5-706e-4d70-a701-ab5f82c893d2-tls-certs" pod "27762cc5-706e-4d70-a701-ab5f82c893d2" (UID: "27762cc5-706e-4d70-a701-ab5f82c893d2"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:53:39.303324 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:39.301677    4888 generic.go:356] "Generic (PLEG): container finished" podID="27762cc5-706e-4d70-a701-ab5f82c893d2" containerID="2236c97a36263fcd0d91f1d504acbe0ea7719143a7d47b8d57dbdc02b8c167a5" exitCode=0
Oct 30 18:53:39.303324 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:39.301786    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-2ade09/e2e-conflict-a-d331d324-kserve-64f575d657-9zzz5" event={"ID":"27762cc5-706e-4d70-a701-ab5f82c893d2","Type":"ContainerDied","Data":"2236c97a36263fcd0d91f1d504acbe0ea7719143a7d47b8d57dbdc02b8c167a5"}
Oct 30 18:53:39.303324 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:39.301821    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-2ade09/e2e-conflict-a-d331d324-kserve-64f575d657-9zzz5" event={"ID":"27762cc5-706e-4d70-a701-ab5f82c893d2","Type":"ContainerDied","Data":"0b29f868e4137d126bb229808bfc29b34553c081f9c0c2380575f6004b9e8b15"}
Oct 30 18:53:39.303324 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:39.301845    4888 scope.go:122] "RemoveContainer" containerID="2236c97a36263fcd0d91f1d504acbe0ea7719143a7d47b8d57dbdc02b8c167a5"
Oct 30 18:53:39.303324 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:39.302007    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-2ade09/e2e-conflict-a-d331d324-kserve-64f575d657-9zzz5"
Oct 30 18:53:39.311540 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:39.311500    4888 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/27762cc5-706e-4d70-a701-ab5f82c893d2-dshm\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:53:39.311540 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:39.311535    4888 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/27762cc5-706e-4d70-a701-ab5f82c893d2-home\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:53:39.311727 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:39.311549    4888 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/27762cc5-706e-4d70-a701-ab5f82c893d2-tmp-dir\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:53:39.311727 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:39.311561    4888 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/27762cc5-706e-4d70-a701-ab5f82c893d2-tls-certs\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:53:39.311727 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:39.311575    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9x66z\" (UniqueName: \"kubernetes.io/projected/27762cc5-706e-4d70-a701-ab5f82c893d2-kube-api-access-9x66z\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 18:53:39.317368 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:39.317346    4888 scope.go:122] "RemoveContainer" containerID="2236c97a36263fcd0d91f1d504acbe0ea7719143a7d47b8d57dbdc02b8c167a5"
Oct 30 18:53:39.317948 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:53:39.317874    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2236c97a36263fcd0d91f1d504acbe0ea7719143a7d47b8d57dbdc02b8c167a5\": container with ID starting with 2236c97a36263fcd0d91f1d504acbe0ea7719143a7d47b8d57dbdc02b8c167a5 not found: ID does not exist" containerID="2236c97a36263fcd0d91f1d504acbe0ea7719143a7d47b8d57dbdc02b8c167a5"
Oct 30 18:53:39.317948 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:39.317916    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2236c97a36263fcd0d91f1d504acbe0ea7719143a7d47b8d57dbdc02b8c167a5"} err="failed to get container status \"2236c97a36263fcd0d91f1d504acbe0ea7719143a7d47b8d57dbdc02b8c167a5\": rpc error: code = NotFound desc = could not find container \"2236c97a36263fcd0d91f1d504acbe0ea7719143a7d47b8d57dbdc02b8c167a5\": container with ID starting with 2236c97a36263fcd0d91f1d504acbe0ea7719143a7d47b8d57dbdc02b8c167a5 not found: ID does not exist"
Oct 30 18:53:39.440458 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:39.440386    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4p2zp"
Oct 30 18:53:39.440959 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:39.440942    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4p2zp"
Oct 30 18:53:39.441710 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:39.441688    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4p2zp"
Oct 30 18:53:39.476307 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:39.476146    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-2ade09/e2e-conflict-a-d331d324-kserve-64f575d657-9zzz5"]
Oct 30 18:53:39.480025 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:39.479996    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-2ade09/e2e-conflict-a-d331d324-kserve-64f575d657-9zzz5"]
Oct 30 18:53:39.890338 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:39.890309    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="27762cc5-706e-4d70-a701-ab5f82c893d2" path="/var/lib/kubelet/pods/27762cc5-706e-4d70-a701-ab5f82c893d2/volumes"
Oct 30 18:53:44.805614 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:44.805580    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-ngqb5"]
Oct 30 18:53:44.806126 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:44.806113    4888 cpu_manager.go:414] "containerMap: removing container" podUID="27762cc5-706e-4d70-a701-ab5f82c893d2" containerName="main"
Oct 30 18:53:44.806168 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:44.806127    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="27762cc5-706e-4d70-a701-ab5f82c893d2" containerName="main"
Oct 30 18:53:44.806254 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:44.806245    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="27762cc5-706e-4d70-a701-ab5f82c893d2" containerName="main"
Oct 30 18:53:44.809676 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:44.809657    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-ngqb5"
Oct 30 18:53:44.874644 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:44.874609    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-ngqb5"]
Oct 30 18:53:44.881332 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:44.881275    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d33b7312-dc19-402a-9c2e-12e0b7eee906-istio-podinfo\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-ngqb5\" (UID: \"d33b7312-dc19-402a-9c2e-12e0b7eee906\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-ngqb5"
Oct 30 18:53:44.881684 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:44.881648    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d33b7312-dc19-402a-9c2e-12e0b7eee906-workload-certs\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-ngqb5\" (UID: \"d33b7312-dc19-402a-9c2e-12e0b7eee906\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-ngqb5"
Oct 30 18:53:44.881800 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:44.881701    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d33b7312-dc19-402a-9c2e-12e0b7eee906-istio-envoy\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-ngqb5\" (UID: \"d33b7312-dc19-402a-9c2e-12e0b7eee906\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-ngqb5"
Oct 30 18:53:44.881800 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:44.881739    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d33b7312-dc19-402a-9c2e-12e0b7eee906-workload-socket\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-ngqb5\" (UID: \"d33b7312-dc19-402a-9c2e-12e0b7eee906\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-ngqb5"
Oct 30 18:53:44.881887 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:44.881868    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d33b7312-dc19-402a-9c2e-12e0b7eee906-istiod-ca-cert\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-ngqb5\" (UID: \"d33b7312-dc19-402a-9c2e-12e0b7eee906\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-ngqb5"
Oct 30 18:53:44.881929 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:44.881912    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d33b7312-dc19-402a-9c2e-12e0b7eee906-credential-socket\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-ngqb5\" (UID: \"d33b7312-dc19-402a-9c2e-12e0b7eee906\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-ngqb5"
Oct 30 18:53:44.881966 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:44.881956    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vdt4h\" (UniqueName: \"kubernetes.io/projected/d33b7312-dc19-402a-9c2e-12e0b7eee906-kube-api-access-vdt4h\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-ngqb5\" (UID: \"d33b7312-dc19-402a-9c2e-12e0b7eee906\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-ngqb5"
Oct 30 18:53:44.882005 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:44.881991    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d33b7312-dc19-402a-9c2e-12e0b7eee906-istio-token\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-ngqb5\" (UID: \"d33b7312-dc19-402a-9c2e-12e0b7eee906\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-ngqb5"
Oct 30 18:53:44.882037 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:44.882019    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d33b7312-dc19-402a-9c2e-12e0b7eee906-istio-data\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-ngqb5\" (UID: \"d33b7312-dc19-402a-9c2e-12e0b7eee906\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-ngqb5"
Oct 30 18:53:44.983680 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:44.983646    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d33b7312-dc19-402a-9c2e-12e0b7eee906-istio-podinfo\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-ngqb5\" (UID: \"d33b7312-dc19-402a-9c2e-12e0b7eee906\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-ngqb5"
Oct 30 18:53:44.983868 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:44.983731    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d33b7312-dc19-402a-9c2e-12e0b7eee906-workload-certs\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-ngqb5\" (UID: \"d33b7312-dc19-402a-9c2e-12e0b7eee906\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-ngqb5"
Oct 30 18:53:44.983868 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:44.983748    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d33b7312-dc19-402a-9c2e-12e0b7eee906-istio-envoy\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-ngqb5\" (UID: \"d33b7312-dc19-402a-9c2e-12e0b7eee906\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-ngqb5"
Oct 30 18:53:44.983868 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:44.983767    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d33b7312-dc19-402a-9c2e-12e0b7eee906-workload-socket\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-ngqb5\" (UID: \"d33b7312-dc19-402a-9c2e-12e0b7eee906\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-ngqb5"
Oct 30 18:53:44.983868 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:44.983821    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d33b7312-dc19-402a-9c2e-12e0b7eee906-istiod-ca-cert\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-ngqb5\" (UID: \"d33b7312-dc19-402a-9c2e-12e0b7eee906\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-ngqb5"
Oct 30 18:53:44.983868 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:44.983861    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d33b7312-dc19-402a-9c2e-12e0b7eee906-credential-socket\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-ngqb5\" (UID: \"d33b7312-dc19-402a-9c2e-12e0b7eee906\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-ngqb5"
Oct 30 18:53:44.984127 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:44.983935    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vdt4h\" (UniqueName: \"kubernetes.io/projected/d33b7312-dc19-402a-9c2e-12e0b7eee906-kube-api-access-vdt4h\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-ngqb5\" (UID: \"d33b7312-dc19-402a-9c2e-12e0b7eee906\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-ngqb5"
Oct 30 18:53:44.984127 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:44.983972    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d33b7312-dc19-402a-9c2e-12e0b7eee906-istio-token\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-ngqb5\" (UID: \"d33b7312-dc19-402a-9c2e-12e0b7eee906\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-ngqb5"
Oct 30 18:53:44.984127 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:44.983998    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d33b7312-dc19-402a-9c2e-12e0b7eee906-istio-data\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-ngqb5\" (UID: \"d33b7312-dc19-402a-9c2e-12e0b7eee906\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-ngqb5"
Oct 30 18:53:44.984340 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:44.984314    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d33b7312-dc19-402a-9c2e-12e0b7eee906-workload-certs\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-ngqb5\" (UID: \"d33b7312-dc19-402a-9c2e-12e0b7eee906\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-ngqb5"
Oct 30 18:53:44.984437 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:44.984322    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d33b7312-dc19-402a-9c2e-12e0b7eee906-credential-socket\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-ngqb5\" (UID: \"d33b7312-dc19-402a-9c2e-12e0b7eee906\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-ngqb5"
Oct 30 18:53:44.984437 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:44.984368    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d33b7312-dc19-402a-9c2e-12e0b7eee906-istio-data\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-ngqb5\" (UID: \"d33b7312-dc19-402a-9c2e-12e0b7eee906\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-ngqb5"
Oct 30 18:53:44.984850 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:44.984827    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d33b7312-dc19-402a-9c2e-12e0b7eee906-workload-socket\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-ngqb5\" (UID: \"d33b7312-dc19-402a-9c2e-12e0b7eee906\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-ngqb5"
Oct 30 18:53:44.985151 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:44.985128    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d33b7312-dc19-402a-9c2e-12e0b7eee906-istiod-ca-cert\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-ngqb5\" (UID: \"d33b7312-dc19-402a-9c2e-12e0b7eee906\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-ngqb5"
Oct 30 18:53:44.986599 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:44.986573    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d33b7312-dc19-402a-9c2e-12e0b7eee906-istio-envoy\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-ngqb5\" (UID: \"d33b7312-dc19-402a-9c2e-12e0b7eee906\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-ngqb5"
Oct 30 18:53:44.986700 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:44.986612    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d33b7312-dc19-402a-9c2e-12e0b7eee906-istio-podinfo\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-ngqb5\" (UID: \"d33b7312-dc19-402a-9c2e-12e0b7eee906\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-ngqb5"
Oct 30 18:53:45.047921 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:45.047878    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d33b7312-dc19-402a-9c2e-12e0b7eee906-istio-token\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-ngqb5\" (UID: \"d33b7312-dc19-402a-9c2e-12e0b7eee906\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-ngqb5"
Oct 30 18:53:45.048098 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:45.047966    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vdt4h\" (UniqueName: \"kubernetes.io/projected/d33b7312-dc19-402a-9c2e-12e0b7eee906-kube-api-access-vdt4h\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-ngqb5\" (UID: \"d33b7312-dc19-402a-9c2e-12e0b7eee906\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-ngqb5"
Oct 30 18:53:45.124228 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:45.124199    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-ngqb5"
Oct 30 18:53:45.164101 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:45.164063    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-rjmdz"]
Oct 30 18:53:45.574927 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:45.574896    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-ngqb5"]
Oct 30 18:53:45.578926 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:53:45.578878    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd33b7312_dc19_402a_9c2e_12e0b7eee906.slice/crio-04ac7af79bbf2806ade7e2746c66ab428c55a06cd254f8c3a2d19cbd262cb97f WatchSource:0}: Error finding container 04ac7af79bbf2806ade7e2746c66ab428c55a06cd254f8c3a2d19cbd262cb97f: Status 404 returned error can't find the container with id 04ac7af79bbf2806ade7e2746c66ab428c55a06cd254f8c3a2d19cbd262cb97f
Oct 30 18:53:45.581735 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:45.581700    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:53:45.581835 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:45.581773    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:53:45.581835 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:45.581802    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:53:46.429311 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:46.408650    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-ngqb5" event={"ID":"d33b7312-dc19-402a-9c2e-12e0b7eee906","Type":"ContainerStarted","Data":"d0fa2b0cac5d3e4044a8a2d1a39fa843e6d134086a59257db9c99b6cb7c6619f"}
Oct 30 18:53:46.429987 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:46.429925    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-ngqb5" event={"ID":"d33b7312-dc19-402a-9c2e-12e0b7eee906","Type":"ContainerStarted","Data":"04ac7af79bbf2806ade7e2746c66ab428c55a06cd254f8c3a2d19cbd262cb97f"}
Oct 30 18:53:46.660313 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:46.659919    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-ngqb5" podStartSLOduration=2.6599001920000003 podStartE2EDuration="2.659900192s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:53:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:53:46.60280113 +0000 UTC m=+1190.850207155" watchObservedRunningTime="2026-10-30 18:53:46.659900192 +0000 UTC m=+1190.907306221"
Oct 30 18:53:47.136311 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:47.132609    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-ngqb5"
Oct 30 18:53:47.173306 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:47.140400    4888 patch_prober.go:29] interesting pod/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-ngqb5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused" start-of-body=
Oct 30 18:53:47.182328 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:47.173487    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-ngqb5" podUID="d33b7312-dc19-402a-9c2e-12e0b7eee906" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused"
Oct 30 18:53:48.052743 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:48.050886    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-3b6013/premium-w3-3b6013-kserve-74df5fb95d-k747z" podUID="a9309776-20d5-4c06-bcae-04b8e54493e8" containerName="main" probeResult="failure" output="Get \"https://10.132.0.82:8000/ready\": context deadline exceeded"
Oct 30 18:53:48.052743 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:48.052361    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-rkm4t" podUID="1ab20f74-2c73-4d56-a604-aad4f8289450" containerName="main" probeResult="failure" output="Get \"https://10.132.0.53:8000/ready\": context deadline exceeded"
Oct 30 18:53:48.174223 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:48.174191    4888 patch_prober.go:29] interesting pod/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-ngqb5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused" start-of-body=
Oct 30 18:53:48.174529 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:48.174486    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-ngqb5" podUID="d33b7312-dc19-402a-9c2e-12e0b7eee906" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused"
Oct 30 18:53:48.219423 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:48.219379    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e7929d/distinct-2-w4-e7929d-kserve-5b479cc87b-htv9p" podUID="f6df655d-86e8-44ff-ace8-27440f56294f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.91:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:53:48.293021 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:48.292978    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-4a7a52/premium-w2-4a7a52-kserve-746d9579f6-hn7hl" podUID="3160b54f-1df9-48de-80d9-8d8a91d47f56" containerName="main" probeResult="failure" output="Get \"https://10.132.0.83:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:53:48.409306 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:48.387304    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-3b6013/distinct-w3-3b6013-kserve-857f55d885-8fcrq" podUID="51927877-4d99-4797-82f7-1684f6ecda6b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.92:8000/ready\": context deadline exceeded"
Oct 30 18:53:49.125049 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:49.125013    4888 patch_prober.go:29] interesting pod/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-ngqb5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused" start-of-body=
Oct 30 18:53:49.125487 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:49.125065    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-ngqb5" podUID="d33b7312-dc19-402a-9c2e-12e0b7eee906" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused"
Oct 30 18:53:50.124737 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:50.124697    4888 patch_prober.go:29] interesting pod/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-ngqb5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused" start-of-body=
Oct 30 18:53:50.124911 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:50.124767    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-ngqb5" podUID="d33b7312-dc19-402a-9c2e-12e0b7eee906" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused"
Oct 30 18:53:51.125366 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:51.125332    4888 patch_prober.go:29] interesting pod/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-ngqb5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused" start-of-body=
Oct 30 18:53:51.125743 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:51.125386    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-ngqb5" podUID="d33b7312-dc19-402a-9c2e-12e0b7eee906" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused"
Oct 30 18:53:52.124950 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:52.124915    4888 patch_prober.go:29] interesting pod/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-ngqb5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused" start-of-body=
Oct 30 18:53:52.125219 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:52.124969    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-ngqb5" podUID="d33b7312-dc19-402a-9c2e-12e0b7eee906" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused"
Oct 30 18:53:53.125493 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:53.125450    4888 patch_prober.go:29] interesting pod/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-ngqb5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused" start-of-body=
Oct 30 18:53:53.126035 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:53.125517    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-ngqb5" podUID="d33b7312-dc19-402a-9c2e-12e0b7eee906" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused"
Oct 30 18:53:53.624950 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:53.624908    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w3-3b6013/distinct-2-w3-3b6013-kserve-7cc5454db-mqknj"]
Oct 30 18:53:53.630481 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:53.630452    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-3b6013/distinct-2-w3-3b6013-kserve-7cc5454db-mqknj"
Oct 30 18:53:53.730694 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:53.730657    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-3b6013/distinct-2-w3-3b6013-kserve-7cc5454db-mqknj"]
Oct 30 18:53:53.736732 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:53.736704    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-3b6013\"/\"distinct-2-w3-3b6013-kserve-self-signed-certs\""
Oct 30 18:53:53.789026 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:53.788972    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e76c7b20-be2f-4ed2-8dc4-bef127ecc0c9-model-cache\") pod \"distinct-2-w3-3b6013-kserve-7cc5454db-mqknj\" (UID: \"e76c7b20-be2f-4ed2-8dc4-bef127ecc0c9\") " pod="e2e-models-e2e-worker-w3-3b6013/distinct-2-w3-3b6013-kserve-7cc5454db-mqknj"
Oct 30 18:53:53.794355 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:53.794307    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e76c7b20-be2f-4ed2-8dc4-bef127ecc0c9-tmp-dir\") pod \"distinct-2-w3-3b6013-kserve-7cc5454db-mqknj\" (UID: \"e76c7b20-be2f-4ed2-8dc4-bef127ecc0c9\") " pod="e2e-models-e2e-worker-w3-3b6013/distinct-2-w3-3b6013-kserve-7cc5454db-mqknj"
Oct 30 18:53:53.794965 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:53.794939    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e76c7b20-be2f-4ed2-8dc4-bef127ecc0c9-home\") pod \"distinct-2-w3-3b6013-kserve-7cc5454db-mqknj\" (UID: \"e76c7b20-be2f-4ed2-8dc4-bef127ecc0c9\") " pod="e2e-models-e2e-worker-w3-3b6013/distinct-2-w3-3b6013-kserve-7cc5454db-mqknj"
Oct 30 18:53:53.795342 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:53.795323    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e76c7b20-be2f-4ed2-8dc4-bef127ecc0c9-dshm\") pod \"distinct-2-w3-3b6013-kserve-7cc5454db-mqknj\" (UID: \"e76c7b20-be2f-4ed2-8dc4-bef127ecc0c9\") " pod="e2e-models-e2e-worker-w3-3b6013/distinct-2-w3-3b6013-kserve-7cc5454db-mqknj"
Oct 30 18:53:53.796032 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:53.796011    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h22l6\" (UniqueName: \"kubernetes.io/projected/e76c7b20-be2f-4ed2-8dc4-bef127ecc0c9-kube-api-access-h22l6\") pod \"distinct-2-w3-3b6013-kserve-7cc5454db-mqknj\" (UID: \"e76c7b20-be2f-4ed2-8dc4-bef127ecc0c9\") " pod="e2e-models-e2e-worker-w3-3b6013/distinct-2-w3-3b6013-kserve-7cc5454db-mqknj"
Oct 30 18:53:53.796599 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:53.796580    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e76c7b20-be2f-4ed2-8dc4-bef127ecc0c9-tls-certs\") pod \"distinct-2-w3-3b6013-kserve-7cc5454db-mqknj\" (UID: \"e76c7b20-be2f-4ed2-8dc4-bef127ecc0c9\") " pod="e2e-models-e2e-worker-w3-3b6013/distinct-2-w3-3b6013-kserve-7cc5454db-mqknj"
Oct 30 18:53:53.898915 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:53.897896    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e76c7b20-be2f-4ed2-8dc4-bef127ecc0c9-tls-certs\") pod \"distinct-2-w3-3b6013-kserve-7cc5454db-mqknj\" (UID: \"e76c7b20-be2f-4ed2-8dc4-bef127ecc0c9\") " pod="e2e-models-e2e-worker-w3-3b6013/distinct-2-w3-3b6013-kserve-7cc5454db-mqknj"
Oct 30 18:53:53.899969 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:53.899943    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e76c7b20-be2f-4ed2-8dc4-bef127ecc0c9-model-cache\") pod \"distinct-2-w3-3b6013-kserve-7cc5454db-mqknj\" (UID: \"e76c7b20-be2f-4ed2-8dc4-bef127ecc0c9\") " pod="e2e-models-e2e-worker-w3-3b6013/distinct-2-w3-3b6013-kserve-7cc5454db-mqknj"
Oct 30 18:53:53.900627 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:53.900544    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e76c7b20-be2f-4ed2-8dc4-bef127ecc0c9-model-cache\") pod \"distinct-2-w3-3b6013-kserve-7cc5454db-mqknj\" (UID: \"e76c7b20-be2f-4ed2-8dc4-bef127ecc0c9\") " pod="e2e-models-e2e-worker-w3-3b6013/distinct-2-w3-3b6013-kserve-7cc5454db-mqknj"
Oct 30 18:53:53.900901 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:53.900886    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e76c7b20-be2f-4ed2-8dc4-bef127ecc0c9-tmp-dir\") pod \"distinct-2-w3-3b6013-kserve-7cc5454db-mqknj\" (UID: \"e76c7b20-be2f-4ed2-8dc4-bef127ecc0c9\") " pod="e2e-models-e2e-worker-w3-3b6013/distinct-2-w3-3b6013-kserve-7cc5454db-mqknj"
Oct 30 18:53:53.901386 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:53.901371    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e76c7b20-be2f-4ed2-8dc4-bef127ecc0c9-home\") pod \"distinct-2-w3-3b6013-kserve-7cc5454db-mqknj\" (UID: \"e76c7b20-be2f-4ed2-8dc4-bef127ecc0c9\") " pod="e2e-models-e2e-worker-w3-3b6013/distinct-2-w3-3b6013-kserve-7cc5454db-mqknj"
Oct 30 18:53:53.901852 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:53.901320    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e76c7b20-be2f-4ed2-8dc4-bef127ecc0c9-tmp-dir\") pod \"distinct-2-w3-3b6013-kserve-7cc5454db-mqknj\" (UID: \"e76c7b20-be2f-4ed2-8dc4-bef127ecc0c9\") " pod="e2e-models-e2e-worker-w3-3b6013/distinct-2-w3-3b6013-kserve-7cc5454db-mqknj"
Oct 30 18:53:53.901969 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:53.901761    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e76c7b20-be2f-4ed2-8dc4-bef127ecc0c9-home\") pod \"distinct-2-w3-3b6013-kserve-7cc5454db-mqknj\" (UID: \"e76c7b20-be2f-4ed2-8dc4-bef127ecc0c9\") " pod="e2e-models-e2e-worker-w3-3b6013/distinct-2-w3-3b6013-kserve-7cc5454db-mqknj"
Oct 30 18:53:53.902582 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:53.902564    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e76c7b20-be2f-4ed2-8dc4-bef127ecc0c9-dshm\") pod \"distinct-2-w3-3b6013-kserve-7cc5454db-mqknj\" (UID: \"e76c7b20-be2f-4ed2-8dc4-bef127ecc0c9\") " pod="e2e-models-e2e-worker-w3-3b6013/distinct-2-w3-3b6013-kserve-7cc5454db-mqknj"
Oct 30 18:53:53.903071 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:53.902792    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h22l6\" (UniqueName: \"kubernetes.io/projected/e76c7b20-be2f-4ed2-8dc4-bef127ecc0c9-kube-api-access-h22l6\") pod \"distinct-2-w3-3b6013-kserve-7cc5454db-mqknj\" (UID: \"e76c7b20-be2f-4ed2-8dc4-bef127ecc0c9\") " pod="e2e-models-e2e-worker-w3-3b6013/distinct-2-w3-3b6013-kserve-7cc5454db-mqknj"
Oct 30 18:53:53.905572 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:53.905551    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e76c7b20-be2f-4ed2-8dc4-bef127ecc0c9-tls-certs\") pod \"distinct-2-w3-3b6013-kserve-7cc5454db-mqknj\" (UID: \"e76c7b20-be2f-4ed2-8dc4-bef127ecc0c9\") " pod="e2e-models-e2e-worker-w3-3b6013/distinct-2-w3-3b6013-kserve-7cc5454db-mqknj"
Oct 30 18:53:53.915070 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:53.915051    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e76c7b20-be2f-4ed2-8dc4-bef127ecc0c9-dshm\") pod \"distinct-2-w3-3b6013-kserve-7cc5454db-mqknj\" (UID: \"e76c7b20-be2f-4ed2-8dc4-bef127ecc0c9\") " pod="e2e-models-e2e-worker-w3-3b6013/distinct-2-w3-3b6013-kserve-7cc5454db-mqknj"
Oct 30 18:53:53.966566 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:53.966534    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h22l6\" (UniqueName: \"kubernetes.io/projected/e76c7b20-be2f-4ed2-8dc4-bef127ecc0c9-kube-api-access-h22l6\") pod \"distinct-2-w3-3b6013-kserve-7cc5454db-mqknj\" (UID: \"e76c7b20-be2f-4ed2-8dc4-bef127ecc0c9\") " pod="e2e-models-e2e-worker-w3-3b6013/distinct-2-w3-3b6013-kserve-7cc5454db-mqknj"
Oct 30 18:53:54.193634 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:54.193533    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-ngqb5"
Oct 30 18:53:54.194416 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:54.194391    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-ngqb5"
Oct 30 18:53:54.195901 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:54.195878    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-ngqb5"
Oct 30 18:53:54.246950 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:54.246912    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-3b6013/distinct-2-w3-3b6013-kserve-7cc5454db-mqknj"
Oct 30 18:53:54.638657 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:54.638634    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-3b6013/distinct-2-w3-3b6013-kserve-7cc5454db-mqknj"]
Oct 30 18:53:54.639155 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:53:54.639125    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode76c7b20_be2f_4ed2_8dc4_bef127ecc0c9.slice/crio-53f445b7941fb221dfa77916e51c718be86985aff233d4c90f5fe6b43a29a282 WatchSource:0}: Error finding container 53f445b7941fb221dfa77916e51c718be86985aff233d4c90f5fe6b43a29a282: Status 404 returned error can't find the container with id 53f445b7941fb221dfa77916e51c718be86985aff233d4c90f5fe6b43a29a282
Oct 30 18:53:55.586559 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:55.586523    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-3b6013/distinct-2-w3-3b6013-kserve-7cc5454db-mqknj" event={"ID":"e76c7b20-be2f-4ed2-8dc4-bef127ecc0c9","Type":"ContainerStarted","Data":"539d053fe28c0a83e4a27cf2c3b304bc6df03c142a6f0cfbf7d6be44c8d708d4"}
Oct 30 18:53:55.587165 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:55.586569    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-3b6013/distinct-2-w3-3b6013-kserve-7cc5454db-mqknj" event={"ID":"e76c7b20-be2f-4ed2-8dc4-bef127ecc0c9","Type":"ContainerStarted","Data":"53f445b7941fb221dfa77916e51c718be86985aff233d4c90f5fe6b43a29a282"}
Oct 30 18:53:55.706332 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:55.706255    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w3-3b6013/distinct-2-w3-3b6013-kserve-7cc5454db-mqknj" podStartSLOduration=2.706237678 podStartE2EDuration="2.706237678s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:53:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:53:55.70499945 +0000 UTC m=+1199.952405475" watchObservedRunningTime="2026-10-30 18:53:55.706237678 +0000 UTC m=+1199.953643718"
Oct 30 18:53:56.251207 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:56.251171    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-3b6013/distinct-2-w3-3b6013-kserve-7cc5454db-mqknj"
Oct 30 18:53:56.264649 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:56.264626    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-7x4rb_382e8b31-a7fa-43f3-9f15-4c61627b0c04/console-operator/2.log"
Oct 30 18:53:56.269448 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:56.269428    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-7x4rb_382e8b31-a7fa-43f3-9f15-4c61627b0c04/console-operator/2.log"
Oct 30 18:53:56.272424 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:56.272406    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xfcdl_3f3fe764-2652-46c9-8e54-7882c0e6bcf1/ovn-acl-logging/0.log"
Oct 30 18:53:56.276953 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:53:56.276935    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xfcdl_3f3fe764-2652-46c9-8e54-7882c0e6bcf1/ovn-acl-logging/0.log"
Oct 30 18:54:04.248073 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:04.248040    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w3-3b6013/distinct-2-w3-3b6013-kserve-7cc5454db-mqknj"
Oct 30 18:54:04.260527 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:04.260503    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w3-3b6013/distinct-2-w3-3b6013-kserve-7cc5454db-mqknj"
Oct 30 18:54:04.655664 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:04.655633    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-3b6013/distinct-2-w3-3b6013-kserve-7cc5454db-mqknj"
Oct 30 18:54:05.999545 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:54:05.999457    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-jbsw6" containerName="extractor"
Oct 30 18:54:06.692308 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:06.691401    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-3b6013/distinct-2-w3-3b6013-kserve-7cc5454db-mqknj" podUID="e76c7b20-be2f-4ed2-8dc4-bef127ecc0c9" containerName="main" probeResult="failure" output="Get \"https://10.132.0.98:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:54:06.989370 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:06.973400    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-3b6013/premium-w3-3b6013-kserve-74df5fb95d-k747z" podUID="a9309776-20d5-4c06-bcae-04b8e54493e8" containerName="main" probeResult="failure" output="Get \"https://10.132.0.82:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:54:07.007310 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:06.995421    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-rkm4t" podUID="1ab20f74-2c73-4d56-a604-aad4f8289450" containerName="main" probeResult="failure" output="Get \"https://10.132.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:54:07.129310 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:07.128394    4888 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-e7929d/distinct-w4-e7929d-kserve-5bc9ff5bf5-kmrwv" podUID="abfd315d-a01a-42dd-9c06-2a80d3f8579a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.88:8000/health\": context deadline exceeded"
Oct 30 18:54:07.224306 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:07.223426    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e7929d/distinct-2-w4-e7929d-kserve-5b479cc87b-htv9p" podUID="f6df655d-86e8-44ff-ace8-27440f56294f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.91:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:54:07.285365 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:07.284869    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-4a7a52/premium-w2-4a7a52-kserve-746d9579f6-hn7hl" podUID="3160b54f-1df9-48de-80d9-8d8a91d47f56" containerName="main" probeResult="failure" output="Get \"https://10.132.0.83:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:54:07.650600 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:07.650558    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e7929d/distinct-w4-e7929d-kserve-5bc9ff5bf5-kmrwv" podUID="abfd315d-a01a-42dd-9c06-2a80d3f8579a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.88:8000/ready\": context deadline exceeded"
Oct 30 18:54:07.977017 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:07.974688    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-3b6013/premium-w3-3b6013-kserve-74df5fb95d-k747z" podUID="a9309776-20d5-4c06-bcae-04b8e54493e8" containerName="main" probeResult="failure" output="Get \"https://10.132.0.82:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:54:07.977017 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:07.975487    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-3b6013/premium-w3-3b6013-kserve-74df5fb95d-k747z"
Oct 30 18:54:07.998687 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:07.995619    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-rkm4t" podUID="1ab20f74-2c73-4d56-a604-aad4f8289450" containerName="main" probeResult="failure" output="Get \"https://10.132.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:54:07.998687 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:07.996020    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-rkm4t"
Oct 30 18:54:08.115041 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:08.109852    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-3b6013/premium-w3-3b6013-kserve-74df5fb95d-k747z"
Oct 30 18:54:08.176497 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:08.176400    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-rkm4t"
Oct 30 18:54:15.084432 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:15.084392    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4xpmd"]
Oct 30 18:54:15.088636 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:15.088609    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4xpmd"
Oct 30 18:54:15.203619 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:15.203590    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a6c278cc-4cc3-4584-b83e-376f5af90380-workload-certs\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4xpmd\" (UID: \"a6c278cc-4cc3-4584-b83e-376f5af90380\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4xpmd"
Oct 30 18:54:15.203818 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:15.203630    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a6c278cc-4cc3-4584-b83e-376f5af90380-credential-socket\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4xpmd\" (UID: \"a6c278cc-4cc3-4584-b83e-376f5af90380\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4xpmd"
Oct 30 18:54:15.203818 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:15.203650    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a6c278cc-4cc3-4584-b83e-376f5af90380-istio-data\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4xpmd\" (UID: \"a6c278cc-4cc3-4584-b83e-376f5af90380\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4xpmd"
Oct 30 18:54:15.203953 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:15.203834    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p5wb8\" (UniqueName: \"kubernetes.io/projected/a6c278cc-4cc3-4584-b83e-376f5af90380-kube-api-access-p5wb8\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4xpmd\" (UID: \"a6c278cc-4cc3-4584-b83e-376f5af90380\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4xpmd"
Oct 30 18:54:15.203953 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:15.203916    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a6c278cc-4cc3-4584-b83e-376f5af90380-istio-token\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4xpmd\" (UID: \"a6c278cc-4cc3-4584-b83e-376f5af90380\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4xpmd"
Oct 30 18:54:15.204060 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:15.203985    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a6c278cc-4cc3-4584-b83e-376f5af90380-workload-socket\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4xpmd\" (UID: \"a6c278cc-4cc3-4584-b83e-376f5af90380\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4xpmd"
Oct 30 18:54:15.204060 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:15.204019    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a6c278cc-4cc3-4584-b83e-376f5af90380-istiod-ca-cert\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4xpmd\" (UID: \"a6c278cc-4cc3-4584-b83e-376f5af90380\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4xpmd"
Oct 30 18:54:15.204166 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:15.204143    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a6c278cc-4cc3-4584-b83e-376f5af90380-istio-envoy\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4xpmd\" (UID: \"a6c278cc-4cc3-4584-b83e-376f5af90380\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4xpmd"
Oct 30 18:54:15.204220 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:15.204179    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a6c278cc-4cc3-4584-b83e-376f5af90380-istio-podinfo\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4xpmd\" (UID: \"a6c278cc-4cc3-4584-b83e-376f5af90380\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4xpmd"
Oct 30 18:54:15.212210 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:15.212183    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4xpmd"]
Oct 30 18:54:15.305794 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:15.305766    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a6c278cc-4cc3-4584-b83e-376f5af90380-istio-envoy\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4xpmd\" (UID: \"a6c278cc-4cc3-4584-b83e-376f5af90380\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4xpmd"
Oct 30 18:54:15.305995 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:15.305803    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a6c278cc-4cc3-4584-b83e-376f5af90380-istio-podinfo\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4xpmd\" (UID: \"a6c278cc-4cc3-4584-b83e-376f5af90380\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4xpmd"
Oct 30 18:54:15.305995 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:15.305861    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a6c278cc-4cc3-4584-b83e-376f5af90380-workload-certs\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4xpmd\" (UID: \"a6c278cc-4cc3-4584-b83e-376f5af90380\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4xpmd"
Oct 30 18:54:15.305995 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:15.305899    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a6c278cc-4cc3-4584-b83e-376f5af90380-credential-socket\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4xpmd\" (UID: \"a6c278cc-4cc3-4584-b83e-376f5af90380\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4xpmd"
Oct 30 18:54:15.305995 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:15.305922    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a6c278cc-4cc3-4584-b83e-376f5af90380-istio-data\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4xpmd\" (UID: \"a6c278cc-4cc3-4584-b83e-376f5af90380\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4xpmd"
Oct 30 18:54:15.306328 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:15.306015    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p5wb8\" (UniqueName: \"kubernetes.io/projected/a6c278cc-4cc3-4584-b83e-376f5af90380-kube-api-access-p5wb8\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4xpmd\" (UID: \"a6c278cc-4cc3-4584-b83e-376f5af90380\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4xpmd"
Oct 30 18:54:15.306328 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:15.306076    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a6c278cc-4cc3-4584-b83e-376f5af90380-istio-token\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4xpmd\" (UID: \"a6c278cc-4cc3-4584-b83e-376f5af90380\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4xpmd"
Oct 30 18:54:15.306328 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:15.306138    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a6c278cc-4cc3-4584-b83e-376f5af90380-workload-socket\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4xpmd\" (UID: \"a6c278cc-4cc3-4584-b83e-376f5af90380\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4xpmd"
Oct 30 18:54:15.306328 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:15.306170    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a6c278cc-4cc3-4584-b83e-376f5af90380-istiod-ca-cert\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4xpmd\" (UID: \"a6c278cc-4cc3-4584-b83e-376f5af90380\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4xpmd"
Oct 30 18:54:15.306328 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:15.306294    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a6c278cc-4cc3-4584-b83e-376f5af90380-credential-socket\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4xpmd\" (UID: \"a6c278cc-4cc3-4584-b83e-376f5af90380\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4xpmd"
Oct 30 18:54:15.307312 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:15.306610    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a6c278cc-4cc3-4584-b83e-376f5af90380-workload-socket\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4xpmd\" (UID: \"a6c278cc-4cc3-4584-b83e-376f5af90380\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4xpmd"
Oct 30 18:54:15.307312 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:15.306857    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a6c278cc-4cc3-4584-b83e-376f5af90380-istio-data\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4xpmd\" (UID: \"a6c278cc-4cc3-4584-b83e-376f5af90380\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4xpmd"
Oct 30 18:54:15.307312 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:15.306938    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a6c278cc-4cc3-4584-b83e-376f5af90380-workload-certs\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4xpmd\" (UID: \"a6c278cc-4cc3-4584-b83e-376f5af90380\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4xpmd"
Oct 30 18:54:15.307539 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:15.307514    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a6c278cc-4cc3-4584-b83e-376f5af90380-istiod-ca-cert\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4xpmd\" (UID: \"a6c278cc-4cc3-4584-b83e-376f5af90380\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4xpmd"
Oct 30 18:54:15.308803 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:15.308778    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a6c278cc-4cc3-4584-b83e-376f5af90380-istio-envoy\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4xpmd\" (UID: \"a6c278cc-4cc3-4584-b83e-376f5af90380\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4xpmd"
Oct 30 18:54:15.308909 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:15.308865    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a6c278cc-4cc3-4584-b83e-376f5af90380-istio-podinfo\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4xpmd\" (UID: \"a6c278cc-4cc3-4584-b83e-376f5af90380\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4xpmd"
Oct 30 18:54:15.369066 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:15.368992    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a6c278cc-4cc3-4584-b83e-376f5af90380-istio-token\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4xpmd\" (UID: \"a6c278cc-4cc3-4584-b83e-376f5af90380\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4xpmd"
Oct 30 18:54:15.369066 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:15.369052    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p5wb8\" (UniqueName: \"kubernetes.io/projected/a6c278cc-4cc3-4584-b83e-376f5af90380-kube-api-access-p5wb8\") pod \"e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4xpmd\" (UID: \"a6c278cc-4cc3-4584-b83e-376f5af90380\") " pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4xpmd"
Oct 30 18:54:15.403024 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:15.402993    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4xpmd"
Oct 30 18:54:15.837935 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:15.837894    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4xpmd"]
Oct 30 18:54:15.838326 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:54:15.838298    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda6c278cc_4cc3_4584_b83e_376f5af90380.slice/crio-b163fe81cccc74dfb72891104e1bba7e5b453d1f2966fc752d7f2e9408901162 WatchSource:0}: Error finding container b163fe81cccc74dfb72891104e1bba7e5b453d1f2966fc752d7f2e9408901162: Status 404 returned error can't find the container with id b163fe81cccc74dfb72891104e1bba7e5b453d1f2966fc752d7f2e9408901162
Oct 30 18:54:15.840767 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:15.840740    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:54:15.840843 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:15.840805    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:54:15.840883 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:15.840853    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:54:16.854132 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:16.854098    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4xpmd" event={"ID":"a6c278cc-4cc3-4584-b83e-376f5af90380","Type":"ContainerStarted","Data":"7db3612372458c749c6ad16c7a0326ecb1f9259423087b1413072520e71655c7"}
Oct 30 18:54:16.854665 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:16.854160    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4xpmd" event={"ID":"a6c278cc-4cc3-4584-b83e-376f5af90380","Type":"ContainerStarted","Data":"b163fe81cccc74dfb72891104e1bba7e5b453d1f2966fc752d7f2e9408901162"}
Oct 30 18:54:17.027574 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:17.027512    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4xpmd" podStartSLOduration=2.02749469 podStartE2EDuration="2.02749469s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:54:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:54:17.027479427 +0000 UTC m=+1221.274885454" watchObservedRunningTime="2026-10-30 18:54:17.02749469 +0000 UTC m=+1221.274900716"
Oct 30 18:54:17.403331 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:17.403276    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4xpmd"
Oct 30 18:54:17.405082 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:17.405044    4888 patch_prober.go:29] interesting pod/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4xpmd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.99:15021/healthz/ready\": dial tcp 10.132.0.99:15021: connect: connection refused" start-of-body=
Oct 30 18:54:17.405226 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:17.405119    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4xpmd" podUID="a6c278cc-4cc3-4584-b83e-376f5af90380" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.99:15021/healthz/ready\": dial tcp 10.132.0.99:15021: connect: connection refused"
Oct 30 18:54:18.403685 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:18.403647    4888 patch_prober.go:29] interesting pod/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4xpmd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.99:15021/healthz/ready\": dial tcp 10.132.0.99:15021: connect: connection refused" start-of-body=
Oct 30 18:54:18.404084 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:18.403711    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4xpmd" podUID="a6c278cc-4cc3-4584-b83e-376f5af90380" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.99:15021/healthz/ready\": dial tcp 10.132.0.99:15021: connect: connection refused"
Oct 30 18:54:19.403816 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:19.403782    4888 patch_prober.go:29] interesting pod/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4xpmd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.99:15021/healthz/ready\": dial tcp 10.132.0.99:15021: connect: connection refused" start-of-body=
Oct 30 18:54:19.404321 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:19.403852    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4xpmd" podUID="a6c278cc-4cc3-4584-b83e-376f5af90380" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.99:15021/healthz/ready\": dial tcp 10.132.0.99:15021: connect: connection refused"
Oct 30 18:54:20.403543 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:20.403506    4888 patch_prober.go:29] interesting pod/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4xpmd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.99:15021/healthz/ready\": dial tcp 10.132.0.99:15021: connect: connection refused" start-of-body=
Oct 30 18:54:20.403756 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:20.403589    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4xpmd" podUID="a6c278cc-4cc3-4584-b83e-376f5af90380" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.99:15021/healthz/ready\": dial tcp 10.132.0.99:15021: connect: connection refused"
Oct 30 18:54:21.305468 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:21.305433    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-rjmdz"]
Oct 30 18:54:21.404077 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:21.404046    4888 patch_prober.go:29] interesting pod/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4xpmd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.99:15021/healthz/ready\": dial tcp 10.132.0.99:15021: connect: connection refused" start-of-body=
Oct 30 18:54:21.404256 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:21.404093    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4xpmd" podUID="a6c278cc-4cc3-4584-b83e-376f5af90380" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.99:15021/healthz/ready\": dial tcp 10.132.0.99:15021: connect: connection refused"
Oct 30 18:54:22.406308 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:22.405840    4888 patch_prober.go:29] interesting pod/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4xpmd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.99:15021/healthz/ready\": dial tcp 10.132.0.99:15021: connect: connection refused" start-of-body=
Oct 30 18:54:22.406308 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:22.405896    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4xpmd" podUID="a6c278cc-4cc3-4584-b83e-376f5af90380" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.99:15021/healthz/ready\": dial tcp 10.132.0.99:15021: connect: connection refused"
Oct 30 18:54:23.654411 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:23.654369    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e7929d/distinct-w4-e7929d-kserve-5bc9ff5bf5-kmrwv" podUID="abfd315d-a01a-42dd-9c06-2a80d3f8579a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.88:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:54:24.417463 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:24.417417    4888 patch_prober.go:29] interesting pod/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4xpmd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.99:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 18:54:24.418165 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:24.418117    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4xpmd" podUID="a6c278cc-4cc3-4584-b83e-376f5af90380" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.99:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:54:25.459449 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:25.459409    4888 patch_prober.go:29] interesting pod/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4xpmd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.99:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 18:54:25.459844 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:25.459478    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4xpmd" podUID="a6c278cc-4cc3-4584-b83e-376f5af90380" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.99:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:54:25.824001 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:25.823972    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4xpmd"
Oct 30 18:54:25.824909 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:25.824887    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4xpmd"
Oct 30 18:54:25.825756 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:25.825740    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4xpmd"
Oct 30 18:54:35.620691 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:54:35.620656    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-jbsw6" containerName="extractor"
Oct 30 18:54:37.407523 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:37.407493    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-9k7xv"]
Oct 30 18:54:37.413116 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:37.413093    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-9k7xv"
Oct 30 18:54:37.504812 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:37.504779    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-66baf45c-openshift-default-dockercfg-sqvln\""
Oct 30 18:54:37.532466 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:37.532425    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/427c5128-a124-4dd6-9748-a4dca0a85a4f-istio-podinfo\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-9k7xv\" (UID: \"427c5128-a124-4dd6-9748-a4dca0a85a4f\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-9k7xv"
Oct 30 18:54:37.532774 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:37.532752    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/427c5128-a124-4dd6-9748-a4dca0a85a4f-istio-token\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-9k7xv\" (UID: \"427c5128-a124-4dd6-9748-a4dca0a85a4f\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-9k7xv"
Oct 30 18:54:37.532939 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:37.532923    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/427c5128-a124-4dd6-9748-a4dca0a85a4f-workload-socket\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-9k7xv\" (UID: \"427c5128-a124-4dd6-9748-a4dca0a85a4f\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-9k7xv"
Oct 30 18:54:37.533039 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:37.533026    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/427c5128-a124-4dd6-9748-a4dca0a85a4f-workload-certs\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-9k7xv\" (UID: \"427c5128-a124-4dd6-9748-a4dca0a85a4f\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-9k7xv"
Oct 30 18:54:37.533137 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:37.533124    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/427c5128-a124-4dd6-9748-a4dca0a85a4f-istiod-ca-cert\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-9k7xv\" (UID: \"427c5128-a124-4dd6-9748-a4dca0a85a4f\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-9k7xv"
Oct 30 18:54:37.540406 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:37.540365    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/427c5128-a124-4dd6-9748-a4dca0a85a4f-istio-data\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-9k7xv\" (UID: \"427c5128-a124-4dd6-9748-a4dca0a85a4f\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-9k7xv"
Oct 30 18:54:37.540698 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:37.540521    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/427c5128-a124-4dd6-9748-a4dca0a85a4f-credential-socket\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-9k7xv\" (UID: \"427c5128-a124-4dd6-9748-a4dca0a85a4f\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-9k7xv"
Oct 30 18:54:37.540698 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:37.540588    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/427c5128-a124-4dd6-9748-a4dca0a85a4f-istio-envoy\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-9k7xv\" (UID: \"427c5128-a124-4dd6-9748-a4dca0a85a4f\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-9k7xv"
Oct 30 18:54:37.540698 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:37.540630    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8nfjh\" (UniqueName: \"kubernetes.io/projected/427c5128-a124-4dd6-9748-a4dca0a85a4f-kube-api-access-8nfjh\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-9k7xv\" (UID: \"427c5128-a124-4dd6-9748-a4dca0a85a4f\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-9k7xv"
Oct 30 18:54:37.562124 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:37.562093    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-9k7xv"]
Oct 30 18:54:37.642056 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:37.642015    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/427c5128-a124-4dd6-9748-a4dca0a85a4f-workload-socket\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-9k7xv\" (UID: \"427c5128-a124-4dd6-9748-a4dca0a85a4f\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-9k7xv"
Oct 30 18:54:37.642056 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:37.642060    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/427c5128-a124-4dd6-9748-a4dca0a85a4f-workload-certs\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-9k7xv\" (UID: \"427c5128-a124-4dd6-9748-a4dca0a85a4f\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-9k7xv"
Oct 30 18:54:37.642319 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:37.642078    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/427c5128-a124-4dd6-9748-a4dca0a85a4f-istiod-ca-cert\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-9k7xv\" (UID: \"427c5128-a124-4dd6-9748-a4dca0a85a4f\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-9k7xv"
Oct 30 18:54:37.642319 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:37.642200    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/427c5128-a124-4dd6-9748-a4dca0a85a4f-istio-data\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-9k7xv\" (UID: \"427c5128-a124-4dd6-9748-a4dca0a85a4f\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-9k7xv"
Oct 30 18:54:37.642319 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:37.642234    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/427c5128-a124-4dd6-9748-a4dca0a85a4f-credential-socket\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-9k7xv\" (UID: \"427c5128-a124-4dd6-9748-a4dca0a85a4f\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-9k7xv"
Oct 30 18:54:37.642319 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:37.642307    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/427c5128-a124-4dd6-9748-a4dca0a85a4f-istio-envoy\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-9k7xv\" (UID: \"427c5128-a124-4dd6-9748-a4dca0a85a4f\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-9k7xv"
Oct 30 18:54:37.642557 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:37.642343    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8nfjh\" (UniqueName: \"kubernetes.io/projected/427c5128-a124-4dd6-9748-a4dca0a85a4f-kube-api-access-8nfjh\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-9k7xv\" (UID: \"427c5128-a124-4dd6-9748-a4dca0a85a4f\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-9k7xv"
Oct 30 18:54:37.642557 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:37.642469    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/427c5128-a124-4dd6-9748-a4dca0a85a4f-istio-podinfo\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-9k7xv\" (UID: \"427c5128-a124-4dd6-9748-a4dca0a85a4f\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-9k7xv"
Oct 30 18:54:37.642557 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:37.642505    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/427c5128-a124-4dd6-9748-a4dca0a85a4f-istio-token\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-9k7xv\" (UID: \"427c5128-a124-4dd6-9748-a4dca0a85a4f\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-9k7xv"
Oct 30 18:54:37.642557 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:37.642525    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/427c5128-a124-4dd6-9748-a4dca0a85a4f-workload-certs\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-9k7xv\" (UID: \"427c5128-a124-4dd6-9748-a4dca0a85a4f\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-9k7xv"
Oct 30 18:54:37.642768 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:37.642668    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/427c5128-a124-4dd6-9748-a4dca0a85a4f-istio-data\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-9k7xv\" (UID: \"427c5128-a124-4dd6-9748-a4dca0a85a4f\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-9k7xv"
Oct 30 18:54:37.642768 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:37.642679    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/427c5128-a124-4dd6-9748-a4dca0a85a4f-credential-socket\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-9k7xv\" (UID: \"427c5128-a124-4dd6-9748-a4dca0a85a4f\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-9k7xv"
Oct 30 18:54:37.642900 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:37.642777    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/427c5128-a124-4dd6-9748-a4dca0a85a4f-workload-socket\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-9k7xv\" (UID: \"427c5128-a124-4dd6-9748-a4dca0a85a4f\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-9k7xv"
Oct 30 18:54:37.642900 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:37.642840    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/427c5128-a124-4dd6-9748-a4dca0a85a4f-istiod-ca-cert\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-9k7xv\" (UID: \"427c5128-a124-4dd6-9748-a4dca0a85a4f\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-9k7xv"
Oct 30 18:54:37.644885 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:37.644861    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/427c5128-a124-4dd6-9748-a4dca0a85a4f-istio-envoy\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-9k7xv\" (UID: \"427c5128-a124-4dd6-9748-a4dca0a85a4f\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-9k7xv"
Oct 30 18:54:37.645438 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:37.645413    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/427c5128-a124-4dd6-9748-a4dca0a85a4f-istio-podinfo\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-9k7xv\" (UID: \"427c5128-a124-4dd6-9748-a4dca0a85a4f\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-9k7xv"
Oct 30 18:54:37.711145 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:37.711078    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8nfjh\" (UniqueName: \"kubernetes.io/projected/427c5128-a124-4dd6-9748-a4dca0a85a4f-kube-api-access-8nfjh\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-9k7xv\" (UID: \"427c5128-a124-4dd6-9748-a4dca0a85a4f\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-9k7xv"
Oct 30 18:54:37.711844 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:37.711820    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/427c5128-a124-4dd6-9748-a4dca0a85a4f-istio-token\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-9k7xv\" (UID: \"427c5128-a124-4dd6-9748-a4dca0a85a4f\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-9k7xv"
Oct 30 18:54:37.727394 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:37.727369    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-9k7xv"
Oct 30 18:54:38.162239 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:38.162200    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-9k7xv"]
Oct 30 18:54:38.162391 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:54:38.162199    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod427c5128_a124_4dd6_9748_a4dca0a85a4f.slice/crio-597061b1625dbe99704d710f120fd053a1f27013a69d0f79d325c36791974e24 WatchSource:0}: Error finding container 597061b1625dbe99704d710f120fd053a1f27013a69d0f79d325c36791974e24: Status 404 returned error can't find the container with id 597061b1625dbe99704d710f120fd053a1f27013a69d0f79d325c36791974e24
Oct 30 18:54:38.164535 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:38.164497    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:54:38.164617 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:38.164582    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:54:38.164657 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:38.164623    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:54:39.173669 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:39.173626    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-9k7xv" event={"ID":"427c5128-a124-4dd6-9748-a4dca0a85a4f","Type":"ContainerStarted","Data":"096083c6914a7bf98e5b8b3f42b9e4584ee0f3434a1d4dbf182585688b1ff6ef"}
Oct 30 18:54:39.174082 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:39.173721    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-9k7xv" event={"ID":"427c5128-a124-4dd6-9748-a4dca0a85a4f","Type":"ContainerStarted","Data":"597061b1625dbe99704d710f120fd053a1f27013a69d0f79d325c36791974e24"}
Oct 30 18:54:39.346598 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:39.346546    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-9k7xv" podStartSLOduration=2.346531671 podStartE2EDuration="2.346531671s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:54:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:54:39.346104757 +0000 UTC m=+1243.593510782" watchObservedRunningTime="2026-10-30 18:54:39.346531671 +0000 UTC m=+1243.593937693"
Oct 30 18:54:39.727884 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:39.727855    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-9k7xv"
Oct 30 18:54:39.732804 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:39.732780    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-9k7xv"
Oct 30 18:54:40.183615 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:40.183582    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-9k7xv"
Oct 30 18:54:40.184586 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:40.184562    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-9k7xv"
Oct 30 18:54:45.219436 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:45.219384    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e7929d/distinct-2-w4-e7929d-kserve-5b479cc87b-htv9p" podUID="f6df655d-86e8-44ff-ace8-27440f56294f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.91:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:54:45.328501 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:45.328443    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-4a7a52/premium-w2-4a7a52-kserve-746d9579f6-hn7hl" podUID="3160b54f-1df9-48de-80d9-8d8a91d47f56" containerName="main" probeResult="failure" output="Get \"https://10.132.0.83:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:54:45.647413 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:45.647370    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-3b6013/distinct-2-w3-3b6013-kserve-7cc5454db-mqknj" podUID="e76c7b20-be2f-4ed2-8dc4-bef127ecc0c9" containerName="main" probeResult="failure" output="Get \"https://10.132.0.98:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:54:45.676191 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:45.675376    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e7929d/distinct-w4-e7929d-kserve-5bc9ff5bf5-kmrwv" podUID="abfd315d-a01a-42dd-9c06-2a80d3f8579a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.88:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:54:46.118812 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:46.118763    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-3b6013/premium-w3-3b6013-kserve-74df5fb95d-k747z" podUID="a9309776-20d5-4c06-bcae-04b8e54493e8" containerName="main" probeResult="failure" output="Get \"https://10.132.0.82:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:54:46.188629 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:46.188590    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-rkm4t" podUID="1ab20f74-2c73-4d56-a604-aad4f8289450" containerName="main" probeResult="failure" output="Get \"https://10.132.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:54:46.218415 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:46.218368    4888 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-e7929d/distinct-2-w4-e7929d-kserve-5b479cc87b-htv9p" podUID="f6df655d-86e8-44ff-ace8-27440f56294f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.91:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:54:46.219120 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:46.218678    4888 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-4a7a52/premium-w2-4a7a52-kserve-746d9579f6-hn7hl" podUID="3160b54f-1df9-48de-80d9-8d8a91d47f56" containerName="main" probeResult="failure" output="Get \"https://10.132.0.83:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:54:46.222390 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:46.222357    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e7929d/distinct-2-w4-e7929d-kserve-5b479cc87b-htv9p" podUID="f6df655d-86e8-44ff-ace8-27440f56294f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.91:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:54:46.223264 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:46.223244    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-e7929d/distinct-2-w4-e7929d-kserve-5b479cc87b-htv9p"
Oct 30 18:54:46.331464 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:46.331417    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-4a7a52/premium-w2-4a7a52-kserve-746d9579f6-hn7hl" podUID="3160b54f-1df9-48de-80d9-8d8a91d47f56" containerName="main" probeResult="failure" output="Get \"https://10.132.0.83:8000/ready\": context deadline exceeded"
Oct 30 18:54:46.332240 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:46.332219    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-4a7a52/premium-w2-4a7a52-kserve-746d9579f6-hn7hl"
Oct 30 18:54:46.387387 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:46.385819    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-3b6013/distinct-w3-3b6013-kserve-857f55d885-8fcrq" podUID="51927877-4d99-4797-82f7-1684f6ecda6b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.92:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:54:47.124305 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:47.065725    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e7929d/distinct-w4-e7929d-kserve-5bc9ff5bf5-kmrwv" podUID="abfd315d-a01a-42dd-9c06-2a80d3f8579a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.88:8000/ready\": context deadline exceeded"
Oct 30 18:54:47.124305 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:47.118890    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-3b6013/distinct-2-w3-3b6013-kserve-7cc5454db-mqknj" podUID="e76c7b20-be2f-4ed2-8dc4-bef127ecc0c9" containerName="main" probeResult="failure" output="Get \"https://10.132.0.98:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:54:47.251523 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:47.212932    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-e7929d/distinct-w4-e7929d-kserve-5bc9ff5bf5-kmrwv"
Oct 30 18:54:47.251523 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:47.213608    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-3b6013/distinct-2-w3-3b6013-kserve-7cc5454db-mqknj"
Oct 30 18:54:47.251523 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:47.229441    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e7929d/distinct-2-w4-e7929d-kserve-5b479cc87b-htv9p" podUID="f6df655d-86e8-44ff-ace8-27440f56294f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.91:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:54:47.251523 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:47.229751    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-3b6013/premium-w3-3b6013-kserve-74df5fb95d-k747z" podUID="a9309776-20d5-4c06-bcae-04b8e54493e8" containerName="main" probeResult="failure" output="Get \"https://10.132.0.82:8000/ready\": context deadline exceeded"
Oct 30 18:54:47.251523 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:47.229974    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-rkm4t" podUID="1ab20f74-2c73-4d56-a604-aad4f8289450" containerName="main" probeResult="failure" output="Get \"https://10.132.0.53:8000/ready\": context deadline exceeded"
Oct 30 18:54:47.251523 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:47.230238    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-3b6013/premium-w3-3b6013-kserve-74df5fb95d-k747z"
Oct 30 18:54:47.251523 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:47.230339    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-rkm4t"
Oct 30 18:54:47.570367 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:47.560001    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-3b6013/distinct-w3-3b6013-kserve-857f55d885-8fcrq" podUID="51927877-4d99-4797-82f7-1684f6ecda6b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.92:8000/ready\": context deadline exceeded"
Oct 30 18:54:47.578244 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:47.578217    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-3b6013/distinct-w3-3b6013-kserve-857f55d885-8fcrq"
Oct 30 18:54:47.696983 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:47.683433    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-4a7a52/premium-w2-4a7a52-kserve-746d9579f6-hn7hl" podUID="3160b54f-1df9-48de-80d9-8d8a91d47f56" containerName="main" probeResult="failure" output="Get \"https://10.132.0.83:8000/ready\": context deadline exceeded"
Oct 30 18:54:47.715476 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:47.712744    4888 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-e7929d/distinct-w4-e7929d-kserve-5bc9ff5bf5-kmrwv" podUID="abfd315d-a01a-42dd-9c06-2a80d3f8579a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.88:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:54:48.076499 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:48.076439    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e7929d/distinct-w4-e7929d-kserve-5bc9ff5bf5-kmrwv" podUID="abfd315d-a01a-42dd-9c06-2a80d3f8579a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.88:8000/ready\": context deadline exceeded"
Oct 30 18:54:48.267429 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:48.267375    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-3b6013/distinct-2-w3-3b6013-kserve-7cc5454db-mqknj" podUID="e76c7b20-be2f-4ed2-8dc4-bef127ecc0c9" containerName="main" probeResult="failure" output="Get \"https://10.132.0.98:8000/ready\": context deadline exceeded"
Oct 30 18:54:48.292461 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:48.280190    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-rkm4t" podUID="1ab20f74-2c73-4d56-a604-aad4f8289450" containerName="main" probeResult="failure" output="Get \"https://10.132.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:54:48.294633 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:48.294582    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e7929d/distinct-2-w4-e7929d-kserve-5b479cc87b-htv9p" podUID="f6df655d-86e8-44ff-ace8-27440f56294f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.91:8000/ready\": context deadline exceeded"
Oct 30 18:54:48.310353 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:48.310308    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-3b6013/premium-w3-3b6013-kserve-74df5fb95d-k747z" podUID="a9309776-20d5-4c06-bcae-04b8e54493e8" containerName="main" probeResult="failure" output="Get \"https://10.132.0.82:8000/ready\": context deadline exceeded"
Oct 30 18:54:48.589369 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:48.587363    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-3b6013/distinct-w3-3b6013-kserve-857f55d885-8fcrq" podUID="51927877-4d99-4797-82f7-1684f6ecda6b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.92:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:54:48.710014 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:48.698354    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-4a7a52/premium-w2-4a7a52-kserve-746d9579f6-hn7hl" podUID="3160b54f-1df9-48de-80d9-8d8a91d47f56" containerName="main" probeResult="failure" output="Get \"https://10.132.0.83:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:54:49.122404 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:49.122359    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e7929d/distinct-w4-e7929d-kserve-5bc9ff5bf5-kmrwv" podUID="abfd315d-a01a-42dd-9c06-2a80d3f8579a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.88:8000/ready\": context deadline exceeded"
Oct 30 18:54:49.224310 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:49.215816    4888 prober.go:122] "Probe failed" probeType="Liveness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-rkm4t" podUID="1ab20f74-2c73-4d56-a604-aad4f8289450" containerName="main" probeResult="failure" output="Get \"https://10.132.0.53:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:54:49.291665 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:49.291639    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-e7929d/distinct-2-w4-e7929d-kserve-5b479cc87b-htv9p"
Oct 30 18:54:49.295709 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:49.295690    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-e7929d/distinct-w4-e7929d-kserve-5bc9ff5bf5-kmrwv"
Oct 30 18:54:49.309313 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:49.292045    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-rkm4t" podUID="1ab20f74-2c73-4d56-a604-aad4f8289450" containerName="main" probeResult="failure" output="Get \"https://10.132.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:54:49.310250 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:49.292122    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-3b6013/distinct-2-w3-3b6013-kserve-7cc5454db-mqknj" podUID="e76c7b20-be2f-4ed2-8dc4-bef127ecc0c9" containerName="main" probeResult="failure" output="Get \"https://10.132.0.98:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:54:49.355201 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:49.355158    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-rkm4t"
Oct 30 18:54:49.368213 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:49.366809    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-3b6013/premium-w3-3b6013-kserve-74df5fb95d-k747z" podUID="a9309776-20d5-4c06-bcae-04b8e54493e8" containerName="main" probeResult="failure" output="Get \"https://10.132.0.82:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:54:49.430915 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:49.430831    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-3b6013/premium-w3-3b6013-kserve-74df5fb95d-k747z"
Oct 30 18:54:49.513963 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:49.513934    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-4a7a52/premium-w2-4a7a52-kserve-746d9579f6-hn7hl"
Oct 30 18:54:49.558641 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:49.558614    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-3b6013/distinct-w3-3b6013-kserve-857f55d885-8fcrq"
Oct 30 18:54:49.679228 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:49.679199    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-3b6013/distinct-2-w3-3b6013-kserve-7cc5454db-mqknj"
Oct 30 18:54:53.358363 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:53.357526    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-ntnwm"]
Oct 30 18:54:53.366385 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:53.365697    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-ntnwm"
Oct 30 18:54:53.480523 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:53.480491    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-ntnwm"]
Oct 30 18:54:53.551306 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:53.551226    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/73999d2f-99a7-43f6-99e9-de05ee01f6ce-istiod-ca-cert\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-ntnwm\" (UID: \"73999d2f-99a7-43f6-99e9-de05ee01f6ce\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-ntnwm"
Oct 30 18:54:53.551510 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:53.551435    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/73999d2f-99a7-43f6-99e9-de05ee01f6ce-workload-socket\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-ntnwm\" (UID: \"73999d2f-99a7-43f6-99e9-de05ee01f6ce\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-ntnwm"
Oct 30 18:54:53.551510 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:53.551468    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/73999d2f-99a7-43f6-99e9-de05ee01f6ce-istio-podinfo\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-ntnwm\" (UID: \"73999d2f-99a7-43f6-99e9-de05ee01f6ce\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-ntnwm"
Oct 30 18:54:53.551630 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:53.551541    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/73999d2f-99a7-43f6-99e9-de05ee01f6ce-workload-certs\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-ntnwm\" (UID: \"73999d2f-99a7-43f6-99e9-de05ee01f6ce\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-ntnwm"
Oct 30 18:54:53.551630 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:53.551566    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/73999d2f-99a7-43f6-99e9-de05ee01f6ce-istio-data\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-ntnwm\" (UID: \"73999d2f-99a7-43f6-99e9-de05ee01f6ce\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-ntnwm"
Oct 30 18:54:53.551630 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:53.551589    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/73999d2f-99a7-43f6-99e9-de05ee01f6ce-istio-token\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-ntnwm\" (UID: \"73999d2f-99a7-43f6-99e9-de05ee01f6ce\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-ntnwm"
Oct 30 18:54:53.551630 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:53.551618    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lmt4t\" (UniqueName: \"kubernetes.io/projected/73999d2f-99a7-43f6-99e9-de05ee01f6ce-kube-api-access-lmt4t\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-ntnwm\" (UID: \"73999d2f-99a7-43f6-99e9-de05ee01f6ce\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-ntnwm"
Oct 30 18:54:53.551841 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:53.551655    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/73999d2f-99a7-43f6-99e9-de05ee01f6ce-credential-socket\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-ntnwm\" (UID: \"73999d2f-99a7-43f6-99e9-de05ee01f6ce\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-ntnwm"
Oct 30 18:54:53.551841 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:53.551693    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/73999d2f-99a7-43f6-99e9-de05ee01f6ce-istio-envoy\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-ntnwm\" (UID: \"73999d2f-99a7-43f6-99e9-de05ee01f6ce\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-ntnwm"
Oct 30 18:54:53.654106 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:53.653218    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/73999d2f-99a7-43f6-99e9-de05ee01f6ce-istiod-ca-cert\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-ntnwm\" (UID: \"73999d2f-99a7-43f6-99e9-de05ee01f6ce\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-ntnwm"
Oct 30 18:54:53.654106 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:53.653432    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/73999d2f-99a7-43f6-99e9-de05ee01f6ce-workload-socket\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-ntnwm\" (UID: \"73999d2f-99a7-43f6-99e9-de05ee01f6ce\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-ntnwm"
Oct 30 18:54:53.654106 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:53.653467    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/73999d2f-99a7-43f6-99e9-de05ee01f6ce-istio-podinfo\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-ntnwm\" (UID: \"73999d2f-99a7-43f6-99e9-de05ee01f6ce\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-ntnwm"
Oct 30 18:54:53.654106 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:53.653543    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/73999d2f-99a7-43f6-99e9-de05ee01f6ce-workload-certs\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-ntnwm\" (UID: \"73999d2f-99a7-43f6-99e9-de05ee01f6ce\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-ntnwm"
Oct 30 18:54:53.654106 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:53.653570    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/73999d2f-99a7-43f6-99e9-de05ee01f6ce-istio-data\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-ntnwm\" (UID: \"73999d2f-99a7-43f6-99e9-de05ee01f6ce\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-ntnwm"
Oct 30 18:54:53.654106 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:53.653595    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/73999d2f-99a7-43f6-99e9-de05ee01f6ce-istio-token\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-ntnwm\" (UID: \"73999d2f-99a7-43f6-99e9-de05ee01f6ce\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-ntnwm"
Oct 30 18:54:53.654106 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:53.653631    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lmt4t\" (UniqueName: \"kubernetes.io/projected/73999d2f-99a7-43f6-99e9-de05ee01f6ce-kube-api-access-lmt4t\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-ntnwm\" (UID: \"73999d2f-99a7-43f6-99e9-de05ee01f6ce\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-ntnwm"
Oct 30 18:54:53.654106 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:53.653670    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/73999d2f-99a7-43f6-99e9-de05ee01f6ce-credential-socket\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-ntnwm\" (UID: \"73999d2f-99a7-43f6-99e9-de05ee01f6ce\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-ntnwm"
Oct 30 18:54:53.654106 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:53.653708    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/73999d2f-99a7-43f6-99e9-de05ee01f6ce-istio-envoy\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-ntnwm\" (UID: \"73999d2f-99a7-43f6-99e9-de05ee01f6ce\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-ntnwm"
Oct 30 18:54:53.654736 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:53.654576    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/73999d2f-99a7-43f6-99e9-de05ee01f6ce-workload-certs\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-ntnwm\" (UID: \"73999d2f-99a7-43f6-99e9-de05ee01f6ce\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-ntnwm"
Oct 30 18:54:53.654736 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:53.654641    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/73999d2f-99a7-43f6-99e9-de05ee01f6ce-workload-socket\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-ntnwm\" (UID: \"73999d2f-99a7-43f6-99e9-de05ee01f6ce\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-ntnwm"
Oct 30 18:54:53.655696 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:53.655160    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/73999d2f-99a7-43f6-99e9-de05ee01f6ce-istiod-ca-cert\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-ntnwm\" (UID: \"73999d2f-99a7-43f6-99e9-de05ee01f6ce\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-ntnwm"
Oct 30 18:54:53.655696 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:53.655637    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/73999d2f-99a7-43f6-99e9-de05ee01f6ce-istio-data\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-ntnwm\" (UID: \"73999d2f-99a7-43f6-99e9-de05ee01f6ce\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-ntnwm"
Oct 30 18:54:53.656113 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:53.656091    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/73999d2f-99a7-43f6-99e9-de05ee01f6ce-credential-socket\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-ntnwm\" (UID: \"73999d2f-99a7-43f6-99e9-de05ee01f6ce\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-ntnwm"
Oct 30 18:54:53.657081 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:53.657053    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/73999d2f-99a7-43f6-99e9-de05ee01f6ce-istio-envoy\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-ntnwm\" (UID: \"73999d2f-99a7-43f6-99e9-de05ee01f6ce\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-ntnwm"
Oct 30 18:54:53.658841 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:53.658814    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/73999d2f-99a7-43f6-99e9-de05ee01f6ce-istio-podinfo\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-ntnwm\" (UID: \"73999d2f-99a7-43f6-99e9-de05ee01f6ce\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-ntnwm"
Oct 30 18:54:53.717916 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:53.717884    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lmt4t\" (UniqueName: \"kubernetes.io/projected/73999d2f-99a7-43f6-99e9-de05ee01f6ce-kube-api-access-lmt4t\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-ntnwm\" (UID: \"73999d2f-99a7-43f6-99e9-de05ee01f6ce\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-ntnwm"
Oct 30 18:54:53.719870 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:53.719840    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/73999d2f-99a7-43f6-99e9-de05ee01f6ce-istio-token\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-ntnwm\" (UID: \"73999d2f-99a7-43f6-99e9-de05ee01f6ce\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-ntnwm"
Oct 30 18:54:54.000350 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:54.000241    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-ntnwm"
Oct 30 18:54:54.449372 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:54.448798    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-ntnwm"]
Oct 30 18:54:54.450431 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:54:54.450399    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73999d2f_99a7_43f6_99e9_de05ee01f6ce.slice/crio-db284f3aae7efee03d1b133be389244cdaf3aa2df19f7d523388ec2c367cd070 WatchSource:0}: Error finding container db284f3aae7efee03d1b133be389244cdaf3aa2df19f7d523388ec2c367cd070: Status 404 returned error can't find the container with id db284f3aae7efee03d1b133be389244cdaf3aa2df19f7d523388ec2c367cd070
Oct 30 18:54:54.453772 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:54.453734    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:54:54.453905 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:54.453819    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:54:54.453905 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:54.453863    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:54:55.445035 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:55.444997    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-ntnwm" event={"ID":"73999d2f-99a7-43f6-99e9-de05ee01f6ce","Type":"ContainerStarted","Data":"ceb8b2d77237b854c9009c948cbf397b3de38c3588c00fa6d87fd12a69512bac"}
Oct 30 18:54:55.445238 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:55.445059    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-ntnwm" event={"ID":"73999d2f-99a7-43f6-99e9-de05ee01f6ce","Type":"ContainerStarted","Data":"db284f3aae7efee03d1b133be389244cdaf3aa2df19f7d523388ec2c367cd070"}
Oct 30 18:54:55.686315 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:55.674621    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-ntnwm" podStartSLOduration=2.674597308 podStartE2EDuration="2.674597308s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:54:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:54:55.61852651 +0000 UTC m=+1259.865932528" watchObservedRunningTime="2026-10-30 18:54:55.674597308 +0000 UTC m=+1259.922003334"
Oct 30 18:54:56.003180 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:56.003145    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-ntnwm"
Oct 30 18:54:56.003557 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:56.003532    4888 patch_prober.go:29] interesting pod/e2e-worker-w3-3b6013-openshift-default-85cf856477-ntnwm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.101:15021/healthz/ready\": dial tcp 10.132.0.101:15021: connect: connection refused" start-of-body=
Oct 30 18:54:56.003723 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:56.003692    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-ntnwm" podUID="73999d2f-99a7-43f6-99e9-de05ee01f6ce" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.101:15021/healthz/ready\": dial tcp 10.132.0.101:15021: connect: connection refused"
Oct 30 18:54:57.001213 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:57.001175    4888 patch_prober.go:29] interesting pod/e2e-worker-w3-3b6013-openshift-default-85cf856477-ntnwm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.101:15021/healthz/ready\": dial tcp 10.132.0.101:15021: connect: connection refused" start-of-body=
Oct 30 18:54:57.001838 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:57.001805    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-ntnwm" podUID="73999d2f-99a7-43f6-99e9-de05ee01f6ce" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.101:15021/healthz/ready\": dial tcp 10.132.0.101:15021: connect: connection refused"
Oct 30 18:54:58.001614 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:58.001581    4888 patch_prober.go:29] interesting pod/e2e-worker-w3-3b6013-openshift-default-85cf856477-ntnwm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.101:15021/healthz/ready\": dial tcp 10.132.0.101:15021: connect: connection refused" start-of-body=
Oct 30 18:54:58.002020 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:58.001630    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-ntnwm" podUID="73999d2f-99a7-43f6-99e9-de05ee01f6ce" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.101:15021/healthz/ready\": dial tcp 10.132.0.101:15021: connect: connection refused"
Oct 30 18:54:59.000825 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:59.000781    4888 patch_prober.go:29] interesting pod/e2e-worker-w3-3b6013-openshift-default-85cf856477-ntnwm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.101:15021/healthz/ready\": dial tcp 10.132.0.101:15021: connect: connection refused" start-of-body=
Oct 30 18:54:59.001015 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:54:59.000856    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-ntnwm" podUID="73999d2f-99a7-43f6-99e9-de05ee01f6ce" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.101:15021/healthz/ready\": dial tcp 10.132.0.101:15021: connect: connection refused"
Oct 30 18:55:00.000795 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:55:00.000751    4888 patch_prober.go:29] interesting pod/e2e-worker-w3-3b6013-openshift-default-85cf856477-ntnwm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.101:15021/healthz/ready\": dial tcp 10.132.0.101:15021: connect: connection refused" start-of-body=
Oct 30 18:55:00.001320 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:55:00.000827    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-ntnwm" podUID="73999d2f-99a7-43f6-99e9-de05ee01f6ce" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.101:15021/healthz/ready\": dial tcp 10.132.0.101:15021: connect: connection refused"
Oct 30 18:55:01.001578 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:55:01.001547    4888 patch_prober.go:29] interesting pod/e2e-worker-w3-3b6013-openshift-default-85cf856477-ntnwm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.101:15021/healthz/ready\": dial tcp 10.132.0.101:15021: connect: connection refused" start-of-body=
Oct 30 18:55:01.001988 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:55:01.001612    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-ntnwm" podUID="73999d2f-99a7-43f6-99e9-de05ee01f6ce" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.101:15021/healthz/ready\": dial tcp 10.132.0.101:15021: connect: connection refused"
Oct 30 18:55:02.072513 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:55:02.072484    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-ntnwm"
Oct 30 18:55:02.073051 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:55:02.073012    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-ntnwm"
Oct 30 18:55:02.073987 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:55:02.073967    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-ntnwm"
Oct 30 18:55:05.619089 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:55:05.619052    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-jbsw6" containerName="extractor"
Oct 30 18:55:08.485573 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:55:08.485540    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-kgfbt"]
Oct 30 18:55:08.491177 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:55:08.491157    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-kgfbt"
Oct 30 18:55:08.566149 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:55:08.566118    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aa727ad6-0a18-4ce1-b48d-f2f01cd83677-istio-data\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-kgfbt\" (UID: \"aa727ad6-0a18-4ce1-b48d-f2f01cd83677\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-kgfbt"
Oct 30 18:55:08.566333 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:55:08.566161    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aa727ad6-0a18-4ce1-b48d-f2f01cd83677-istio-envoy\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-kgfbt\" (UID: \"aa727ad6-0a18-4ce1-b48d-f2f01cd83677\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-kgfbt"
Oct 30 18:55:08.566333 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:55:08.566193    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aa727ad6-0a18-4ce1-b48d-f2f01cd83677-workload-certs\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-kgfbt\" (UID: \"aa727ad6-0a18-4ce1-b48d-f2f01cd83677\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-kgfbt"
Oct 30 18:55:08.566333 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:55:08.566306    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aa727ad6-0a18-4ce1-b48d-f2f01cd83677-istiod-ca-cert\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-kgfbt\" (UID: \"aa727ad6-0a18-4ce1-b48d-f2f01cd83677\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-kgfbt"
Oct 30 18:55:08.566461 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:55:08.566365    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aa727ad6-0a18-4ce1-b48d-f2f01cd83677-istio-token\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-kgfbt\" (UID: \"aa727ad6-0a18-4ce1-b48d-f2f01cd83677\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-kgfbt"
Oct 30 18:55:08.566461 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:55:08.566433    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa727ad6-0a18-4ce1-b48d-f2f01cd83677-workload-socket\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-kgfbt\" (UID: \"aa727ad6-0a18-4ce1-b48d-f2f01cd83677\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-kgfbt"
Oct 30 18:55:08.566547 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:55:08.566482    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w6k7k\" (UniqueName: \"kubernetes.io/projected/aa727ad6-0a18-4ce1-b48d-f2f01cd83677-kube-api-access-w6k7k\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-kgfbt\" (UID: \"aa727ad6-0a18-4ce1-b48d-f2f01cd83677\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-kgfbt"
Oct 30 18:55:08.566597 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:55:08.566585    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aa727ad6-0a18-4ce1-b48d-f2f01cd83677-istio-podinfo\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-kgfbt\" (UID: \"aa727ad6-0a18-4ce1-b48d-f2f01cd83677\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-kgfbt"
Oct 30 18:55:08.566633 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:55:08.566619    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa727ad6-0a18-4ce1-b48d-f2f01cd83677-credential-socket\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-kgfbt\" (UID: \"aa727ad6-0a18-4ce1-b48d-f2f01cd83677\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-kgfbt"
Oct 30 18:55:08.667844 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:55:08.667803    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa727ad6-0a18-4ce1-b48d-f2f01cd83677-workload-socket\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-kgfbt\" (UID: \"aa727ad6-0a18-4ce1-b48d-f2f01cd83677\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-kgfbt"
Oct 30 18:55:08.668037 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:55:08.667860    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w6k7k\" (UniqueName: \"kubernetes.io/projected/aa727ad6-0a18-4ce1-b48d-f2f01cd83677-kube-api-access-w6k7k\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-kgfbt\" (UID: \"aa727ad6-0a18-4ce1-b48d-f2f01cd83677\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-kgfbt"
Oct 30 18:55:08.668037 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:55:08.667992    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aa727ad6-0a18-4ce1-b48d-f2f01cd83677-istio-podinfo\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-kgfbt\" (UID: \"aa727ad6-0a18-4ce1-b48d-f2f01cd83677\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-kgfbt"
Oct 30 18:55:08.668156 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:55:08.668041    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa727ad6-0a18-4ce1-b48d-f2f01cd83677-credential-socket\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-kgfbt\" (UID: \"aa727ad6-0a18-4ce1-b48d-f2f01cd83677\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-kgfbt"
Oct 30 18:55:08.668156 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:55:08.668078    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aa727ad6-0a18-4ce1-b48d-f2f01cd83677-istio-data\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-kgfbt\" (UID: \"aa727ad6-0a18-4ce1-b48d-f2f01cd83677\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-kgfbt"
Oct 30 18:55:08.668156 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:55:08.668115    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aa727ad6-0a18-4ce1-b48d-f2f01cd83677-istio-envoy\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-kgfbt\" (UID: \"aa727ad6-0a18-4ce1-b48d-f2f01cd83677\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-kgfbt"
Oct 30 18:55:08.668156 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:55:08.668146    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aa727ad6-0a18-4ce1-b48d-f2f01cd83677-workload-certs\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-kgfbt\" (UID: \"aa727ad6-0a18-4ce1-b48d-f2f01cd83677\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-kgfbt"
Oct 30 18:55:08.668400 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:55:08.668188    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aa727ad6-0a18-4ce1-b48d-f2f01cd83677-istiod-ca-cert\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-kgfbt\" (UID: \"aa727ad6-0a18-4ce1-b48d-f2f01cd83677\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-kgfbt"
Oct 30 18:55:08.668400 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:55:08.668188    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa727ad6-0a18-4ce1-b48d-f2f01cd83677-workload-socket\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-kgfbt\" (UID: \"aa727ad6-0a18-4ce1-b48d-f2f01cd83677\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-kgfbt"
Oct 30 18:55:08.668400 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:55:08.668316    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aa727ad6-0a18-4ce1-b48d-f2f01cd83677-istio-token\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-kgfbt\" (UID: \"aa727ad6-0a18-4ce1-b48d-f2f01cd83677\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-kgfbt"
Oct 30 18:55:08.668566 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:55:08.668461    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa727ad6-0a18-4ce1-b48d-f2f01cd83677-credential-socket\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-kgfbt\" (UID: \"aa727ad6-0a18-4ce1-b48d-f2f01cd83677\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-kgfbt"
Oct 30 18:55:08.668566 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:55:08.668498    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aa727ad6-0a18-4ce1-b48d-f2f01cd83677-istio-data\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-kgfbt\" (UID: \"aa727ad6-0a18-4ce1-b48d-f2f01cd83677\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-kgfbt"
Oct 30 18:55:08.668743 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:55:08.668725    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aa727ad6-0a18-4ce1-b48d-f2f01cd83677-workload-certs\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-kgfbt\" (UID: \"aa727ad6-0a18-4ce1-b48d-f2f01cd83677\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-kgfbt"
Oct 30 18:55:08.668921 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:55:08.668900    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aa727ad6-0a18-4ce1-b48d-f2f01cd83677-istiod-ca-cert\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-kgfbt\" (UID: \"aa727ad6-0a18-4ce1-b48d-f2f01cd83677\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-kgfbt"
Oct 30 18:55:08.670991 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:55:08.670968    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aa727ad6-0a18-4ce1-b48d-f2f01cd83677-istio-envoy\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-kgfbt\" (UID: \"aa727ad6-0a18-4ce1-b48d-f2f01cd83677\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-kgfbt"
Oct 30 18:55:08.671322 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:55:08.671274    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aa727ad6-0a18-4ce1-b48d-f2f01cd83677-istio-podinfo\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-kgfbt\" (UID: \"aa727ad6-0a18-4ce1-b48d-f2f01cd83677\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-kgfbt"
Oct 30 18:55:08.705544 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:55:08.705514    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-kgfbt"]
Oct 30 18:55:08.734863 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:55:08.734836    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aa727ad6-0a18-4ce1-b48d-f2f01cd83677-istio-token\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-kgfbt\" (UID: \"aa727ad6-0a18-4ce1-b48d-f2f01cd83677\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-kgfbt"
Oct 30 18:55:08.735067 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:55:08.735003    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w6k7k\" (UniqueName: \"kubernetes.io/projected/aa727ad6-0a18-4ce1-b48d-f2f01cd83677-kube-api-access-w6k7k\") pod \"e2e-worker-w3-3b6013-openshift-default-85cf856477-kgfbt\" (UID: \"aa727ad6-0a18-4ce1-b48d-f2f01cd83677\") " pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-kgfbt"
Oct 30 18:55:08.805239 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:55:08.805151    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-kgfbt"
Oct 30 18:55:09.240567 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:55:09.240539    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-kgfbt"]
Oct 30 18:55:09.241301 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:55:09.241255    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa727ad6_0a18_4ce1_b48d_f2f01cd83677.slice/crio-295f35158c3a5062de68cad069aadb069c2f31643b7c3ead00214c2d33b7ef2c WatchSource:0}: Error finding container 295f35158c3a5062de68cad069aadb069c2f31643b7c3ead00214c2d33b7ef2c: Status 404 returned error can't find the container with id 295f35158c3a5062de68cad069aadb069c2f31643b7c3ead00214c2d33b7ef2c
Oct 30 18:55:09.243895 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:55:09.243862    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:55:09.243963 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:55:09.243931    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:55:09.244011 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:55:09.243961    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:55:09.580104 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:55:09.580068    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-kgfbt" event={"ID":"aa727ad6-0a18-4ce1-b48d-f2f01cd83677","Type":"ContainerStarted","Data":"e1994a709972cdd033ccff7397b1b7e1a34a7c132b3dc54cfc81feb6a3aab552"}
Oct 30 18:55:09.580630 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:55:09.580152    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-kgfbt" event={"ID":"aa727ad6-0a18-4ce1-b48d-f2f01cd83677","Type":"ContainerStarted","Data":"295f35158c3a5062de68cad069aadb069c2f31643b7c3ead00214c2d33b7ef2c"}
Oct 30 18:55:09.706517 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:55:09.706463    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-kgfbt" podStartSLOduration=1.7064501760000002 podStartE2EDuration="1.706450176s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:55:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:55:09.702261103 +0000 UTC m=+1273.949667129" watchObservedRunningTime="2026-10-30 18:55:09.706450176 +0000 UTC m=+1273.953856276"
Oct 30 18:55:09.806339 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:55:09.806306    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-kgfbt"
Oct 30 18:55:10.807614 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:55:10.807578    4888 patch_prober.go:29] interesting pod/e2e-worker-w3-3b6013-openshift-default-85cf856477-kgfbt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.102:15021/healthz/ready\": dial tcp 10.132.0.102:15021: connect: connection refused" start-of-body=
Oct 30 18:55:10.808036 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:55:10.807640    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-kgfbt" podUID="aa727ad6-0a18-4ce1-b48d-f2f01cd83677" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.102:15021/healthz/ready\": dial tcp 10.132.0.102:15021: connect: connection refused"
Oct 30 18:55:11.806794 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:55:11.806761    4888 patch_prober.go:29] interesting pod/e2e-worker-w3-3b6013-openshift-default-85cf856477-kgfbt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.102:15021/healthz/ready\": dial tcp 10.132.0.102:15021: connect: connection refused" start-of-body=
Oct 30 18:55:11.806984 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:55:11.806818    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-kgfbt" podUID="aa727ad6-0a18-4ce1-b48d-f2f01cd83677" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.102:15021/healthz/ready\": dial tcp 10.132.0.102:15021: connect: connection refused"
Oct 30 18:55:12.806101 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:55:12.806066    4888 patch_prober.go:29] interesting pod/e2e-worker-w3-3b6013-openshift-default-85cf856477-kgfbt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.102:15021/healthz/ready\": dial tcp 10.132.0.102:15021: connect: connection refused" start-of-body=
Oct 30 18:55:12.806603 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:55:12.806126    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-kgfbt" podUID="aa727ad6-0a18-4ce1-b48d-f2f01cd83677" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.102:15021/healthz/ready\": dial tcp 10.132.0.102:15021: connect: connection refused"
Oct 30 18:55:13.806091 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:55:13.806056    4888 patch_prober.go:29] interesting pod/e2e-worker-w3-3b6013-openshift-default-85cf856477-kgfbt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.102:15021/healthz/ready\": dial tcp 10.132.0.102:15021: connect: connection refused" start-of-body=
Oct 30 18:55:13.806317 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:55:13.806131    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-kgfbt" podUID="aa727ad6-0a18-4ce1-b48d-f2f01cd83677" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.102:15021/healthz/ready\": dial tcp 10.132.0.102:15021: connect: connection refused"
Oct 30 18:55:14.806242 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:55:14.806206    4888 patch_prober.go:29] interesting pod/e2e-worker-w3-3b6013-openshift-default-85cf856477-kgfbt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.102:15021/healthz/ready\": dial tcp 10.132.0.102:15021: connect: connection refused" start-of-body=
Oct 30 18:55:14.806468 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:55:14.806265    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-kgfbt" podUID="aa727ad6-0a18-4ce1-b48d-f2f01cd83677" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.102:15021/healthz/ready\": dial tcp 10.132.0.102:15021: connect: connection refused"
Oct 30 18:55:15.806421 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:55:15.806383    4888 patch_prober.go:29] interesting pod/e2e-worker-w3-3b6013-openshift-default-85cf856477-kgfbt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.102:15021/healthz/ready\": dial tcp 10.132.0.102:15021: connect: connection refused" start-of-body=
Oct 30 18:55:15.806636 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:55:15.806463    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-kgfbt" podUID="aa727ad6-0a18-4ce1-b48d-f2f01cd83677" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.102:15021/healthz/ready\": dial tcp 10.132.0.102:15021: connect: connection refused"
Oct 30 18:55:16.731920 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:55:16.731890    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-x959p"]
Oct 30 18:55:16.736111 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:55:16.736094    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-x959p"
Oct 30 18:55:16.789720 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:55:16.789686    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Oct 30 18:55:16.789720 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:55:16.789708    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Oct 30 18:55:16.789884 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:55:16.789710    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Oct 30 18:55:16.789923 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:55:16.789898    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-jhrl9\""
Oct 30 18:55:16.810433 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:55:16.810412    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-kgfbt"
Oct 30 18:55:16.810839 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:55:16.810723    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-kgfbt"
Oct 30 18:55:16.811499 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:55:16.811482    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-kgfbt"
Oct 30 18:55:16.846627 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:55:16.846605    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-x959p"]
Oct 30 18:55:16.898145 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:55:16.898111    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/ff78b45c-608b-4562-99d5-d4cf35e59c71-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-x959p\" (UID: \"ff78b45c-608b-4562-99d5-d4cf35e59c71\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-x959p"
Oct 30 18:55:16.898565 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:55:16.898531    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ff78b45c-608b-4562-99d5-d4cf35e59c71-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-x959p\" (UID: \"ff78b45c-608b-4562-99d5-d4cf35e59c71\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-x959p"
Oct 30 18:55:16.898700 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:55:16.898572    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ff78b45c-608b-4562-99d5-d4cf35e59c71-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-x959p\" (UID: \"ff78b45c-608b-4562-99d5-d4cf35e59c71\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-x959p"
Oct 30 18:55:16.898700 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:55:16.898607    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/ff78b45c-608b-4562-99d5-d4cf35e59c71-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-x959p\" (UID: \"ff78b45c-608b-4562-99d5-d4cf35e59c71\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-x959p"
Oct 30 18:55:16.898817 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:55:16.898709    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/ff78b45c-608b-4562-99d5-d4cf35e59c71-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-x959p\" (UID: \"ff78b45c-608b-4562-99d5-d4cf35e59c71\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-x959p"
Oct 30 18:55:16.898817 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:55:16.898774    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/ff78b45c-608b-4562-99d5-d4cf35e59c71-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-x959p\" (UID: \"ff78b45c-608b-4562-99d5-d4cf35e59c71\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-x959p"
Oct 30 18:55:16.898817 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:55:16.898799    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qd2xz\" (UniqueName: \"kubernetes.io/projected/ff78b45c-608b-4562-99d5-d4cf35e59c71-kube-api-access-qd2xz\") pod \"istiod-openshift-gateway-7ddd9cfdc8-x959p\" (UID: \"ff78b45c-608b-4562-99d5-d4cf35e59c71\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-x959p"
Oct 30 18:55:17.002816 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:55:17.000719    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/ff78b45c-608b-4562-99d5-d4cf35e59c71-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-x959p\" (UID: \"ff78b45c-608b-4562-99d5-d4cf35e59c71\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-x959p"
Oct 30 18:55:17.002816 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:55:17.000869    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/ff78b45c-608b-4562-99d5-d4cf35e59c71-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-x959p\" (UID: \"ff78b45c-608b-4562-99d5-d4cf35e59c71\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-x959p"
Oct 30 18:55:17.002816 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:55:17.000957    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/ff78b45c-608b-4562-99d5-d4cf35e59c71-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-x959p\" (UID: \"ff78b45c-608b-4562-99d5-d4cf35e59c71\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-x959p"
Oct 30 18:55:17.002816 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:55:17.000981    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qd2xz\" (UniqueName: \"kubernetes.io/projected/ff78b45c-608b-4562-99d5-d4cf35e59c71-kube-api-access-qd2xz\") pod \"istiod-openshift-gateway-7ddd9cfdc8-x959p\" (UID: \"ff78b45c-608b-4562-99d5-d4cf35e59c71\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-x959p"
Oct 30 18:55:17.002816 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:55:17.001104    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/ff78b45c-608b-4562-99d5-d4cf35e59c71-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-x959p\" (UID: \"ff78b45c-608b-4562-99d5-d4cf35e59c71\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-x959p"
Oct 30 18:55:17.002816 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:55:17.001326    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ff78b45c-608b-4562-99d5-d4cf35e59c71-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-x959p\" (UID: \"ff78b45c-608b-4562-99d5-d4cf35e59c71\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-x959p"
Oct 30 18:55:17.002816 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:55:17.001356    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ff78b45c-608b-4562-99d5-d4cf35e59c71-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-x959p\" (UID: \"ff78b45c-608b-4562-99d5-d4cf35e59c71\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-x959p"
Oct 30 18:55:17.005380 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:55:17.004362    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ff78b45c-608b-4562-99d5-d4cf35e59c71-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-x959p\" (UID: \"ff78b45c-608b-4562-99d5-d4cf35e59c71\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-x959p"
Oct 30 18:55:17.005380 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:55:17.004372    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/ff78b45c-608b-4562-99d5-d4cf35e59c71-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-x959p\" (UID: \"ff78b45c-608b-4562-99d5-d4cf35e59c71\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-x959p"
Oct 30 18:55:17.013566 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:55:17.006723    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/ff78b45c-608b-4562-99d5-d4cf35e59c71-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-x959p\" (UID: \"ff78b45c-608b-4562-99d5-d4cf35e59c71\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-x959p"
Oct 30 18:55:17.013566 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:55:17.006982    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/ff78b45c-608b-4562-99d5-d4cf35e59c71-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-x959p\" (UID: \"ff78b45c-608b-4562-99d5-d4cf35e59c71\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-x959p"
Oct 30 18:55:17.017746 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:55:17.017719    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/ff78b45c-608b-4562-99d5-d4cf35e59c71-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-x959p\" (UID: \"ff78b45c-608b-4562-99d5-d4cf35e59c71\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-x959p"
Oct 30 18:55:17.069738 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:55:17.069700    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qd2xz\" (UniqueName: \"kubernetes.io/projected/ff78b45c-608b-4562-99d5-d4cf35e59c71-kube-api-access-qd2xz\") pod \"istiod-openshift-gateway-7ddd9cfdc8-x959p\" (UID: \"ff78b45c-608b-4562-99d5-d4cf35e59c71\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-x959p"
Oct 30 18:55:17.070249 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:55:17.070226    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ff78b45c-608b-4562-99d5-d4cf35e59c71-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-x959p\" (UID: \"ff78b45c-608b-4562-99d5-d4cf35e59c71\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-x959p"
Oct 30 18:55:17.349653 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:55:17.349621    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-x959p"
Oct 30 18:55:17.798913 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:55:17.798884    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff78b45c_608b_4562_99d5_d4cf35e59c71.slice/crio-97f2abd085766f36b69571f4ce0c4aa9d1d147e8937d3d8963a3d32527aea750 WatchSource:0}: Error finding container 97f2abd085766f36b69571f4ce0c4aa9d1d147e8937d3d8963a3d32527aea750: Status 404 returned error can't find the container with id 97f2abd085766f36b69571f4ce0c4aa9d1d147e8937d3d8963a3d32527aea750
Oct 30 18:55:17.801366 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:55:17.801340    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-x959p"]
Oct 30 18:55:17.801470 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:55:17.801383    4888 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 18:55:18.659168 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:55:18.659134    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-x959p" event={"ID":"ff78b45c-608b-4562-99d5-d4cf35e59c71","Type":"ContainerStarted","Data":"97f2abd085766f36b69571f4ce0c4aa9d1d147e8937d3d8963a3d32527aea750"}
Oct 30 18:55:19.398307 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:55:19.397872    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-rjmdz"]
Oct 30 18:55:22.708764 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:55:22.707916    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-3b6013/distinct-2-w3-3b6013-kserve-7cc5454db-mqknj" podUID="e76c7b20-be2f-4ed2-8dc4-bef127ecc0c9" containerName="main" probeResult="failure" output="Get \"https://10.132.0.98:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:55:22.734088 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:55:22.732589    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-4a7a52/premium-w2-4a7a52-kserve-746d9579f6-hn7hl" podUID="3160b54f-1df9-48de-80d9-8d8a91d47f56" containerName="main" probeResult="failure" output="Get \"https://10.132.0.83:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:55:23.083369 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:55:23.078072    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e7929d/distinct-w4-e7929d-kserve-5bc9ff5bf5-kmrwv" podUID="abfd315d-a01a-42dd-9c06-2a80d3f8579a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.88:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:55:25.433183 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:55:25.433147    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:55:25.433531 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:55:25.433232    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:55:25.872747 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:55:25.872710    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-x959p" event={"ID":"ff78b45c-608b-4562-99d5-d4cf35e59c71","Type":"ContainerStarted","Data":"7820fbcdf5394f25ef2fc531ceb664c18fdcf0a10a25f309ff353f2aeaa0642e"}
Oct 30 18:55:25.872956 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:55:25.872938    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-x959p"
Oct 30 18:55:26.097440 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:55:26.097386    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-x959p" podStartSLOduration=2.467090525 podStartE2EDuration="10.097374471s" totalImagesPullingTime="7.630283946s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:55:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:55:26.046993355 +0000 UTC m=+1290.294399381" watchObservedRunningTime="2026-10-30 18:55:26.097374471 +0000 UTC m=+1290.344780494"
Oct 30 18:55:26.885489 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:55:26.885442    4888 patch_prober.go:29] interesting pod/istiod-openshift-gateway-7ddd9cfdc8-x959p container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 18:55:26.885973 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:55:26.885512    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-x959p" podUID="ff78b45c-608b-4562-99d5-d4cf35e59c71" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 18:55:29.892416 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:55:29.892386    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-x959p"
Oct 30 18:55:35.645043 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:55:35.644956    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-jbsw6" containerName="extractor"
Oct 30 18:55:37.631949 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:55:37.631906    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-3b6013/distinct-w3-3b6013-kserve-857f55d885-8fcrq" podUID="51927877-4d99-4797-82f7-1684f6ecda6b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.92:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:55:37.706531 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:55:37.706483    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-3b6013/distinct-2-w3-3b6013-kserve-7cc5454db-mqknj" podUID="e76c7b20-be2f-4ed2-8dc4-bef127ecc0c9" containerName="main" probeResult="failure" output="Get \"https://10.132.0.98:8000/ready\": context deadline exceeded"
Oct 30 18:55:37.719439 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:55:37.719365    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-4a7a52/premium-w2-4a7a52-kserve-746d9579f6-hn7hl" podUID="3160b54f-1df9-48de-80d9-8d8a91d47f56" containerName="main" probeResult="failure" output="Get \"https://10.132.0.83:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:55:38.090490 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:55:38.090432    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e7929d/distinct-w4-e7929d-kserve-5bc9ff5bf5-kmrwv" podUID="abfd315d-a01a-42dd-9c06-2a80d3f8579a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.88:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:55:38.235439 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:55:38.235393    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e7929d/distinct-2-w4-e7929d-kserve-5b479cc87b-htv9p" podUID="f6df655d-86e8-44ff-ace8-27440f56294f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.91:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:55:38.320478 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:55:38.319316    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-rkm4t" podUID="1ab20f74-2c73-4d56-a604-aad4f8289450" containerName="main" probeResult="failure" output="Get \"https://10.132.0.53:8000/ready\": context deadline exceeded"
Oct 30 18:55:38.437557 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:55:38.437455    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-3b6013/premium-w3-3b6013-kserve-74df5fb95d-k747z" podUID="a9309776-20d5-4c06-bcae-04b8e54493e8" containerName="main" probeResult="failure" output="Get \"https://10.132.0.82:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:55:45.080928 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:55:45.080861    4888 generic.go:356] "Generic (PLEG): container finished" podID="427c5128-a124-4dd6-9748-a4dca0a85a4f" containerID="096083c6914a7bf98e5b8b3f42b9e4584ee0f3434a1d4dbf182585688b1ff6ef" exitCode=0
Oct 30 18:55:45.080928 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:55:45.080914    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-9k7xv" event={"ID":"427c5128-a124-4dd6-9748-a4dca0a85a4f","Type":"ContainerDied","Data":"096083c6914a7bf98e5b8b3f42b9e4584ee0f3434a1d4dbf182585688b1ff6ef"}
Oct 30 18:55:45.081618 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:55:45.081419    4888 scope.go:122] "RemoveContainer" containerID="096083c6914a7bf98e5b8b3f42b9e4584ee0f3434a1d4dbf182585688b1ff6ef"
Oct 30 18:55:45.083031 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:55:45.083005    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:55:45.083096 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:55:45.083087    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:55:45.083152 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:55:45.083125    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:55:46.103825 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:55:46.103795    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-9k7xv" event={"ID":"427c5128-a124-4dd6-9748-a4dca0a85a4f","Type":"ContainerStarted","Data":"c5011ae29c57bbfe0a14858cc96b1fe1836fcf5bab80c336877452bfa7ab26b2"}
Oct 30 18:55:46.742493 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:55:46.742462    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-9k7xv"
Oct 30 18:55:46.743460 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:55:46.743433    4888 patch_prober.go:29] interesting pod/e2e-mt-66baf45c-openshift-default-974f9d7b9-9k7xv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.100:15021/healthz/ready\": dial tcp 10.132.0.100:15021: connect: connection refused" start-of-body=
Oct 30 18:55:46.743680 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:55:46.743655    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-9k7xv" podUID="427c5128-a124-4dd6-9748-a4dca0a85a4f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.100:15021/healthz/ready\": dial tcp 10.132.0.100:15021: connect: connection refused"
Oct 30 18:55:47.728365 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:55:47.728331    4888 patch_prober.go:29] interesting pod/e2e-mt-66baf45c-openshift-default-974f9d7b9-9k7xv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.100:15021/healthz/ready\": dial tcp 10.132.0.100:15021: connect: connection refused" start-of-body=
Oct 30 18:55:47.728755 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:55:47.728392    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-9k7xv" podUID="427c5128-a124-4dd6-9748-a4dca0a85a4f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.100:15021/healthz/ready\": dial tcp 10.132.0.100:15021: connect: connection refused"
Oct 30 18:55:48.728707 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:55:48.728669    4888 patch_prober.go:29] interesting pod/e2e-mt-66baf45c-openshift-default-974f9d7b9-9k7xv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.100:15021/healthz/ready\": dial tcp 10.132.0.100:15021: connect: connection refused" start-of-body=
Oct 30 18:55:48.729117 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:55:48.728744    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-9k7xv" podUID="427c5128-a124-4dd6-9748-a4dca0a85a4f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.100:15021/healthz/ready\": dial tcp 10.132.0.100:15021: connect: connection refused"
Oct 30 18:55:49.728203 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:55:49.728170    4888 patch_prober.go:29] interesting pod/e2e-mt-66baf45c-openshift-default-974f9d7b9-9k7xv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.100:15021/healthz/ready\": dial tcp 10.132.0.100:15021: connect: connection refused" start-of-body=
Oct 30 18:55:49.728453 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:55:49.728244    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-9k7xv" podUID="427c5128-a124-4dd6-9748-a4dca0a85a4f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.100:15021/healthz/ready\": dial tcp 10.132.0.100:15021: connect: connection refused"
Oct 30 18:55:50.728454 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:55:50.728425    4888 patch_prober.go:29] interesting pod/e2e-mt-66baf45c-openshift-default-974f9d7b9-9k7xv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.100:15021/healthz/ready\": dial tcp 10.132.0.100:15021: connect: connection refused" start-of-body=
Oct 30 18:55:50.728843 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:55:50.728474    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-9k7xv" podUID="427c5128-a124-4dd6-9748-a4dca0a85a4f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.100:15021/healthz/ready\": dial tcp 10.132.0.100:15021: connect: connection refused"
Oct 30 18:55:51.732536 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:55:51.732510    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-9k7xv"
Oct 30 18:55:51.732935 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:55:51.732916    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-9k7xv"
Oct 30 18:55:51.733489 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:55:51.733470    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-9k7xv"
Oct 30 18:55:57.653590 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:55:57.653561    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-rjmdz"]
Oct 30 18:56:05.618375 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:56:05.618336    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-jbsw6" containerName="extractor"
Oct 30 18:56:25.651130 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:56:25.651094    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-26cqt"]
Oct 30 18:56:25.655549 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:56:25.655527    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-26cqt"
Oct 30 18:56:25.763987 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:56:25.763958    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-26cqt"]
Oct 30 18:56:25.765324 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:56:25.765276    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/93de7d1c-bc7a-499d-97f6-3320e2e4e948-istio-podinfo\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-26cqt\" (UID: \"93de7d1c-bc7a-499d-97f6-3320e2e4e948\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-26cqt"
Oct 30 18:56:25.765459 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:56:25.765365    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/93de7d1c-bc7a-499d-97f6-3320e2e4e948-istio-token\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-26cqt\" (UID: \"93de7d1c-bc7a-499d-97f6-3320e2e4e948\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-26cqt"
Oct 30 18:56:25.765459 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:56:25.765433    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/93de7d1c-bc7a-499d-97f6-3320e2e4e948-workload-certs\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-26cqt\" (UID: \"93de7d1c-bc7a-499d-97f6-3320e2e4e948\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-26cqt"
Oct 30 18:56:25.765560 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:56:25.765489    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/93de7d1c-bc7a-499d-97f6-3320e2e4e948-credential-socket\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-26cqt\" (UID: \"93de7d1c-bc7a-499d-97f6-3320e2e4e948\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-26cqt"
Oct 30 18:56:25.765786 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:56:25.765569    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/93de7d1c-bc7a-499d-97f6-3320e2e4e948-workload-socket\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-26cqt\" (UID: \"93de7d1c-bc7a-499d-97f6-3320e2e4e948\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-26cqt"
Oct 30 18:56:25.765786 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:56:25.765585    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/93de7d1c-bc7a-499d-97f6-3320e2e4e948-istio-envoy\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-26cqt\" (UID: \"93de7d1c-bc7a-499d-97f6-3320e2e4e948\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-26cqt"
Oct 30 18:56:25.765786 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:56:25.765637    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/93de7d1c-bc7a-499d-97f6-3320e2e4e948-istio-data\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-26cqt\" (UID: \"93de7d1c-bc7a-499d-97f6-3320e2e4e948\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-26cqt"
Oct 30 18:56:25.765786 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:56:25.765709    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/93de7d1c-bc7a-499d-97f6-3320e2e4e948-istiod-ca-cert\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-26cqt\" (UID: \"93de7d1c-bc7a-499d-97f6-3320e2e4e948\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-26cqt"
Oct 30 18:56:25.765786 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:56:25.765778    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dqkzq\" (UniqueName: \"kubernetes.io/projected/93de7d1c-bc7a-499d-97f6-3320e2e4e948-kube-api-access-dqkzq\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-26cqt\" (UID: \"93de7d1c-bc7a-499d-97f6-3320e2e4e948\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-26cqt"
Oct 30 18:56:25.866992 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:56:25.866966    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/93de7d1c-bc7a-499d-97f6-3320e2e4e948-istio-podinfo\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-26cqt\" (UID: \"93de7d1c-bc7a-499d-97f6-3320e2e4e948\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-26cqt"
Oct 30 18:56:25.866992 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:56:25.866996    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/93de7d1c-bc7a-499d-97f6-3320e2e4e948-istio-token\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-26cqt\" (UID: \"93de7d1c-bc7a-499d-97f6-3320e2e4e948\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-26cqt"
Oct 30 18:56:25.867237 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:56:25.867026    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/93de7d1c-bc7a-499d-97f6-3320e2e4e948-workload-certs\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-26cqt\" (UID: \"93de7d1c-bc7a-499d-97f6-3320e2e4e948\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-26cqt"
Oct 30 18:56:25.867237 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:56:25.867074    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/93de7d1c-bc7a-499d-97f6-3320e2e4e948-credential-socket\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-26cqt\" (UID: \"93de7d1c-bc7a-499d-97f6-3320e2e4e948\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-26cqt"
Oct 30 18:56:25.867237 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:56:25.867140    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/93de7d1c-bc7a-499d-97f6-3320e2e4e948-workload-socket\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-26cqt\" (UID: \"93de7d1c-bc7a-499d-97f6-3320e2e4e948\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-26cqt"
Oct 30 18:56:25.867237 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:56:25.867157    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/93de7d1c-bc7a-499d-97f6-3320e2e4e948-istio-envoy\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-26cqt\" (UID: \"93de7d1c-bc7a-499d-97f6-3320e2e4e948\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-26cqt"
Oct 30 18:56:25.867237 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:56:25.867178    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/93de7d1c-bc7a-499d-97f6-3320e2e4e948-istio-data\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-26cqt\" (UID: \"93de7d1c-bc7a-499d-97f6-3320e2e4e948\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-26cqt"
Oct 30 18:56:25.867237 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:56:25.867194    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/93de7d1c-bc7a-499d-97f6-3320e2e4e948-istiod-ca-cert\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-26cqt\" (UID: \"93de7d1c-bc7a-499d-97f6-3320e2e4e948\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-26cqt"
Oct 30 18:56:25.867237 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:56:25.867234    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dqkzq\" (UniqueName: \"kubernetes.io/projected/93de7d1c-bc7a-499d-97f6-3320e2e4e948-kube-api-access-dqkzq\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-26cqt\" (UID: \"93de7d1c-bc7a-499d-97f6-3320e2e4e948\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-26cqt"
Oct 30 18:56:25.867630 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:56:25.867549    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/93de7d1c-bc7a-499d-97f6-3320e2e4e948-workload-certs\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-26cqt\" (UID: \"93de7d1c-bc7a-499d-97f6-3320e2e4e948\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-26cqt"
Oct 30 18:56:25.867630 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:56:25.867615    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/93de7d1c-bc7a-499d-97f6-3320e2e4e948-workload-socket\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-26cqt\" (UID: \"93de7d1c-bc7a-499d-97f6-3320e2e4e948\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-26cqt"
Oct 30 18:56:25.867738 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:56:25.867698    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/93de7d1c-bc7a-499d-97f6-3320e2e4e948-istio-data\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-26cqt\" (UID: \"93de7d1c-bc7a-499d-97f6-3320e2e4e948\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-26cqt"
Oct 30 18:56:25.867834 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:56:25.867807    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/93de7d1c-bc7a-499d-97f6-3320e2e4e948-credential-socket\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-26cqt\" (UID: \"93de7d1c-bc7a-499d-97f6-3320e2e4e948\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-26cqt"
Oct 30 18:56:25.868159 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:56:25.868138    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/93de7d1c-bc7a-499d-97f6-3320e2e4e948-istiod-ca-cert\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-26cqt\" (UID: \"93de7d1c-bc7a-499d-97f6-3320e2e4e948\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-26cqt"
Oct 30 18:56:25.869682 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:56:25.869660    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/93de7d1c-bc7a-499d-97f6-3320e2e4e948-istio-envoy\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-26cqt\" (UID: \"93de7d1c-bc7a-499d-97f6-3320e2e4e948\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-26cqt"
Oct 30 18:56:25.870419 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:56:25.870399    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/93de7d1c-bc7a-499d-97f6-3320e2e4e948-istio-podinfo\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-26cqt\" (UID: \"93de7d1c-bc7a-499d-97f6-3320e2e4e948\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-26cqt"
Oct 30 18:56:25.929666 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:56:25.929607    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/93de7d1c-bc7a-499d-97f6-3320e2e4e948-istio-token\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-26cqt\" (UID: \"93de7d1c-bc7a-499d-97f6-3320e2e4e948\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-26cqt"
Oct 30 18:56:25.929830 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:56:25.929810    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dqkzq\" (UniqueName: \"kubernetes.io/projected/93de7d1c-bc7a-499d-97f6-3320e2e4e948-kube-api-access-dqkzq\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-26cqt\" (UID: \"93de7d1c-bc7a-499d-97f6-3320e2e4e948\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-26cqt"
Oct 30 18:56:25.969912 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:56:25.969886    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-26cqt"
Oct 30 18:56:26.403710 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:56:26.403681    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93de7d1c_bc7a_499d_97f6_3320e2e4e948.slice/crio-2015051f1f4c05f64b2a3a9459b5b65cf7d87ecdb219f150cb0c5f4d95951307 WatchSource:0}: Error finding container 2015051f1f4c05f64b2a3a9459b5b65cf7d87ecdb219f150cb0c5f4d95951307: Status 404 returned error can't find the container with id 2015051f1f4c05f64b2a3a9459b5b65cf7d87ecdb219f150cb0c5f4d95951307
Oct 30 18:56:26.405859 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:56:26.405836    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-26cqt"]
Oct 30 18:56:26.406815 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:56:26.406779    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:56:26.406925 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:56:26.406863    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:56:26.406925 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:56:26.406903    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:56:26.413682 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:56:26.413661    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-26cqt" event={"ID":"93de7d1c-bc7a-499d-97f6-3320e2e4e948","Type":"ContainerStarted","Data":"2015051f1f4c05f64b2a3a9459b5b65cf7d87ecdb219f150cb0c5f4d95951307"}
Oct 30 18:56:27.423428 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:56:27.423388    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-26cqt" event={"ID":"93de7d1c-bc7a-499d-97f6-3320e2e4e948","Type":"ContainerStarted","Data":"3e98e7afa364c552b0b15f972202160c04c2b8deae9efa1e9e19fef9b57dc4ed"}
Oct 30 18:56:27.660332 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:56:27.660264    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-26cqt" podStartSLOduration=2.660250877 podStartE2EDuration="2.660250877s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:56:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:56:27.60835328 +0000 UTC m=+1351.855759306" watchObservedRunningTime="2026-10-30 18:56:27.660250877 +0000 UTC m=+1351.907656896"
Oct 30 18:56:27.973616 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:56:27.973576    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-26cqt"
Oct 30 18:56:27.975359 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:56:27.975329    4888 patch_prober.go:29] interesting pod/e2e-worker-w0-2ade09-openshift-default-785b7cf675-26cqt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.104:15021/healthz/ready\": dial tcp 10.132.0.104:15021: connect: connection refused" start-of-body=
Oct 30 18:56:27.975563 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:56:27.975534    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-26cqt" podUID="93de7d1c-bc7a-499d-97f6-3320e2e4e948" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.104:15021/healthz/ready\": dial tcp 10.132.0.104:15021: connect: connection refused"
Oct 30 18:56:28.971310 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:56:28.970626    4888 patch_prober.go:29] interesting pod/e2e-worker-w0-2ade09-openshift-default-785b7cf675-26cqt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.104:15021/healthz/ready\": dial tcp 10.132.0.104:15021: connect: connection refused" start-of-body=
Oct 30 18:56:28.971310 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:56:28.970683    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-26cqt" podUID="93de7d1c-bc7a-499d-97f6-3320e2e4e948" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.104:15021/healthz/ready\": dial tcp 10.132.0.104:15021: connect: connection refused"
Oct 30 18:56:29.988366 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:56:29.971641    4888 patch_prober.go:29] interesting pod/e2e-worker-w0-2ade09-openshift-default-785b7cf675-26cqt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.104:15021/healthz/ready\": dial tcp 10.132.0.104:15021: connect: connection refused" start-of-body=
Oct 30 18:56:29.988366 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:56:29.971701    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-26cqt" podUID="93de7d1c-bc7a-499d-97f6-3320e2e4e948" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.104:15021/healthz/ready\": dial tcp 10.132.0.104:15021: connect: connection refused"
Oct 30 18:56:30.979309 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:56:30.971602    4888 patch_prober.go:29] interesting pod/e2e-worker-w0-2ade09-openshift-default-785b7cf675-26cqt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.104:15021/healthz/ready\": dial tcp 10.132.0.104:15021: connect: connection refused" start-of-body=
Oct 30 18:56:30.979309 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:56:30.971668    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-26cqt" podUID="93de7d1c-bc7a-499d-97f6-3320e2e4e948" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.104:15021/healthz/ready\": dial tcp 10.132.0.104:15021: connect: connection refused"
Oct 30 18:56:31.998688 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:56:31.998647    4888 patch_prober.go:29] interesting pod/e2e-worker-w0-2ade09-openshift-default-785b7cf675-26cqt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.104:15021/healthz/ready\": dial tcp 10.132.0.104:15021: connect: connection refused" start-of-body=
Oct 30 18:56:31.999191 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:56:31.998705    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-26cqt" podUID="93de7d1c-bc7a-499d-97f6-3320e2e4e948" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.104:15021/healthz/ready\": dial tcp 10.132.0.104:15021: connect: connection refused"
Oct 30 18:56:32.971161 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:56:32.971124    4888 patch_prober.go:29] interesting pod/e2e-worker-w0-2ade09-openshift-default-785b7cf675-26cqt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.104:15021/healthz/ready\": dial tcp 10.132.0.104:15021: connect: connection refused" start-of-body=
Oct 30 18:56:32.971420 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:56:32.971175    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-26cqt" podUID="93de7d1c-bc7a-499d-97f6-3320e2e4e948" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.104:15021/healthz/ready\": dial tcp 10.132.0.104:15021: connect: connection refused"
Oct 30 18:56:33.974815 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:56:33.974786    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-26cqt"
Oct 30 18:56:33.975322 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:56:33.975164    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-26cqt"
Oct 30 18:56:33.975901 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:56:33.975882    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-26cqt"
Oct 30 18:56:35.618513 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:56:35.618469    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-jbsw6" containerName="extractor"
Oct 30 18:56:45.320979 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:56:45.320939    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-rjmdz"]
Oct 30 18:57:05.618422 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:57:05.618337    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-jbsw6" containerName="extractor"
Oct 30 18:57:11.007740 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:57:11.007709    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-8sg49"]
Oct 30 18:57:11.013119 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:57:11.013100    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-8sg49"
Oct 30 18:57:11.075611 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:57:11.075583    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/863c528e-6d99-4867-bbff-02bdc504c923-istio-podinfo\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-8sg49\" (UID: \"863c528e-6d99-4867-bbff-02bdc504c923\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-8sg49"
Oct 30 18:57:11.075735 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:57:11.075630    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/863c528e-6d99-4867-bbff-02bdc504c923-istio-data\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-8sg49\" (UID: \"863c528e-6d99-4867-bbff-02bdc504c923\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-8sg49"
Oct 30 18:57:11.075735 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:57:11.075648    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kdk88\" (UniqueName: \"kubernetes.io/projected/863c528e-6d99-4867-bbff-02bdc504c923-kube-api-access-kdk88\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-8sg49\" (UID: \"863c528e-6d99-4867-bbff-02bdc504c923\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-8sg49"
Oct 30 18:57:11.075845 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:57:11.075747    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/863c528e-6d99-4867-bbff-02bdc504c923-istio-envoy\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-8sg49\" (UID: \"863c528e-6d99-4867-bbff-02bdc504c923\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-8sg49"
Oct 30 18:57:11.075845 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:57:11.075806    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/863c528e-6d99-4867-bbff-02bdc504c923-istiod-ca-cert\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-8sg49\" (UID: \"863c528e-6d99-4867-bbff-02bdc504c923\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-8sg49"
Oct 30 18:57:11.075929 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:57:11.075868    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/863c528e-6d99-4867-bbff-02bdc504c923-workload-socket\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-8sg49\" (UID: \"863c528e-6d99-4867-bbff-02bdc504c923\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-8sg49"
Oct 30 18:57:11.075974 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:57:11.075932    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/863c528e-6d99-4867-bbff-02bdc504c923-credential-socket\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-8sg49\" (UID: \"863c528e-6d99-4867-bbff-02bdc504c923\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-8sg49"
Oct 30 18:57:11.076010 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:57:11.075984    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/863c528e-6d99-4867-bbff-02bdc504c923-istio-token\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-8sg49\" (UID: \"863c528e-6d99-4867-bbff-02bdc504c923\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-8sg49"
Oct 30 18:57:11.076065 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:57:11.076027    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/863c528e-6d99-4867-bbff-02bdc504c923-workload-certs\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-8sg49\" (UID: \"863c528e-6d99-4867-bbff-02bdc504c923\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-8sg49"
Oct 30 18:57:11.176808 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:57:11.176774    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/863c528e-6d99-4867-bbff-02bdc504c923-workload-socket\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-8sg49\" (UID: \"863c528e-6d99-4867-bbff-02bdc504c923\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-8sg49"
Oct 30 18:57:11.176985 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:57:11.176832    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/863c528e-6d99-4867-bbff-02bdc504c923-credential-socket\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-8sg49\" (UID: \"863c528e-6d99-4867-bbff-02bdc504c923\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-8sg49"
Oct 30 18:57:11.176985 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:57:11.176892    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/863c528e-6d99-4867-bbff-02bdc504c923-istio-token\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-8sg49\" (UID: \"863c528e-6d99-4867-bbff-02bdc504c923\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-8sg49"
Oct 30 18:57:11.176985 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:57:11.176950    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/863c528e-6d99-4867-bbff-02bdc504c923-workload-certs\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-8sg49\" (UID: \"863c528e-6d99-4867-bbff-02bdc504c923\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-8sg49"
Oct 30 18:57:11.177159 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:57:11.177072    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/863c528e-6d99-4867-bbff-02bdc504c923-istio-podinfo\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-8sg49\" (UID: \"863c528e-6d99-4867-bbff-02bdc504c923\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-8sg49"
Oct 30 18:57:11.177159 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:57:11.177130    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/863c528e-6d99-4867-bbff-02bdc504c923-istio-data\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-8sg49\" (UID: \"863c528e-6d99-4867-bbff-02bdc504c923\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-8sg49"
Oct 30 18:57:11.177263 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:57:11.177156    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kdk88\" (UniqueName: \"kubernetes.io/projected/863c528e-6d99-4867-bbff-02bdc504c923-kube-api-access-kdk88\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-8sg49\" (UID: \"863c528e-6d99-4867-bbff-02bdc504c923\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-8sg49"
Oct 30 18:57:11.177263 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:57:11.177186    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/863c528e-6d99-4867-bbff-02bdc504c923-istio-envoy\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-8sg49\" (UID: \"863c528e-6d99-4867-bbff-02bdc504c923\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-8sg49"
Oct 30 18:57:11.177263 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:57:11.177227    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/863c528e-6d99-4867-bbff-02bdc504c923-workload-socket\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-8sg49\" (UID: \"863c528e-6d99-4867-bbff-02bdc504c923\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-8sg49"
Oct 30 18:57:11.177502 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:57:11.177267    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/863c528e-6d99-4867-bbff-02bdc504c923-istiod-ca-cert\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-8sg49\" (UID: \"863c528e-6d99-4867-bbff-02bdc504c923\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-8sg49"
Oct 30 18:57:11.177895 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:57:11.177861    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/863c528e-6d99-4867-bbff-02bdc504c923-workload-certs\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-8sg49\" (UID: \"863c528e-6d99-4867-bbff-02bdc504c923\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-8sg49"
Oct 30 18:57:11.178030 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:57:11.178012    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/863c528e-6d99-4867-bbff-02bdc504c923-istio-data\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-8sg49\" (UID: \"863c528e-6d99-4867-bbff-02bdc504c923\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-8sg49"
Oct 30 18:57:11.178156 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:57:11.178082    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/863c528e-6d99-4867-bbff-02bdc504c923-credential-socket\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-8sg49\" (UID: \"863c528e-6d99-4867-bbff-02bdc504c923\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-8sg49"
Oct 30 18:57:11.178266 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:57:11.178207    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/863c528e-6d99-4867-bbff-02bdc504c923-istiod-ca-cert\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-8sg49\" (UID: \"863c528e-6d99-4867-bbff-02bdc504c923\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-8sg49"
Oct 30 18:57:11.180505 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:57:11.180473    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/863c528e-6d99-4867-bbff-02bdc504c923-istio-envoy\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-8sg49\" (UID: \"863c528e-6d99-4867-bbff-02bdc504c923\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-8sg49"
Oct 30 18:57:11.180844 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:57:11.180815    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/863c528e-6d99-4867-bbff-02bdc504c923-istio-podinfo\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-8sg49\" (UID: \"863c528e-6d99-4867-bbff-02bdc504c923\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-8sg49"
Oct 30 18:57:11.229334 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:57:11.229277    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-8sg49"]
Oct 30 18:57:11.240317 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:57:11.240259    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/863c528e-6d99-4867-bbff-02bdc504c923-istio-token\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-8sg49\" (UID: \"863c528e-6d99-4867-bbff-02bdc504c923\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-8sg49"
Oct 30 18:57:11.240493 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:57:11.240465    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kdk88\" (UniqueName: \"kubernetes.io/projected/863c528e-6d99-4867-bbff-02bdc504c923-kube-api-access-kdk88\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-8sg49\" (UID: \"863c528e-6d99-4867-bbff-02bdc504c923\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-8sg49"
Oct 30 18:57:11.327139 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:57:11.327104    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-8sg49"
Oct 30 18:57:11.815690 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:57:11.815659    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-8sg49"]
Oct 30 18:57:11.816279 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:57:11.816253    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod863c528e_6d99_4867_bbff_02bdc504c923.slice/crio-52a92e271aa6358e9c96015bda2272c78c7a88ab5f3726f0501a82792b6e48d3 WatchSource:0}: Error finding container 52a92e271aa6358e9c96015bda2272c78c7a88ab5f3726f0501a82792b6e48d3: Status 404 returned error can't find the container with id 52a92e271aa6358e9c96015bda2272c78c7a88ab5f3726f0501a82792b6e48d3
Oct 30 18:57:11.819621 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:57:11.819589    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:57:11.819711 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:57:11.819657    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:57:11.819711 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:57:11.819689    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 18:57:11.969316 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:57:11.968016    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-8sg49" event={"ID":"863c528e-6d99-4867-bbff-02bdc504c923","Type":"ContainerStarted","Data":"ae9b4220c08c1c6b8b0ad2f8eb31f638f52579d81c08ffc2d0a288a2c2f6dd87"}
Oct 30 18:57:11.969316 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:57:11.968079    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-8sg49" event={"ID":"863c528e-6d99-4867-bbff-02bdc504c923","Type":"ContainerStarted","Data":"52a92e271aa6358e9c96015bda2272c78c7a88ab5f3726f0501a82792b6e48d3"}
Oct 30 18:57:12.091954 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:57:12.091887    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-8sg49" podStartSLOduration=2.091869857 podStartE2EDuration="2.091869857s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:57:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:57:12.090335829 +0000 UTC m=+1396.337741855" watchObservedRunningTime="2026-10-30 18:57:12.091869857 +0000 UTC m=+1396.339275884"
Oct 30 18:57:12.328609 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:57:12.328573    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-8sg49"
Oct 30 18:57:12.330049 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:57:12.330022    4888 patch_prober.go:29] interesting pod/e2e-worker-w0-2ade09-openshift-default-785b7cf675-8sg49 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.105:15021/healthz/ready\": dial tcp 10.132.0.105:15021: connect: connection refused" start-of-body=
Oct 30 18:57:12.330135 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:57:12.330104    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-8sg49" podUID="863c528e-6d99-4867-bbff-02bdc504c923" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.105:15021/healthz/ready\": dial tcp 10.132.0.105:15021: connect: connection refused"
Oct 30 18:57:13.339228 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:57:13.339192    4888 patch_prober.go:29] interesting pod/e2e-worker-w0-2ade09-openshift-default-785b7cf675-8sg49 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.105:15021/healthz/ready\": dial tcp 10.132.0.105:15021: connect: connection refused" start-of-body=
Oct 30 18:57:13.339879 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:57:13.339847    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-8sg49" podUID="863c528e-6d99-4867-bbff-02bdc504c923" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.105:15021/healthz/ready\": dial tcp 10.132.0.105:15021: connect: connection refused"
Oct 30 18:57:14.328223 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:57:14.328189    4888 patch_prober.go:29] interesting pod/e2e-worker-w0-2ade09-openshift-default-785b7cf675-8sg49 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.105:15021/healthz/ready\": dial tcp 10.132.0.105:15021: connect: connection refused" start-of-body=
Oct 30 18:57:14.328415 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:57:14.328270    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-8sg49" podUID="863c528e-6d99-4867-bbff-02bdc504c923" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.105:15021/healthz/ready\": dial tcp 10.132.0.105:15021: connect: connection refused"
Oct 30 18:57:15.327920 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:57:15.327879    4888 patch_prober.go:29] interesting pod/e2e-worker-w0-2ade09-openshift-default-785b7cf675-8sg49 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.105:15021/healthz/ready\": dial tcp 10.132.0.105:15021: connect: connection refused" start-of-body=
Oct 30 18:57:15.328432 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:57:15.327933    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-8sg49" podUID="863c528e-6d99-4867-bbff-02bdc504c923" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.105:15021/healthz/ready\": dial tcp 10.132.0.105:15021: connect: connection refused"
Oct 30 18:57:16.328625 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:57:16.328577    4888 patch_prober.go:29] interesting pod/e2e-worker-w0-2ade09-openshift-default-785b7cf675-8sg49 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.105:15021/healthz/ready\": dial tcp 10.132.0.105:15021: connect: connection refused" start-of-body=
Oct 30 18:57:16.329125 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:57:16.328654    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-8sg49" podUID="863c528e-6d99-4867-bbff-02bdc504c923" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.105:15021/healthz/ready\": dial tcp 10.132.0.105:15021: connect: connection refused"
Oct 30 18:57:17.328355 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:57:17.328319    4888 patch_prober.go:29] interesting pod/e2e-worker-w0-2ade09-openshift-default-785b7cf675-8sg49 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.105:15021/healthz/ready\": dial tcp 10.132.0.105:15021: connect: connection refused" start-of-body=
Oct 30 18:57:17.328611 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:57:17.328384    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-8sg49" podUID="863c528e-6d99-4867-bbff-02bdc504c923" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.105:15021/healthz/ready\": dial tcp 10.132.0.105:15021: connect: connection refused"
Oct 30 18:57:18.327970 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:57:18.327933    4888 patch_prober.go:29] interesting pod/e2e-worker-w0-2ade09-openshift-default-785b7cf675-8sg49 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.105:15021/healthz/ready\": dial tcp 10.132.0.105:15021: connect: connection refused" start-of-body=
Oct 30 18:57:18.328495 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:57:18.327991    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-8sg49" podUID="863c528e-6d99-4867-bbff-02bdc504c923" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.105:15021/healthz/ready\": dial tcp 10.132.0.105:15021: connect: connection refused"
Oct 30 18:57:19.327850 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:57:19.327811    4888 patch_prober.go:29] interesting pod/e2e-worker-w0-2ade09-openshift-default-785b7cf675-8sg49 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.105:15021/healthz/ready\": dial tcp 10.132.0.105:15021: connect: connection refused" start-of-body=
Oct 30 18:57:19.328187 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:57:19.328147    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-8sg49" podUID="863c528e-6d99-4867-bbff-02bdc504c923" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.105:15021/healthz/ready\": dial tcp 10.132.0.105:15021: connect: connection refused"
Oct 30 18:57:21.376518 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:57:21.376473    4888 patch_prober.go:29] interesting pod/e2e-worker-w0-2ade09-openshift-default-785b7cf675-8sg49 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.105:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 18:57:21.377049 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:57:21.376547    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-8sg49" podUID="863c528e-6d99-4867-bbff-02bdc504c923" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.105:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 18:57:21.705728 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:57:21.705659    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-8sg49"
Oct 30 18:57:21.706154 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:57:21.706132    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-8sg49"
Oct 30 18:57:21.707048 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:57:21.707025    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-8sg49"
Oct 30 18:57:35.619048 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:57:35.619009    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-jbsw6" containerName="extractor"
Oct 30 18:57:42.511940 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:57:42.511908    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-rjmdz"]
Oct 30 18:57:47.748656 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:57:47.748628    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-85ab6a-57bf4bc848-2lrb7"]
Oct 30 18:57:47.752852 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:57:47.752833    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-85ab6a-57bf4bc848-2lrb7"
Oct 30 18:57:47.852233 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:57:47.852203    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-85ab6a-57bf4bc848-2lrb7"]
Oct 30 18:57:47.875928 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:57:47.875899    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3ba1c3a8-aa3d-45d6-bb89-c00b9993596e-maas-api-tls\") pod \"maas-api-e2e-api-a-85ab6a-57bf4bc848-2lrb7\" (UID: \"3ba1c3a8-aa3d-45d6-bb89-c00b9993596e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-85ab6a-57bf4bc848-2lrb7"
Oct 30 18:57:47.876105 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:57:47.876083    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fm7ph\" (UniqueName: \"kubernetes.io/projected/3ba1c3a8-aa3d-45d6-bb89-c00b9993596e-kube-api-access-fm7ph\") pod \"maas-api-e2e-api-a-85ab6a-57bf4bc848-2lrb7\" (UID: \"3ba1c3a8-aa3d-45d6-bb89-c00b9993596e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-85ab6a-57bf4bc848-2lrb7"
Oct 30 18:57:47.876184 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:57:47.876125    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3ba1c3a8-aa3d-45d6-bb89-c00b9993596e-openshift-service-ca\") pod \"maas-api-e2e-api-a-85ab6a-57bf4bc848-2lrb7\" (UID: \"3ba1c3a8-aa3d-45d6-bb89-c00b9993596e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-85ab6a-57bf4bc848-2lrb7"
Oct 30 18:57:47.876184 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:57:47.876160    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3ba1c3a8-aa3d-45d6-bb89-c00b9993596e-metrics-certs\") pod \"maas-api-e2e-api-a-85ab6a-57bf4bc848-2lrb7\" (UID: \"3ba1c3a8-aa3d-45d6-bb89-c00b9993596e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-85ab6a-57bf4bc848-2lrb7"
Oct 30 18:57:47.900624 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:57:47.900591    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-85ab6a-serving-cert\""
Oct 30 18:57:47.977458 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:57:47.977419    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fm7ph\" (UniqueName: \"kubernetes.io/projected/3ba1c3a8-aa3d-45d6-bb89-c00b9993596e-kube-api-access-fm7ph\") pod \"maas-api-e2e-api-a-85ab6a-57bf4bc848-2lrb7\" (UID: \"3ba1c3a8-aa3d-45d6-bb89-c00b9993596e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-85ab6a-57bf4bc848-2lrb7"
Oct 30 18:57:47.977656 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:57:47.977490    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3ba1c3a8-aa3d-45d6-bb89-c00b9993596e-openshift-service-ca\") pod \"maas-api-e2e-api-a-85ab6a-57bf4bc848-2lrb7\" (UID: \"3ba1c3a8-aa3d-45d6-bb89-c00b9993596e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-85ab6a-57bf4bc848-2lrb7"
Oct 30 18:57:47.977656 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:57:47.977544    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3ba1c3a8-aa3d-45d6-bb89-c00b9993596e-metrics-certs\") pod \"maas-api-e2e-api-a-85ab6a-57bf4bc848-2lrb7\" (UID: \"3ba1c3a8-aa3d-45d6-bb89-c00b9993596e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-85ab6a-57bf4bc848-2lrb7"
Oct 30 18:57:47.977788 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:57:47.977725    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3ba1c3a8-aa3d-45d6-bb89-c00b9993596e-maas-api-tls\") pod \"maas-api-e2e-api-a-85ab6a-57bf4bc848-2lrb7\" (UID: \"3ba1c3a8-aa3d-45d6-bb89-c00b9993596e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-85ab6a-57bf4bc848-2lrb7"
Oct 30 18:57:47.978134 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:57:47.978090    4888 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-85ab6a-serving-cert: secret "maas-api-e2e-api-a-85ab6a-serving-cert" not found
Oct 30 18:57:47.978271 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:57:47.978189    4888 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3ba1c3a8-aa3d-45d6-bb89-c00b9993596e-maas-api-tls podName:3ba1c3a8-aa3d-45d6-bb89-c00b9993596e nodeName:}" failed. No retries permitted until 2026-10-30 18:57:48.478168983 +0000 UTC m=+1432.725574986 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/3ba1c3a8-aa3d-45d6-bb89-c00b9993596e-maas-api-tls") pod "maas-api-e2e-api-a-85ab6a-57bf4bc848-2lrb7" (UID: "3ba1c3a8-aa3d-45d6-bb89-c00b9993596e") : secret "maas-api-e2e-api-a-85ab6a-serving-cert" not found
Oct 30 18:57:47.982160 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:57:47.979594    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3ba1c3a8-aa3d-45d6-bb89-c00b9993596e-openshift-service-ca\") pod \"maas-api-e2e-api-a-85ab6a-57bf4bc848-2lrb7\" (UID: \"3ba1c3a8-aa3d-45d6-bb89-c00b9993596e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-85ab6a-57bf4bc848-2lrb7"
Oct 30 18:57:47.982160 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:57:47.982101    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3ba1c3a8-aa3d-45d6-bb89-c00b9993596e-metrics-certs\") pod \"maas-api-e2e-api-a-85ab6a-57bf4bc848-2lrb7\" (UID: \"3ba1c3a8-aa3d-45d6-bb89-c00b9993596e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-85ab6a-57bf4bc848-2lrb7"
Oct 30 18:57:48.043437 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:57:48.043375    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fm7ph\" (UniqueName: \"kubernetes.io/projected/3ba1c3a8-aa3d-45d6-bb89-c00b9993596e-kube-api-access-fm7ph\") pod \"maas-api-e2e-api-a-85ab6a-57bf4bc848-2lrb7\" (UID: \"3ba1c3a8-aa3d-45d6-bb89-c00b9993596e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-85ab6a-57bf4bc848-2lrb7"
Oct 30 18:57:48.487963 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:57:48.487934    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3ba1c3a8-aa3d-45d6-bb89-c00b9993596e-maas-api-tls\") pod \"maas-api-e2e-api-a-85ab6a-57bf4bc848-2lrb7\" (UID: \"3ba1c3a8-aa3d-45d6-bb89-c00b9993596e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-85ab6a-57bf4bc848-2lrb7"
Oct 30 18:57:48.490572 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:57:48.490548    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3ba1c3a8-aa3d-45d6-bb89-c00b9993596e-maas-api-tls\") pod \"maas-api-e2e-api-a-85ab6a-57bf4bc848-2lrb7\" (UID: \"3ba1c3a8-aa3d-45d6-bb89-c00b9993596e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-85ab6a-57bf4bc848-2lrb7"
Oct 30 18:57:48.667449 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:57:48.667419    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-85ab6a-57bf4bc848-2lrb7"
Oct 30 18:57:49.053983 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:57:49.053942    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-85ab6a-57bf4bc848-2lrb7"]
Oct 30 18:57:49.054471 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:57:49.054321    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ba1c3a8_aa3d_45d6_bb89_c00b9993596e.slice/crio-28e710fc9477c5974ebfea96abe3bc0fa27df6fa11295f5bf71ddc1482a9d337 WatchSource:0}: Error finding container 28e710fc9477c5974ebfea96abe3bc0fa27df6fa11295f5bf71ddc1482a9d337: Status 404 returned error can't find the container with id 28e710fc9477c5974ebfea96abe3bc0fa27df6fa11295f5bf71ddc1482a9d337
Oct 30 18:57:49.304906 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:57:49.304814    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-85ab6a-57bf4bc848-2lrb7" event={"ID":"3ba1c3a8-aa3d-45d6-bb89-c00b9993596e","Type":"ContainerStarted","Data":"28e710fc9477c5974ebfea96abe3bc0fa27df6fa11295f5bf71ddc1482a9d337"}
Oct 30 18:57:50.315258 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:57:50.315167    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-85ab6a-57bf4bc848-2lrb7" event={"ID":"3ba1c3a8-aa3d-45d6-bb89-c00b9993596e","Type":"ContainerStarted","Data":"32cc03c5edf83adbe0b8f775a7c9700ef088fecb7e4ad976f6c80d88ef041e09"}
Oct 30 18:57:50.315718 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:57:50.315266    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-85ab6a-57bf4bc848-2lrb7"
Oct 30 18:57:50.438456 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:57:50.438402    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-85ab6a-57bf4bc848-2lrb7" podStartSLOduration=2.53663975 podStartE2EDuration="3.438388188s" totalImagesPullingTime="901.748438ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:57:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:57:50.436010457 +0000 UTC m=+1434.683416481" watchObservedRunningTime="2026-10-30 18:57:50.438388188 +0000 UTC m=+1434.685794213"
Oct 30 18:57:56.331977 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:57:56.331950    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-85ab6a-57bf4bc848-2lrb7"
Oct 30 18:58:05.619830 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:58:05.619794    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-jbsw6" containerName="extractor"
Oct 30 18:58:26.030753 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:58:26.030718    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-rjmdz"]
Oct 30 18:58:34.357594 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:58:34.357562    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-83d5bd-64886885df-zj2nz"]
Oct 30 18:58:34.366353 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:58:34.366324    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-83d5bd-64886885df-zj2nz"
Oct 30 18:58:34.421941 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:58:34.421909    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-83d5bd-64886885df-zj2nz"]
Oct 30 18:58:34.451367 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:58:34.451338    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/998d1b1d-e656-49b0-8763-540ef647899d-openshift-service-ca\") pod \"maas-api-e2e-api-a-83d5bd-64886885df-zj2nz\" (UID: \"998d1b1d-e656-49b0-8763-540ef647899d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-83d5bd-64886885df-zj2nz"
Oct 30 18:58:34.451542 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:58:34.451409    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/998d1b1d-e656-49b0-8763-540ef647899d-metrics-certs\") pod \"maas-api-e2e-api-a-83d5bd-64886885df-zj2nz\" (UID: \"998d1b1d-e656-49b0-8763-540ef647899d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-83d5bd-64886885df-zj2nz"
Oct 30 18:58:34.451623 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:58:34.451549    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fpdss\" (UniqueName: \"kubernetes.io/projected/998d1b1d-e656-49b0-8763-540ef647899d-kube-api-access-fpdss\") pod \"maas-api-e2e-api-a-83d5bd-64886885df-zj2nz\" (UID: \"998d1b1d-e656-49b0-8763-540ef647899d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-83d5bd-64886885df-zj2nz"
Oct 30 18:58:34.451623 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:58:34.451610    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/998d1b1d-e656-49b0-8763-540ef647899d-maas-api-tls\") pod \"maas-api-e2e-api-a-83d5bd-64886885df-zj2nz\" (UID: \"998d1b1d-e656-49b0-8763-540ef647899d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-83d5bd-64886885df-zj2nz"
Oct 30 18:58:34.467800 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:58:34.467779    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-83d5bd-serving-cert\""
Oct 30 18:58:34.552661 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:58:34.552633    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/998d1b1d-e656-49b0-8763-540ef647899d-maas-api-tls\") pod \"maas-api-e2e-api-a-83d5bd-64886885df-zj2nz\" (UID: \"998d1b1d-e656-49b0-8763-540ef647899d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-83d5bd-64886885df-zj2nz"
Oct 30 18:58:34.552864 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:58:34.552780    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/998d1b1d-e656-49b0-8763-540ef647899d-openshift-service-ca\") pod \"maas-api-e2e-api-a-83d5bd-64886885df-zj2nz\" (UID: \"998d1b1d-e656-49b0-8763-540ef647899d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-83d5bd-64886885df-zj2nz"
Oct 30 18:58:34.552864 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:58:34.552809    4888 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-83d5bd-serving-cert: secret "maas-api-e2e-api-a-83d5bd-serving-cert" not found
Oct 30 18:58:34.552864 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:58:34.552835    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/998d1b1d-e656-49b0-8763-540ef647899d-metrics-certs\") pod \"maas-api-e2e-api-a-83d5bd-64886885df-zj2nz\" (UID: \"998d1b1d-e656-49b0-8763-540ef647899d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-83d5bd-64886885df-zj2nz"
Oct 30 18:58:34.553044 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:58:34.552893    4888 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/998d1b1d-e656-49b0-8763-540ef647899d-maas-api-tls podName:998d1b1d-e656-49b0-8763-540ef647899d nodeName:}" failed. No retries permitted until 2026-10-30 18:58:35.052871581 +0000 UTC m=+1479.300277599 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/998d1b1d-e656-49b0-8763-540ef647899d-maas-api-tls") pod "maas-api-e2e-api-a-83d5bd-64886885df-zj2nz" (UID: "998d1b1d-e656-49b0-8763-540ef647899d") : secret "maas-api-e2e-api-a-83d5bd-serving-cert" not found
Oct 30 18:58:34.553044 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:58:34.552938    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fpdss\" (UniqueName: \"kubernetes.io/projected/998d1b1d-e656-49b0-8763-540ef647899d-kube-api-access-fpdss\") pod \"maas-api-e2e-api-a-83d5bd-64886885df-zj2nz\" (UID: \"998d1b1d-e656-49b0-8763-540ef647899d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-83d5bd-64886885df-zj2nz"
Oct 30 18:58:34.553462 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:58:34.553444    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/998d1b1d-e656-49b0-8763-540ef647899d-openshift-service-ca\") pod \"maas-api-e2e-api-a-83d5bd-64886885df-zj2nz\" (UID: \"998d1b1d-e656-49b0-8763-540ef647899d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-83d5bd-64886885df-zj2nz"
Oct 30 18:58:34.555488 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:58:34.555470    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/998d1b1d-e656-49b0-8763-540ef647899d-metrics-certs\") pod \"maas-api-e2e-api-a-83d5bd-64886885df-zj2nz\" (UID: \"998d1b1d-e656-49b0-8763-540ef647899d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-83d5bd-64886885df-zj2nz"
Oct 30 18:58:34.619680 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:58:34.619611    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fpdss\" (UniqueName: \"kubernetes.io/projected/998d1b1d-e656-49b0-8763-540ef647899d-kube-api-access-fpdss\") pod \"maas-api-e2e-api-a-83d5bd-64886885df-zj2nz\" (UID: \"998d1b1d-e656-49b0-8763-540ef647899d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-83d5bd-64886885df-zj2nz"
Oct 30 18:58:35.066981 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:58:35.066877    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/998d1b1d-e656-49b0-8763-540ef647899d-maas-api-tls\") pod \"maas-api-e2e-api-a-83d5bd-64886885df-zj2nz\" (UID: \"998d1b1d-e656-49b0-8763-540ef647899d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-83d5bd-64886885df-zj2nz"
Oct 30 18:58:35.071924 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:58:35.071893    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/998d1b1d-e656-49b0-8763-540ef647899d-maas-api-tls\") pod \"maas-api-e2e-api-a-83d5bd-64886885df-zj2nz\" (UID: \"998d1b1d-e656-49b0-8763-540ef647899d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-83d5bd-64886885df-zj2nz"
Oct 30 18:58:35.291665 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:58:35.291631    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-83d5bd-64886885df-zj2nz"
Oct 30 18:58:35.619871 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:58:35.619790    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-jbsw6" containerName="extractor"
Oct 30 18:58:35.738387 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:58:35.738362    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-83d5bd-64886885df-zj2nz"]
Oct 30 18:58:35.741413 ip-10-0-134-46 kubenswrapper[4888]: W1030 18:58:35.741388    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod998d1b1d_e656_49b0_8763_540ef647899d.slice/crio-da0ef95b80f29ba342578b597185f930ef1df84b97eb2ab3be729d8ac81168f7 WatchSource:0}: Error finding container da0ef95b80f29ba342578b597185f930ef1df84b97eb2ab3be729d8ac81168f7: Status 404 returned error can't find the container with id da0ef95b80f29ba342578b597185f930ef1df84b97eb2ab3be729d8ac81168f7
Oct 30 18:58:35.826985 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:58:35.826952    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-83d5bd-64886885df-zj2nz" event={"ID":"998d1b1d-e656-49b0-8763-540ef647899d","Type":"ContainerStarted","Data":"da0ef95b80f29ba342578b597185f930ef1df84b97eb2ab3be729d8ac81168f7"}
Oct 30 18:58:36.835696 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:58:36.835658    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-83d5bd-64886885df-zj2nz" event={"ID":"998d1b1d-e656-49b0-8763-540ef647899d","Type":"ContainerStarted","Data":"0a65bab8677ec9b71030c536b117e43b5405b079b9005c88898ca5640f0a7182"}
Oct 30 18:58:36.836142 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:58:36.835742    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-83d5bd-64886885df-zj2nz"
Oct 30 18:58:36.972230 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:58:36.972178    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-83d5bd-64886885df-zj2nz" podStartSLOduration=2.160536102 podStartE2EDuration="2.972164805s" totalImagesPullingTime="811.628703ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:58:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:58:36.969596337 +0000 UTC m=+1481.217002374" watchObservedRunningTime="2026-10-30 18:58:36.972164805 +0000 UTC m=+1481.219570831"
Oct 30 18:58:42.850128 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:58:42.850099    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-83d5bd-64886885df-zj2nz"
Oct 30 18:58:56.430242 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:58:56.430211    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-7x4rb_382e8b31-a7fa-43f3-9f15-4c61627b0c04/console-operator/2.log"
Oct 30 18:58:56.432559 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:58:56.432532    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-7x4rb_382e8b31-a7fa-43f3-9f15-4c61627b0c04/console-operator/2.log"
Oct 30 18:58:56.441150 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:58:56.441125    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xfcdl_3f3fe764-2652-46c9-8e54-7882c0e6bcf1/ovn-acl-logging/0.log"
Oct 30 18:58:56.442590 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:58:56.442571    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xfcdl_3f3fe764-2652-46c9-8e54-7882c0e6bcf1/ovn-acl-logging/0.log"
Oct 30 18:59:05.620251 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:59:05.620215    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-jbsw6" containerName="extractor"
Oct 30 18:59:21.490734 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:59:21.490703    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-rjmdz"]
Oct 30 18:59:35.619836 ip-10-0-134-46 kubenswrapper[4888]: E1030 18:59:35.619802    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-jbsw6" containerName="extractor"
Oct 30 18:59:54.945718 ip-10-0-134-46 kubenswrapper[4888]: I1030 18:59:54.945686    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-rjmdz"]
Oct 30 19:00:00.211070 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:00.211034    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-e7929d-29889780-cw5hd"]
Oct 30 19:00:00.218459 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:00.218430    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-10236a-29889780-8bvj5"]
Oct 30 19:00:00.218614 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:00.218555    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-e7929d-29889780-cw5hd"
Oct 30 19:00:00.222471 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:00.222446    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-10236a-29889780-8bvj5"
Oct 30 19:00:00.264733 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:00.264692    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-85ab6a-29889780-xg49w"]
Oct 30 19:00:00.270536 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:00.270479    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fea8eecd-4a56-48e1-b68f-a3a1418dc395-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w4-e7929d-29889780-cw5hd\" (UID: \"fea8eecd-4a56-48e1-b68f-a3a1418dc395\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-e7929d-29889780-cw5hd"
Oct 30 19:00:00.270729 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:00.270708    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sgdbp\" (UniqueName: \"kubernetes.io/projected/fea8eecd-4a56-48e1-b68f-a3a1418dc395-kube-api-access-sgdbp\") pod \"maas-api-key-cleanup-e2e-worker-w4-e7929d-29889780-cw5hd\" (UID: \"fea8eecd-4a56-48e1-b68f-a3a1418dc395\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-e7929d-29889780-cw5hd"
Oct 30 19:00:00.270794 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:00.270770    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tp829\" (UniqueName: \"kubernetes.io/projected/b40b1755-0472-48e2-be01-417743104be6-kube-api-access-tp829\") pod \"maas-api-key-cleanup-e2e-api-a-10236a-29889780-8bvj5\" (UID: \"b40b1755-0472-48e2-be01-417743104be6\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-10236a-29889780-8bvj5"
Oct 30 19:00:00.270932 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:00.270896    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b40b1755-0472-48e2-be01-417743104be6-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-api-a-10236a-29889780-8bvj5\" (UID: \"b40b1755-0472-48e2-be01-417743104be6\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-10236a-29889780-8bvj5"
Oct 30 19:00:00.272203 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:00.272180    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-85ab6a-29889780-xg49w"
Oct 30 19:00:00.336372 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:00.336343    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-2qg7n\""
Oct 30 19:00:00.338151 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:00.338126    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-e7929d-29889780-cw5hd"]
Oct 30 19:00:00.344878 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:00.344844    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-85ab6a-29889780-xg49w"]
Oct 30 19:00:00.373103 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:00.373069    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sgdbp\" (UniqueName: \"kubernetes.io/projected/fea8eecd-4a56-48e1-b68f-a3a1418dc395-kube-api-access-sgdbp\") pod \"maas-api-key-cleanup-e2e-worker-w4-e7929d-29889780-cw5hd\" (UID: \"fea8eecd-4a56-48e1-b68f-a3a1418dc395\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-e7929d-29889780-cw5hd"
Oct 30 19:00:00.373239 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:00.373119    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tp829\" (UniqueName: \"kubernetes.io/projected/b40b1755-0472-48e2-be01-417743104be6-kube-api-access-tp829\") pod \"maas-api-key-cleanup-e2e-api-a-10236a-29889780-8bvj5\" (UID: \"b40b1755-0472-48e2-be01-417743104be6\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-10236a-29889780-8bvj5"
Oct 30 19:00:00.373239 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:00.373178    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fr4kr\" (UniqueName: \"kubernetes.io/projected/3dc2dd63-b055-4a45-a14e-8854ab1243d7-kube-api-access-fr4kr\") pod \"maas-api-key-cleanup-e2e-api-a-85ab6a-29889780-xg49w\" (UID: \"3dc2dd63-b055-4a45-a14e-8854ab1243d7\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-85ab6a-29889780-xg49w"
Oct 30 19:00:00.373401 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:00.373253    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b40b1755-0472-48e2-be01-417743104be6-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-api-a-10236a-29889780-8bvj5\" (UID: \"b40b1755-0472-48e2-be01-417743104be6\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-10236a-29889780-8bvj5"
Oct 30 19:00:00.373507 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:00.373467    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3dc2dd63-b055-4a45-a14e-8854ab1243d7-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-api-a-85ab6a-29889780-xg49w\" (UID: \"3dc2dd63-b055-4a45-a14e-8854ab1243d7\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-85ab6a-29889780-xg49w"
Oct 30 19:00:00.373630 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:00.373575    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fea8eecd-4a56-48e1-b68f-a3a1418dc395-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w4-e7929d-29889780-cw5hd\" (UID: \"fea8eecd-4a56-48e1-b68f-a3a1418dc395\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-e7929d-29889780-cw5hd"
Oct 30 19:00:00.374121 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:00.374103    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b40b1755-0472-48e2-be01-417743104be6-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-api-a-10236a-29889780-8bvj5\" (UID: \"b40b1755-0472-48e2-be01-417743104be6\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-10236a-29889780-8bvj5"
Oct 30 19:00:00.374452 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:00.374431    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fea8eecd-4a56-48e1-b68f-a3a1418dc395-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w4-e7929d-29889780-cw5hd\" (UID: \"fea8eecd-4a56-48e1-b68f-a3a1418dc395\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-e7929d-29889780-cw5hd"
Oct 30 19:00:00.409374 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:00.409330    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-10236a-29889780-8bvj5"]
Oct 30 19:00:00.436583 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:00.436556    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tp829\" (UniqueName: \"kubernetes.io/projected/b40b1755-0472-48e2-be01-417743104be6-kube-api-access-tp829\") pod \"maas-api-key-cleanup-e2e-api-a-10236a-29889780-8bvj5\" (UID: \"b40b1755-0472-48e2-be01-417743104be6\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-10236a-29889780-8bvj5"
Oct 30 19:00:00.437033 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:00.437007    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sgdbp\" (UniqueName: \"kubernetes.io/projected/fea8eecd-4a56-48e1-b68f-a3a1418dc395-kube-api-access-sgdbp\") pod \"maas-api-key-cleanup-e2e-worker-w4-e7929d-29889780-cw5hd\" (UID: \"fea8eecd-4a56-48e1-b68f-a3a1418dc395\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-e7929d-29889780-cw5hd"
Oct 30 19:00:00.475601 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:00.475522    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3dc2dd63-b055-4a45-a14e-8854ab1243d7-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-api-a-85ab6a-29889780-xg49w\" (UID: \"3dc2dd63-b055-4a45-a14e-8854ab1243d7\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-85ab6a-29889780-xg49w"
Oct 30 19:00:00.475753 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:00.475662    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fr4kr\" (UniqueName: \"kubernetes.io/projected/3dc2dd63-b055-4a45-a14e-8854ab1243d7-kube-api-access-fr4kr\") pod \"maas-api-key-cleanup-e2e-api-a-85ab6a-29889780-xg49w\" (UID: \"3dc2dd63-b055-4a45-a14e-8854ab1243d7\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-85ab6a-29889780-xg49w"
Oct 30 19:00:00.476151 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:00.476126    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3dc2dd63-b055-4a45-a14e-8854ab1243d7-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-api-a-85ab6a-29889780-xg49w\" (UID: \"3dc2dd63-b055-4a45-a14e-8854ab1243d7\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-85ab6a-29889780-xg49w"
Oct 30 19:00:00.536055 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:00.536019    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-e7929d-29889780-cw5hd"
Oct 30 19:00:00.538627 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:00.538605    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fr4kr\" (UniqueName: \"kubernetes.io/projected/3dc2dd63-b055-4a45-a14e-8854ab1243d7-kube-api-access-fr4kr\") pod \"maas-api-key-cleanup-e2e-api-a-85ab6a-29889780-xg49w\" (UID: \"3dc2dd63-b055-4a45-a14e-8854ab1243d7\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-85ab6a-29889780-xg49w"
Oct 30 19:00:00.543176 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:00.543153    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-10236a-29889780-8bvj5"
Oct 30 19:00:00.592567 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:00.592338    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-85ab6a-29889780-xg49w"
Oct 30 19:00:00.951372 ip-10-0-134-46 kubenswrapper[4888]: W1030 19:00:00.951346    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfea8eecd_4a56_48e1_b68f_a3a1418dc395.slice/crio-7656bad1461c166e4d85ac64461731ee792d2d232d44f65f998b808c2eef90c8 WatchSource:0}: Error finding container 7656bad1461c166e4d85ac64461731ee792d2d232d44f65f998b808c2eef90c8: Status 404 returned error can't find the container with id 7656bad1461c166e4d85ac64461731ee792d2d232d44f65f998b808c2eef90c8
Oct 30 19:00:00.951475 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:00.951450    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-e7929d-29889780-cw5hd"]
Oct 30 19:00:01.001123 ip-10-0-134-46 kubenswrapper[4888]: W1030 19:00:01.001092    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb40b1755_0472_48e2_be01_417743104be6.slice/crio-e62e6a14d32dbc315d33e5c69e72b36d08c704322ec1204d4746abe99cb4d6e4 WatchSource:0}: Error finding container e62e6a14d32dbc315d33e5c69e72b36d08c704322ec1204d4746abe99cb4d6e4: Status 404 returned error can't find the container with id e62e6a14d32dbc315d33e5c69e72b36d08c704322ec1204d4746abe99cb4d6e4
Oct 30 19:00:01.002643 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:01.002618    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-10236a-29889780-8bvj5"]
Oct 30 19:00:01.075738 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:01.075714    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-85ab6a-29889780-xg49w"]
Oct 30 19:00:01.171275 ip-10-0-134-46 kubenswrapper[4888]: W1030 19:00:01.171246    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3dc2dd63_b055_4a45_a14e_8854ab1243d7.slice/crio-1fce0f4b3782356b41c027318385f7c409fcae9e524c7db5f363f09f51ce4bf6 WatchSource:0}: Error finding container 1fce0f4b3782356b41c027318385f7c409fcae9e524c7db5f363f09f51ce4bf6: Status 404 returned error can't find the container with id 1fce0f4b3782356b41c027318385f7c409fcae9e524c7db5f363f09f51ce4bf6
Oct 30 19:00:01.602350 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:01.602319    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-85ab6a-29889780-xg49w" event={"ID":"3dc2dd63-b055-4a45-a14e-8854ab1243d7","Type":"ContainerStarted","Data":"e4dc29e1fb31fa7b17dcaa88fd69ca89f4af9f683f8b49d05ba002803fef5daf"}
Oct 30 19:00:01.602792 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:01.602384    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-85ab6a-29889780-xg49w" event={"ID":"3dc2dd63-b055-4a45-a14e-8854ab1243d7","Type":"ContainerStarted","Data":"1fce0f4b3782356b41c027318385f7c409fcae9e524c7db5f363f09f51ce4bf6"}
Oct 30 19:00:01.603995 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:01.603971    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-e7929d-29889780-cw5hd" event={"ID":"fea8eecd-4a56-48e1-b68f-a3a1418dc395","Type":"ContainerStarted","Data":"2ead590f2c6d970873de58f5287a52657a3c96fb035f7d7fa23b930f00632eae"}
Oct 30 19:00:01.604100 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:01.604028    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-e7929d-29889780-cw5hd" event={"ID":"fea8eecd-4a56-48e1-b68f-a3a1418dc395","Type":"ContainerStarted","Data":"7656bad1461c166e4d85ac64461731ee792d2d232d44f65f998b808c2eef90c8"}
Oct 30 19:00:01.605537 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:01.605516    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-10236a-29889780-8bvj5" event={"ID":"b40b1755-0472-48e2-be01-417743104be6","Type":"ContainerStarted","Data":"8f671c1db1e733fe4b0da31f62247effb9087d499dda88c84818524282773718"}
Oct 30 19:00:01.605640 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:01.605557    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-10236a-29889780-8bvj5" event={"ID":"b40b1755-0472-48e2-be01-417743104be6","Type":"ContainerStarted","Data":"e62e6a14d32dbc315d33e5c69e72b36d08c704322ec1204d4746abe99cb4d6e4"}
Oct 30 19:00:01.723492 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:01.723444    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-85ab6a-29889780-xg49w" podStartSLOduration=1.723428003 podStartE2EDuration="1.723428003s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:00:01.722017265 +0000 UTC m=+1565.969423291" watchObservedRunningTime="2026-10-30 19:00:01.723428003 +0000 UTC m=+1565.970834028"
Oct 30 19:00:01.841858 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:01.841766    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-10236a-29889780-8bvj5" podStartSLOduration=1.841752273 podStartE2EDuration="1.841752273s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:00:01.839876253 +0000 UTC m=+1566.087282277" watchObservedRunningTime="2026-10-30 19:00:01.841752273 +0000 UTC m=+1566.089158318"
Oct 30 19:00:01.971555 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:01.971503    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-e7929d-29889780-cw5hd" podStartSLOduration=1.971490426 podStartE2EDuration="1.971490426s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:00:01.968874379 +0000 UTC m=+1566.216280406" watchObservedRunningTime="2026-10-30 19:00:01.971490426 +0000 UTC m=+1566.218896448"
Oct 30 19:00:02.613592 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:02.613561    4888 generic.go:356] "Generic (PLEG): container finished" podID="fea8eecd-4a56-48e1-b68f-a3a1418dc395" containerID="2ead590f2c6d970873de58f5287a52657a3c96fb035f7d7fa23b930f00632eae" exitCode=60
Oct 30 19:00:02.614042 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:02.613604    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-e7929d-29889780-cw5hd" event={"ID":"fea8eecd-4a56-48e1-b68f-a3a1418dc395","Type":"ContainerDied","Data":"2ead590f2c6d970873de58f5287a52657a3c96fb035f7d7fa23b930f00632eae"}
Oct 30 19:00:02.614042 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:02.613941    4888 scope.go:122] "RemoveContainer" containerID="2ead590f2c6d970873de58f5287a52657a3c96fb035f7d7fa23b930f00632eae"
Oct 30 19:00:02.615572 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:02.615544    4888 generic.go:356] "Generic (PLEG): container finished" podID="b40b1755-0472-48e2-be01-417743104be6" containerID="8f671c1db1e733fe4b0da31f62247effb9087d499dda88c84818524282773718" exitCode=60
Oct 30 19:00:02.615707 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:02.615657    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-10236a-29889780-8bvj5" event={"ID":"b40b1755-0472-48e2-be01-417743104be6","Type":"ContainerDied","Data":"8f671c1db1e733fe4b0da31f62247effb9087d499dda88c84818524282773718"}
Oct 30 19:00:02.615896 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:02.615880    4888 scope.go:122] "RemoveContainer" containerID="8f671c1db1e733fe4b0da31f62247effb9087d499dda88c84818524282773718"
Oct 30 19:00:02.617603 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:02.617584    4888 generic.go:356] "Generic (PLEG): container finished" podID="3dc2dd63-b055-4a45-a14e-8854ab1243d7" containerID="e4dc29e1fb31fa7b17dcaa88fd69ca89f4af9f683f8b49d05ba002803fef5daf" exitCode=60
Oct 30 19:00:02.617673 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:02.617648    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-85ab6a-29889780-xg49w" event={"ID":"3dc2dd63-b055-4a45-a14e-8854ab1243d7","Type":"ContainerDied","Data":"e4dc29e1fb31fa7b17dcaa88fd69ca89f4af9f683f8b49d05ba002803fef5daf"}
Oct 30 19:00:02.617916 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:02.617897    4888 scope.go:122] "RemoveContainer" containerID="e4dc29e1fb31fa7b17dcaa88fd69ca89f4af9f683f8b49d05ba002803fef5daf"
Oct 30 19:00:03.636833 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:03.635926    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-e7929d-29889780-cw5hd" event={"ID":"fea8eecd-4a56-48e1-b68f-a3a1418dc395","Type":"ContainerStarted","Data":"8711025529e956e2b89c226d0e8a3dff24562e9e002400abd01edb20364dc713"}
Oct 30 19:00:03.638191 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:03.638165    4888 generic.go:356] "Generic (PLEG): container finished" podID="b40b1755-0472-48e2-be01-417743104be6" containerID="ebe639bd7837376159aa49699bd503f91c0e082df6b3f7c80c236cad1362a382" exitCode=60
Oct 30 19:00:03.638355 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:03.638240    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-10236a-29889780-8bvj5" event={"ID":"b40b1755-0472-48e2-be01-417743104be6","Type":"ContainerDied","Data":"ebe639bd7837376159aa49699bd503f91c0e082df6b3f7c80c236cad1362a382"}
Oct 30 19:00:03.638355 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:03.638343    4888 scope.go:122] "RemoveContainer" containerID="8f671c1db1e733fe4b0da31f62247effb9087d499dda88c84818524282773718"
Oct 30 19:00:03.638530 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:03.638503    4888 scope.go:122] "RemoveContainer" containerID="ebe639bd7837376159aa49699bd503f91c0e082df6b3f7c80c236cad1362a382"
Oct 30 19:00:03.638867 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:00:03.638835    4888 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-api-a-10236a-29889780-8bvj5_odh-ai-gateway-infra(b40b1755-0472-48e2-be01-417743104be6)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-10236a-29889780-8bvj5" podUID="b40b1755-0472-48e2-be01-417743104be6"
Oct 30 19:00:03.640611 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:03.640585    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-85ab6a-29889780-xg49w" event={"ID":"3dc2dd63-b055-4a45-a14e-8854ab1243d7","Type":"ContainerStarted","Data":"573b852cc3639b551d137215b94041de84f1a5c98a82a3149e12ab4197fce27a"}
Oct 30 19:00:04.650188 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:04.650158    4888 generic.go:356] "Generic (PLEG): container finished" podID="fea8eecd-4a56-48e1-b68f-a3a1418dc395" containerID="8711025529e956e2b89c226d0e8a3dff24562e9e002400abd01edb20364dc713" exitCode=60
Oct 30 19:00:04.650657 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:04.650245    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-e7929d-29889780-cw5hd" event={"ID":"fea8eecd-4a56-48e1-b68f-a3a1418dc395","Type":"ContainerDied","Data":"8711025529e956e2b89c226d0e8a3dff24562e9e002400abd01edb20364dc713"}
Oct 30 19:00:04.650657 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:04.650338    4888 scope.go:122] "RemoveContainer" containerID="2ead590f2c6d970873de58f5287a52657a3c96fb035f7d7fa23b930f00632eae"
Oct 30 19:00:04.650657 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:04.650497    4888 scope.go:122] "RemoveContainer" containerID="8711025529e956e2b89c226d0e8a3dff24562e9e002400abd01edb20364dc713"
Oct 30 19:00:04.650846 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:00:04.650798    4888 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w4-e7929d-29889780-cw5hd_odh-ai-gateway-infra(fea8eecd-4a56-48e1-b68f-a3a1418dc395)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-e7929d-29889780-cw5hd" podUID="fea8eecd-4a56-48e1-b68f-a3a1418dc395"
Oct 30 19:00:04.654321 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:04.653388    4888 scope.go:122] "RemoveContainer" containerID="ebe639bd7837376159aa49699bd503f91c0e082df6b3f7c80c236cad1362a382"
Oct 30 19:00:04.654321 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:00:04.653648    4888 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-api-a-10236a-29889780-8bvj5_odh-ai-gateway-infra(b40b1755-0472-48e2-be01-417743104be6)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-10236a-29889780-8bvj5" podUID="b40b1755-0472-48e2-be01-417743104be6"
Oct 30 19:00:04.658725 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:04.658703    4888 generic.go:356] "Generic (PLEG): container finished" podID="3dc2dd63-b055-4a45-a14e-8854ab1243d7" containerID="573b852cc3639b551d137215b94041de84f1a5c98a82a3149e12ab4197fce27a" exitCode=60
Oct 30 19:00:04.658833 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:04.658728    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-85ab6a-29889780-xg49w" event={"ID":"3dc2dd63-b055-4a45-a14e-8854ab1243d7","Type":"ContainerDied","Data":"573b852cc3639b551d137215b94041de84f1a5c98a82a3149e12ab4197fce27a"}
Oct 30 19:00:04.659022 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:04.659007    4888 scope.go:122] "RemoveContainer" containerID="573b852cc3639b551d137215b94041de84f1a5c98a82a3149e12ab4197fce27a"
Oct 30 19:00:04.659328 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:00:04.659305    4888 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-api-a-85ab6a-29889780-xg49w_odh-ai-gateway-infra(3dc2dd63-b055-4a45-a14e-8854ab1243d7)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-85ab6a-29889780-xg49w" podUID="3dc2dd63-b055-4a45-a14e-8854ab1243d7"
Oct 30 19:00:04.665386 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:04.665369    4888 scope.go:122] "RemoveContainer" containerID="e4dc29e1fb31fa7b17dcaa88fd69ca89f4af9f683f8b49d05ba002803fef5daf"
Oct 30 19:00:05.676368 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:05.672915    4888 scope.go:122] "RemoveContainer" containerID="8711025529e956e2b89c226d0e8a3dff24562e9e002400abd01edb20364dc713"
Oct 30 19:00:05.682304 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:00:05.677621    4888 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w4-e7929d-29889780-cw5hd_odh-ai-gateway-infra(fea8eecd-4a56-48e1-b68f-a3a1418dc395)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-e7929d-29889780-cw5hd" podUID="fea8eecd-4a56-48e1-b68f-a3a1418dc395"
Oct 30 19:00:05.704307 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:05.704062    4888 scope.go:122] "RemoveContainer" containerID="573b852cc3639b551d137215b94041de84f1a5c98a82a3149e12ab4197fce27a"
Oct 30 19:00:05.708305 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:00:05.705104    4888 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-api-a-85ab6a-29889780-xg49w_odh-ai-gateway-infra(3dc2dd63-b055-4a45-a14e-8854ab1243d7)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-85ab6a-29889780-xg49w" podUID="3dc2dd63-b055-4a45-a14e-8854ab1243d7"
Oct 30 19:00:05.717485 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:00:05.717452    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-jbsw6" containerName="extractor"
Oct 30 19:00:14.883513 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:14.883484    4888 scope.go:122] "RemoveContainer" containerID="573b852cc3639b551d137215b94041de84f1a5c98a82a3149e12ab4197fce27a"
Oct 30 19:00:15.789033 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:15.789007    4888 generic.go:356] "Generic (PLEG): container finished" podID="3dc2dd63-b055-4a45-a14e-8854ab1243d7" containerID="47ff5eed640e3b1247d1af2d97acb87c7c939638224d545a22cd84ccfd4c93c1" exitCode=60
Oct 30 19:00:15.789150 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:15.789051    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-85ab6a-29889780-xg49w" event={"ID":"3dc2dd63-b055-4a45-a14e-8854ab1243d7","Type":"ContainerDied","Data":"47ff5eed640e3b1247d1af2d97acb87c7c939638224d545a22cd84ccfd4c93c1"}
Oct 30 19:00:15.789150 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:15.789098    4888 scope.go:122] "RemoveContainer" containerID="573b852cc3639b551d137215b94041de84f1a5c98a82a3149e12ab4197fce27a"
Oct 30 19:00:15.789535 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:15.789510    4888 scope.go:122] "RemoveContainer" containerID="47ff5eed640e3b1247d1af2d97acb87c7c939638224d545a22cd84ccfd4c93c1"
Oct 30 19:00:15.789833 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:00:15.789811    4888 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-api-a-85ab6a-29889780-xg49w_odh-ai-gateway-infra(3dc2dd63-b055-4a45-a14e-8854ab1243d7)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-85ab6a-29889780-xg49w" podUID="3dc2dd63-b055-4a45-a14e-8854ab1243d7"
Oct 30 19:00:15.883782 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:15.883760    4888 scope.go:122] "RemoveContainer" containerID="ebe639bd7837376159aa49699bd503f91c0e082df6b3f7c80c236cad1362a382"
Oct 30 19:00:16.643878 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:16.643845    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4p2zp"]
Oct 30 19:00:16.644118 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:16.644091    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4p2zp" podUID="2ba277ac-44b2-4d12-b873-6b584085f0d1" containerName="istio-proxy" containerID="cri-o://5432460e6fed255408e2c642cd55b7ce9c4d906e7636c508abe5d5f10ea2df2b" gracePeriod=30
Oct 30 19:00:16.661486 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:16.661451    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4xpmd"]
Oct 30 19:00:16.661746 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:16.661675    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4xpmd" podUID="a6c278cc-4cc3-4584-b83e-376f5af90380" containerName="istio-proxy" containerID="cri-o://7db3612372458c749c6ad16c7a0326ecb1f9259423087b1413072520e71655c7" gracePeriod=30
Oct 30 19:00:16.666296 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:16.666250    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-ngqb5"]
Oct 30 19:00:16.666513 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:16.666484    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-ngqb5" podUID="d33b7312-dc19-402a-9c2e-12e0b7eee906" containerName="istio-proxy" containerID="cri-o://d0fa2b0cac5d3e4044a8a2d1a39fa843e6d134086a59257db9c99b6cb7c6619f" gracePeriod=30
Oct 30 19:00:16.697422 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:16.697389    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-hnndm"]
Oct 30 19:00:16.697923 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:16.697890    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-hnndm" podUID="d6294bd8-9418-4c1a-99c6-cf52ba98edaa" containerName="istio-proxy" containerID="cri-o://461c7337f2cbaa04e696b602018356667011c43f993fb2076a7082372dff2f93" gracePeriod=30
Oct 30 19:00:16.703585 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:16.703561    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9rstj"]
Oct 30 19:00:16.703787 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:16.703735    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9rstj" podUID="0d5a7945-c6aa-453a-ac8c-1b23cddc1642" containerName="istio-proxy" containerID="cri-o://f0bc676121e67f034883592e084979545dbb256182eae9e11347cd66b32d7caa" gracePeriod=30
Oct 30 19:00:16.823577 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:16.823540    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-10236a-29889780-8bvj5" event={"ID":"b40b1755-0472-48e2-be01-417743104be6","Type":"ContainerStarted","Data":"e2c0ebdde91ee6dc3d7278620f94c788292f32a4bfc9cbdfe11c64c8494ec186"}
Oct 30 19:00:16.937359 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:16.920238    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-85ab6a-29889780-xg49w"]
Oct 30 19:00:17.106562 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:17.106534    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-85ab6a-29889780-xg49w"
Oct 30 19:00:17.127824 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:17.127794    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3dc2dd63-b055-4a45-a14e-8854ab1243d7-kube-api-access-fr4kr\" (UniqueName: \"kubernetes.io/projected/3dc2dd63-b055-4a45-a14e-8854ab1243d7-kube-api-access-fr4kr\") pod \"3dc2dd63-b055-4a45-a14e-8854ab1243d7\" (UID: \"3dc2dd63-b055-4a45-a14e-8854ab1243d7\") "
Oct 30 19:00:17.127974 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:17.127954    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3dc2dd63-b055-4a45-a14e-8854ab1243d7-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3dc2dd63-b055-4a45-a14e-8854ab1243d7-maas-api-service-ca\") pod \"3dc2dd63-b055-4a45-a14e-8854ab1243d7\" (UID: \"3dc2dd63-b055-4a45-a14e-8854ab1243d7\") "
Oct 30 19:00:17.128256 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:17.128236    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3dc2dd63-b055-4a45-a14e-8854ab1243d7-maas-api-service-ca" pod "3dc2dd63-b055-4a45-a14e-8854ab1243d7" (UID: "3dc2dd63-b055-4a45-a14e-8854ab1243d7"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:00:17.128437 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:17.128422    4888 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3dc2dd63-b055-4a45-a14e-8854ab1243d7-maas-api-service-ca\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:00:17.131184 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:17.131162    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3dc2dd63-b055-4a45-a14e-8854ab1243d7-kube-api-access-fr4kr" pod "3dc2dd63-b055-4a45-a14e-8854ab1243d7" (UID: "3dc2dd63-b055-4a45-a14e-8854ab1243d7"). InnerVolumeSpecName "kube-api-access-fr4kr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:00:17.229447 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:17.229362    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fr4kr\" (UniqueName: \"kubernetes.io/projected/3dc2dd63-b055-4a45-a14e-8854ab1243d7-kube-api-access-fr4kr\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:00:17.836207 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:17.836175    4888 generic.go:356] "Generic (PLEG): container finished" podID="b40b1755-0472-48e2-be01-417743104be6" containerID="e2c0ebdde91ee6dc3d7278620f94c788292f32a4bfc9cbdfe11c64c8494ec186" exitCode=60
Oct 30 19:00:17.836402 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:17.836244    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-10236a-29889780-8bvj5" event={"ID":"b40b1755-0472-48e2-be01-417743104be6","Type":"ContainerDied","Data":"e2c0ebdde91ee6dc3d7278620f94c788292f32a4bfc9cbdfe11c64c8494ec186"}
Oct 30 19:00:17.836402 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:17.836364    4888 scope.go:122] "RemoveContainer" containerID="ebe639bd7837376159aa49699bd503f91c0e082df6b3f7c80c236cad1362a382"
Oct 30 19:00:17.836722 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:17.836701    4888 scope.go:122] "RemoveContainer" containerID="e2c0ebdde91ee6dc3d7278620f94c788292f32a4bfc9cbdfe11c64c8494ec186"
Oct 30 19:00:17.837013 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:00:17.836972    4888 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-api-a-10236a-29889780-8bvj5_odh-ai-gateway-infra(b40b1755-0472-48e2-be01-417743104be6)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-10236a-29889780-8bvj5" podUID="b40b1755-0472-48e2-be01-417743104be6"
Oct 30 19:00:17.837945 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:17.837930    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-85ab6a-29889780-xg49w"
Oct 30 19:00:17.838017 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:17.837942    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-85ab6a-29889780-xg49w" event={"ID":"3dc2dd63-b055-4a45-a14e-8854ab1243d7","Type":"ContainerDied","Data":"1fce0f4b3782356b41c027318385f7c409fcae9e524c7db5f363f09f51ce4bf6"}
Oct 30 19:00:17.851651 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:17.851631    4888 scope.go:122] "RemoveContainer" containerID="47ff5eed640e3b1247d1af2d97acb87c7c939638224d545a22cd84ccfd4c93c1"
Oct 30 19:00:17.892632 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:17.892608    4888 scope.go:122] "RemoveContainer" containerID="8711025529e956e2b89c226d0e8a3dff24562e9e002400abd01edb20364dc713"
Oct 30 19:00:18.075646 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:18.075611    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-85ab6a-29889780-xg49w"]
Oct 30 19:00:18.080319 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:18.080294    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-85ab6a-29889780-xg49w"]
Oct 30 19:00:18.585270 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:18.585242    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-10236a-29889780-8bvj5"]
Oct 30 19:00:18.848232 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:18.848159    4888 generic.go:356] "Generic (PLEG): container finished" podID="fea8eecd-4a56-48e1-b68f-a3a1418dc395" containerID="17895e02e754ae88a8a2b73ad570dc4dee5650f8a034f28c3912b4e7c53b0d25" exitCode=60
Oct 30 19:00:18.848430 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:18.848227    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-e7929d-29889780-cw5hd" event={"ID":"fea8eecd-4a56-48e1-b68f-a3a1418dc395","Type":"ContainerDied","Data":"17895e02e754ae88a8a2b73ad570dc4dee5650f8a034f28c3912b4e7c53b0d25"}
Oct 30 19:00:18.848430 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:18.848299    4888 scope.go:122] "RemoveContainer" containerID="8711025529e956e2b89c226d0e8a3dff24562e9e002400abd01edb20364dc713"
Oct 30 19:00:18.848674 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:18.848653    4888 scope.go:122] "RemoveContainer" containerID="17895e02e754ae88a8a2b73ad570dc4dee5650f8a034f28c3912b4e7c53b0d25"
Oct 30 19:00:18.848977 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:00:18.848957    4888 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w4-e7929d-29889780-cw5hd_odh-ai-gateway-infra(fea8eecd-4a56-48e1-b68f-a3a1418dc395)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-e7929d-29889780-cw5hd" podUID="fea8eecd-4a56-48e1-b68f-a3a1418dc395"
Oct 30 19:00:19.046836 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:19.046813    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-10236a-29889780-8bvj5"
Oct 30 19:00:19.150391 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:19.150268    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b40b1755-0472-48e2-be01-417743104be6-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b40b1755-0472-48e2-be01-417743104be6-maas-api-service-ca\") pod \"b40b1755-0472-48e2-be01-417743104be6\" (UID: \"b40b1755-0472-48e2-be01-417743104be6\") "
Oct 30 19:00:19.150391 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:19.150363    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b40b1755-0472-48e2-be01-417743104be6-kube-api-access-tp829\" (UniqueName: \"kubernetes.io/projected/b40b1755-0472-48e2-be01-417743104be6-kube-api-access-tp829\") pod \"b40b1755-0472-48e2-be01-417743104be6\" (UID: \"b40b1755-0472-48e2-be01-417743104be6\") "
Oct 30 19:00:19.150904 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:19.150694    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b40b1755-0472-48e2-be01-417743104be6-maas-api-service-ca" pod "b40b1755-0472-48e2-be01-417743104be6" (UID: "b40b1755-0472-48e2-be01-417743104be6"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:00:19.150942 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:19.150928    4888 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b40b1755-0472-48e2-be01-417743104be6-maas-api-service-ca\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:00:19.153195 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:19.153176    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b40b1755-0472-48e2-be01-417743104be6-kube-api-access-tp829" pod "b40b1755-0472-48e2-be01-417743104be6" (UID: "b40b1755-0472-48e2-be01-417743104be6"). InnerVolumeSpecName "kube-api-access-tp829". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:00:19.252573 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:19.252535    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tp829\" (UniqueName: \"kubernetes.io/projected/b40b1755-0472-48e2-be01-417743104be6-kube-api-access-tp829\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:00:19.862703 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:19.862678    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-10236a-29889780-8bvj5"
Oct 30 19:00:19.862880 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:19.862675    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-10236a-29889780-8bvj5" event={"ID":"b40b1755-0472-48e2-be01-417743104be6","Type":"ContainerDied","Data":"e62e6a14d32dbc315d33e5c69e72b36d08c704322ec1204d4746abe99cb4d6e4"}
Oct 30 19:00:19.862880 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:19.862814    4888 scope.go:122] "RemoveContainer" containerID="e2c0ebdde91ee6dc3d7278620f94c788292f32a4bfc9cbdfe11c64c8494ec186"
Oct 30 19:00:19.891763 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:19.891737    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3dc2dd63-b055-4a45-a14e-8854ab1243d7" path="/var/lib/kubelet/pods/3dc2dd63-b055-4a45-a14e-8854ab1243d7/volumes"
Oct 30 19:00:19.988387 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:19.988360    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-e7929d-29889780-cw5hd"]
Oct 30 19:00:20.041868 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:20.041834    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-10236a-29889780-8bvj5"]
Oct 30 19:00:20.048732 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:20.048704    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-10236a-29889780-8bvj5"]
Oct 30 19:00:20.178497 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:20.178474    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-e7929d-29889780-cw5hd"
Oct 30 19:00:20.288396 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:20.288369    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fea8eecd-4a56-48e1-b68f-a3a1418dc395-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fea8eecd-4a56-48e1-b68f-a3a1418dc395-maas-api-service-ca\") pod \"fea8eecd-4a56-48e1-b68f-a3a1418dc395\" (UID: \"fea8eecd-4a56-48e1-b68f-a3a1418dc395\") "
Oct 30 19:00:20.288550 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:20.288532    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fea8eecd-4a56-48e1-b68f-a3a1418dc395-kube-api-access-sgdbp\" (UniqueName: \"kubernetes.io/projected/fea8eecd-4a56-48e1-b68f-a3a1418dc395-kube-api-access-sgdbp\") pod \"fea8eecd-4a56-48e1-b68f-a3a1418dc395\" (UID: \"fea8eecd-4a56-48e1-b68f-a3a1418dc395\") "
Oct 30 19:00:20.288788 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:20.288767    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fea8eecd-4a56-48e1-b68f-a3a1418dc395-maas-api-service-ca" pod "fea8eecd-4a56-48e1-b68f-a3a1418dc395" (UID: "fea8eecd-4a56-48e1-b68f-a3a1418dc395"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:00:20.289134 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:20.289116    4888 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fea8eecd-4a56-48e1-b68f-a3a1418dc395-maas-api-service-ca\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:00:20.290712 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:20.290682    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fea8eecd-4a56-48e1-b68f-a3a1418dc395-kube-api-access-sgdbp" pod "fea8eecd-4a56-48e1-b68f-a3a1418dc395" (UID: "fea8eecd-4a56-48e1-b68f-a3a1418dc395"). InnerVolumeSpecName "kube-api-access-sgdbp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:00:20.391374 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:20.391333    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sgdbp\" (UniqueName: \"kubernetes.io/projected/fea8eecd-4a56-48e1-b68f-a3a1418dc395-kube-api-access-sgdbp\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:00:20.874884 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:20.874850    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-e7929d-29889780-cw5hd"
Oct 30 19:00:20.875799 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:20.875767    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-e7929d-29889780-cw5hd" event={"ID":"fea8eecd-4a56-48e1-b68f-a3a1418dc395","Type":"ContainerDied","Data":"7656bad1461c166e4d85ac64461731ee792d2d232d44f65f998b808c2eef90c8"}
Oct 30 19:00:20.875969 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:20.875868    4888 scope.go:122] "RemoveContainer" containerID="17895e02e754ae88a8a2b73ad570dc4dee5650f8a034f28c3912b4e7c53b0d25"
Oct 30 19:00:21.050752 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:21.050646    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-e7929d-29889780-cw5hd"]
Oct 30 19:00:21.101757 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:21.101725    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-e7929d-29889780-cw5hd"]
Oct 30 19:00:21.906319 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:21.906252    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b40b1755-0472-48e2-be01-417743104be6" path="/var/lib/kubelet/pods/b40b1755-0472-48e2-be01-417743104be6/volumes"
Oct 30 19:00:21.908457 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:21.907850    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fea8eecd-4a56-48e1-b68f-a3a1418dc395" path="/var/lib/kubelet/pods/fea8eecd-4a56-48e1-b68f-a3a1418dc395/volumes"
Oct 30 19:00:21.909477 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:21.909454    4888 generic.go:356] "Generic (PLEG): container finished" podID="d33b7312-dc19-402a-9c2e-12e0b7eee906" containerID="d0fa2b0cac5d3e4044a8a2d1a39fa843e6d134086a59257db9c99b6cb7c6619f" exitCode=0
Oct 30 19:00:21.909621 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:21.909559    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-ngqb5" event={"ID":"d33b7312-dc19-402a-9c2e-12e0b7eee906","Type":"ContainerDied","Data":"d0fa2b0cac5d3e4044a8a2d1a39fa843e6d134086a59257db9c99b6cb7c6619f"}
Oct 30 19:00:21.913506 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:21.913483    4888 generic.go:356] "Generic (PLEG): container finished" podID="d6294bd8-9418-4c1a-99c6-cf52ba98edaa" containerID="461c7337f2cbaa04e696b602018356667011c43f993fb2076a7082372dff2f93" exitCode=0
Oct 30 19:00:21.913755 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:21.913734    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-hnndm" event={"ID":"d6294bd8-9418-4c1a-99c6-cf52ba98edaa","Type":"ContainerDied","Data":"461c7337f2cbaa04e696b602018356667011c43f993fb2076a7082372dff2f93"}
Oct 30 19:00:21.916678 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:21.916520    4888 generic.go:356] "Generic (PLEG): container finished" podID="0d5a7945-c6aa-453a-ac8c-1b23cddc1642" containerID="f0bc676121e67f034883592e084979545dbb256182eae9e11347cd66b32d7caa" exitCode=0
Oct 30 19:00:21.916678 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:21.916596    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9rstj" event={"ID":"0d5a7945-c6aa-453a-ac8c-1b23cddc1642","Type":"ContainerDied","Data":"f0bc676121e67f034883592e084979545dbb256182eae9e11347cd66b32d7caa"}
Oct 30 19:00:21.919571 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:21.919351    4888 generic.go:356] "Generic (PLEG): container finished" podID="a6c278cc-4cc3-4584-b83e-376f5af90380" containerID="7db3612372458c749c6ad16c7a0326ecb1f9259423087b1413072520e71655c7" exitCode=0
Oct 30 19:00:21.919571 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:21.919508    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4xpmd" event={"ID":"a6c278cc-4cc3-4584-b83e-376f5af90380","Type":"ContainerDied","Data":"7db3612372458c749c6ad16c7a0326ecb1f9259423087b1413072520e71655c7"}
Oct 30 19:00:21.921509 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:21.921437    4888 generic.go:356] "Generic (PLEG): container finished" podID="2ba277ac-44b2-4d12-b873-6b584085f0d1" containerID="5432460e6fed255408e2c642cd55b7ce9c4d906e7636c508abe5d5f10ea2df2b" exitCode=0
Oct 30 19:00:21.921509 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:21.921468    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4p2zp" event={"ID":"2ba277ac-44b2-4d12-b873-6b584085f0d1","Type":"ContainerDied","Data":"5432460e6fed255408e2c642cd55b7ce9c4d906e7636c508abe5d5f10ea2df2b"}
Oct 30 19:00:22.085886 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:22.085856    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4p2zp"
Oct 30 19:00:22.131835 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:22.131809    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-ngqb5"
Oct 30 19:00:22.136699 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:22.136679    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9rstj"
Oct 30 19:00:22.140474 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:22.140452    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-hnndm"
Oct 30 19:00:22.146951 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:22.146928    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4xpmd"
Oct 30 19:00:22.212759 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:22.212728    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/0d5a7945-c6aa-453a-ac8c-1b23cddc1642-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0d5a7945-c6aa-453a-ac8c-1b23cddc1642-istio-podinfo\") pod \"0d5a7945-c6aa-453a-ac8c-1b23cddc1642\" (UID: \"0d5a7945-c6aa-453a-ac8c-1b23cddc1642\") "
Oct 30 19:00:22.212759 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:22.212764    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d33b7312-dc19-402a-9c2e-12e0b7eee906-istio-token\" (UniqueName: \"kubernetes.io/projected/d33b7312-dc19-402a-9c2e-12e0b7eee906-istio-token\") pod \"d33b7312-dc19-402a-9c2e-12e0b7eee906\" (UID: \"d33b7312-dc19-402a-9c2e-12e0b7eee906\") "
Oct 30 19:00:22.213016 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:22.212804    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d6294bd8-9418-4c1a-99c6-cf52ba98edaa-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d6294bd8-9418-4c1a-99c6-cf52ba98edaa-workload-certs\") pod \"d6294bd8-9418-4c1a-99c6-cf52ba98edaa\" (UID: \"d6294bd8-9418-4c1a-99c6-cf52ba98edaa\") "
Oct 30 19:00:22.213016 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:22.212825    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a6c278cc-4cc3-4584-b83e-376f5af90380-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a6c278cc-4cc3-4584-b83e-376f5af90380-istio-envoy\") pod \"a6c278cc-4cc3-4584-b83e-376f5af90380\" (UID: \"a6c278cc-4cc3-4584-b83e-376f5af90380\") "
Oct 30 19:00:22.213016 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:22.212860    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d6294bd8-9418-4c1a-99c6-cf52ba98edaa-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6294bd8-9418-4c1a-99c6-cf52ba98edaa-workload-socket\") pod \"d6294bd8-9418-4c1a-99c6-cf52ba98edaa\" (UID: \"d6294bd8-9418-4c1a-99c6-cf52ba98edaa\") "
Oct 30 19:00:22.213016 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:22.212903    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2ba277ac-44b2-4d12-b873-6b584085f0d1-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ba277ac-44b2-4d12-b873-6b584085f0d1-workload-socket\") pod \"2ba277ac-44b2-4d12-b873-6b584085f0d1\" (UID: \"2ba277ac-44b2-4d12-b873-6b584085f0d1\") "
Oct 30 19:00:22.213016 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:22.212941    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d33b7312-dc19-402a-9c2e-12e0b7eee906-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d33b7312-dc19-402a-9c2e-12e0b7eee906-credential-socket\") pod \"d33b7312-dc19-402a-9c2e-12e0b7eee906\" (UID: \"d33b7312-dc19-402a-9c2e-12e0b7eee906\") "
Oct 30 19:00:22.213016 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:22.212969    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a6c278cc-4cc3-4584-b83e-376f5af90380-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a6c278cc-4cc3-4584-b83e-376f5af90380-credential-socket\") pod \"a6c278cc-4cc3-4584-b83e-376f5af90380\" (UID: \"a6c278cc-4cc3-4584-b83e-376f5af90380\") "
Oct 30 19:00:22.213016 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:22.213002    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d6294bd8-9418-4c1a-99c6-cf52ba98edaa-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d6294bd8-9418-4c1a-99c6-cf52ba98edaa-istiod-ca-cert\") pod \"d6294bd8-9418-4c1a-99c6-cf52ba98edaa\" (UID: \"d6294bd8-9418-4c1a-99c6-cf52ba98edaa\") "
Oct 30 19:00:22.213418 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:22.213027    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/2ba277ac-44b2-4d12-b873-6b584085f0d1-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2ba277ac-44b2-4d12-b873-6b584085f0d1-istio-podinfo\") pod \"2ba277ac-44b2-4d12-b873-6b584085f0d1\" (UID: \"2ba277ac-44b2-4d12-b873-6b584085f0d1\") "
Oct 30 19:00:22.213418 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:22.213087    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2ba277ac-44b2-4d12-b873-6b584085f0d1-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2ba277ac-44b2-4d12-b873-6b584085f0d1-istio-envoy\") pod \"2ba277ac-44b2-4d12-b873-6b584085f0d1\" (UID: \"2ba277ac-44b2-4d12-b873-6b584085f0d1\") "
Oct 30 19:00:22.213418 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:22.213117    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d33b7312-dc19-402a-9c2e-12e0b7eee906-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d33b7312-dc19-402a-9c2e-12e0b7eee906-workload-socket\") pod \"d33b7312-dc19-402a-9c2e-12e0b7eee906\" (UID: \"d33b7312-dc19-402a-9c2e-12e0b7eee906\") "
Oct 30 19:00:22.213418 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:22.213165    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d6294bd8-9418-4c1a-99c6-cf52ba98edaa-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d6294bd8-9418-4c1a-99c6-cf52ba98edaa-istio-envoy\") pod \"d6294bd8-9418-4c1a-99c6-cf52ba98edaa\" (UID: \"d6294bd8-9418-4c1a-99c6-cf52ba98edaa\") "
Oct 30 19:00:22.213418 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:22.213183    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d6294bd8-9418-4c1a-99c6-cf52ba98edaa-workload-socket" pod "d6294bd8-9418-4c1a-99c6-cf52ba98edaa" (UID: "d6294bd8-9418-4c1a-99c6-cf52ba98edaa"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:00:22.213418 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:22.213187    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ba277ac-44b2-4d12-b873-6b584085f0d1-workload-socket" pod "2ba277ac-44b2-4d12-b873-6b584085f0d1" (UID: "2ba277ac-44b2-4d12-b873-6b584085f0d1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:00:22.213418 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:22.213198    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0d5a7945-c6aa-453a-ac8c-1b23cddc1642-kube-api-access-qfxlx\" (UniqueName: \"kubernetes.io/projected/0d5a7945-c6aa-453a-ac8c-1b23cddc1642-kube-api-access-qfxlx\") pod \"0d5a7945-c6aa-453a-ac8c-1b23cddc1642\" (UID: \"0d5a7945-c6aa-453a-ac8c-1b23cddc1642\") "
Oct 30 19:00:22.213418 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:22.213322    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a6c278cc-4cc3-4584-b83e-376f5af90380-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a6c278cc-4cc3-4584-b83e-376f5af90380-workload-socket\") pod \"a6c278cc-4cc3-4584-b83e-376f5af90380\" (UID: \"a6c278cc-4cc3-4584-b83e-376f5af90380\") "
Oct 30 19:00:22.213418 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:22.213360    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a6c278cc-4cc3-4584-b83e-376f5af90380-credential-socket" pod "a6c278cc-4cc3-4584-b83e-376f5af90380" (UID: "a6c278cc-4cc3-4584-b83e-376f5af90380"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:00:22.213418 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:22.213373    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d6294bd8-9418-4c1a-99c6-cf52ba98edaa-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6294bd8-9418-4c1a-99c6-cf52ba98edaa-credential-socket\") pod \"d6294bd8-9418-4c1a-99c6-cf52ba98edaa\" (UID: \"d6294bd8-9418-4c1a-99c6-cf52ba98edaa\") "
Oct 30 19:00:22.213418 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:22.213406    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0d5a7945-c6aa-453a-ac8c-1b23cddc1642-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0d5a7945-c6aa-453a-ac8c-1b23cddc1642-istio-data\") pod \"0d5a7945-c6aa-453a-ac8c-1b23cddc1642\" (UID: \"0d5a7945-c6aa-453a-ac8c-1b23cddc1642\") "
Oct 30 19:00:22.214102 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:22.213443    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2ba277ac-44b2-4d12-b873-6b584085f0d1-istio-token\" (UniqueName: \"kubernetes.io/projected/2ba277ac-44b2-4d12-b873-6b584085f0d1-istio-token\") pod \"2ba277ac-44b2-4d12-b873-6b584085f0d1\" (UID: \"2ba277ac-44b2-4d12-b873-6b584085f0d1\") "
Oct 30 19:00:22.214102 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:22.213485    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d6294bd8-9418-4c1a-99c6-cf52ba98edaa-istio-token\" (UniqueName: \"kubernetes.io/projected/d6294bd8-9418-4c1a-99c6-cf52ba98edaa-istio-token\") pod \"d6294bd8-9418-4c1a-99c6-cf52ba98edaa\" (UID: \"d6294bd8-9418-4c1a-99c6-cf52ba98edaa\") "
Oct 30 19:00:22.214102 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:22.213530    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d33b7312-dc19-402a-9c2e-12e0b7eee906-kube-api-access-vdt4h\" (UniqueName: \"kubernetes.io/projected/d33b7312-dc19-402a-9c2e-12e0b7eee906-kube-api-access-vdt4h\") pod \"d33b7312-dc19-402a-9c2e-12e0b7eee906\" (UID: \"d33b7312-dc19-402a-9c2e-12e0b7eee906\") "
Oct 30 19:00:22.214102 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:22.213561    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a6c278cc-4cc3-4584-b83e-376f5af90380-kube-api-access-p5wb8\" (UniqueName: \"kubernetes.io/projected/a6c278cc-4cc3-4584-b83e-376f5af90380-kube-api-access-p5wb8\") pod \"a6c278cc-4cc3-4584-b83e-376f5af90380\" (UID: \"a6c278cc-4cc3-4584-b83e-376f5af90380\") "
Oct 30 19:00:22.214102 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:22.213593    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d6294bd8-9418-4c1a-99c6-cf52ba98edaa-kube-api-access-52g5c\" (UniqueName: \"kubernetes.io/projected/d6294bd8-9418-4c1a-99c6-cf52ba98edaa-kube-api-access-52g5c\") pod \"d6294bd8-9418-4c1a-99c6-cf52ba98edaa\" (UID: \"d6294bd8-9418-4c1a-99c6-cf52ba98edaa\") "
Oct 30 19:00:22.214102 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:22.213628    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a6c278cc-4cc3-4584-b83e-376f5af90380-istio-token\" (UniqueName: \"kubernetes.io/projected/a6c278cc-4cc3-4584-b83e-376f5af90380-istio-token\") pod \"a6c278cc-4cc3-4584-b83e-376f5af90380\" (UID: \"a6c278cc-4cc3-4584-b83e-376f5af90380\") "
Oct 30 19:00:22.214102 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:22.213666    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a6c278cc-4cc3-4584-b83e-376f5af90380-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a6c278cc-4cc3-4584-b83e-376f5af90380-istiod-ca-cert\") pod \"a6c278cc-4cc3-4584-b83e-376f5af90380\" (UID: \"a6c278cc-4cc3-4584-b83e-376f5af90380\") "
Oct 30 19:00:22.214102 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:22.213706    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d33b7312-dc19-402a-9c2e-12e0b7eee906-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d33b7312-dc19-402a-9c2e-12e0b7eee906-istiod-ca-cert\") pod \"d33b7312-dc19-402a-9c2e-12e0b7eee906\" (UID: \"d33b7312-dc19-402a-9c2e-12e0b7eee906\") "
Oct 30 19:00:22.214102 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:22.213764    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/d33b7312-dc19-402a-9c2e-12e0b7eee906-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d33b7312-dc19-402a-9c2e-12e0b7eee906-istio-podinfo\") pod \"d33b7312-dc19-402a-9c2e-12e0b7eee906\" (UID: \"d33b7312-dc19-402a-9c2e-12e0b7eee906\") "
Oct 30 19:00:22.214102 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:22.213791    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a6c278cc-4cc3-4584-b83e-376f5af90380-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a6c278cc-4cc3-4584-b83e-376f5af90380-istio-data\") pod \"a6c278cc-4cc3-4584-b83e-376f5af90380\" (UID: \"a6c278cc-4cc3-4584-b83e-376f5af90380\") "
Oct 30 19:00:22.214102 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:22.213837    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2ba277ac-44b2-4d12-b873-6b584085f0d1-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ba277ac-44b2-4d12-b873-6b584085f0d1-credential-socket\") pod \"2ba277ac-44b2-4d12-b873-6b584085f0d1\" (UID: \"2ba277ac-44b2-4d12-b873-6b584085f0d1\") "
Oct 30 19:00:22.214102 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:22.213874    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d6294bd8-9418-4c1a-99c6-cf52ba98edaa-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d6294bd8-9418-4c1a-99c6-cf52ba98edaa-istio-data\") pod \"d6294bd8-9418-4c1a-99c6-cf52ba98edaa\" (UID: \"d6294bd8-9418-4c1a-99c6-cf52ba98edaa\") "
Oct 30 19:00:22.214102 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:22.213928    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a6c278cc-4cc3-4584-b83e-376f5af90380-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a6c278cc-4cc3-4584-b83e-376f5af90380-workload-certs\") pod \"a6c278cc-4cc3-4584-b83e-376f5af90380\" (UID: \"a6c278cc-4cc3-4584-b83e-376f5af90380\") "
Oct 30 19:00:22.214102 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:22.213969    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2ba277ac-44b2-4d12-b873-6b584085f0d1-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2ba277ac-44b2-4d12-b873-6b584085f0d1-istiod-ca-cert\") pod \"2ba277ac-44b2-4d12-b873-6b584085f0d1\" (UID: \"2ba277ac-44b2-4d12-b873-6b584085f0d1\") "
Oct 30 19:00:22.214102 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:22.214013    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0d5a7945-c6aa-453a-ac8c-1b23cddc1642-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0d5a7945-c6aa-453a-ac8c-1b23cddc1642-istio-envoy\") pod \"0d5a7945-c6aa-453a-ac8c-1b23cddc1642\" (UID: \"0d5a7945-c6aa-453a-ac8c-1b23cddc1642\") "
Oct 30 19:00:22.214102 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:22.214058    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0d5a7945-c6aa-453a-ac8c-1b23cddc1642-istio-token\" (UniqueName: \"kubernetes.io/projected/0d5a7945-c6aa-453a-ac8c-1b23cddc1642-istio-token\") pod \"0d5a7945-c6aa-453a-ac8c-1b23cddc1642\" (UID: \"0d5a7945-c6aa-453a-ac8c-1b23cddc1642\") "
Oct 30 19:00:22.214102 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:22.214098    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d33b7312-dc19-402a-9c2e-12e0b7eee906-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d33b7312-dc19-402a-9c2e-12e0b7eee906-istio-data\") pod \"d33b7312-dc19-402a-9c2e-12e0b7eee906\" (UID: \"d33b7312-dc19-402a-9c2e-12e0b7eee906\") "
Oct 30 19:00:22.214950 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:22.214135    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d33b7312-dc19-402a-9c2e-12e0b7eee906-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d33b7312-dc19-402a-9c2e-12e0b7eee906-workload-certs\") pod \"d33b7312-dc19-402a-9c2e-12e0b7eee906\" (UID: \"d33b7312-dc19-402a-9c2e-12e0b7eee906\") "
Oct 30 19:00:22.214950 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:22.214173    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0d5a7945-c6aa-453a-ac8c-1b23cddc1642-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d5a7945-c6aa-453a-ac8c-1b23cddc1642-credential-socket\") pod \"0d5a7945-c6aa-453a-ac8c-1b23cddc1642\" (UID: \"0d5a7945-c6aa-453a-ac8c-1b23cddc1642\") "
Oct 30 19:00:22.214950 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:22.214200    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2ba277ac-44b2-4d12-b873-6b584085f0d1-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2ba277ac-44b2-4d12-b873-6b584085f0d1-istio-data\") pod \"2ba277ac-44b2-4d12-b873-6b584085f0d1\" (UID: \"2ba277ac-44b2-4d12-b873-6b584085f0d1\") "
Oct 30 19:00:22.214950 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:22.214252    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2ba277ac-44b2-4d12-b873-6b584085f0d1-kube-api-access-stnfd\" (UniqueName: \"kubernetes.io/projected/2ba277ac-44b2-4d12-b873-6b584085f0d1-kube-api-access-stnfd\") pod \"2ba277ac-44b2-4d12-b873-6b584085f0d1\" (UID: \"2ba277ac-44b2-4d12-b873-6b584085f0d1\") "
Oct 30 19:00:22.214950 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:22.214315    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/d6294bd8-9418-4c1a-99c6-cf52ba98edaa-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d6294bd8-9418-4c1a-99c6-cf52ba98edaa-istio-podinfo\") pod \"d6294bd8-9418-4c1a-99c6-cf52ba98edaa\" (UID: \"d6294bd8-9418-4c1a-99c6-cf52ba98edaa\") "
Oct 30 19:00:22.214950 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:22.214413    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0d5a7945-c6aa-453a-ac8c-1b23cddc1642-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0d5a7945-c6aa-453a-ac8c-1b23cddc1642-workload-certs\") pod \"0d5a7945-c6aa-453a-ac8c-1b23cddc1642\" (UID: \"0d5a7945-c6aa-453a-ac8c-1b23cddc1642\") "
Oct 30 19:00:22.214950 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:22.214453    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d33b7312-dc19-402a-9c2e-12e0b7eee906-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d33b7312-dc19-402a-9c2e-12e0b7eee906-istio-envoy\") pod \"d33b7312-dc19-402a-9c2e-12e0b7eee906\" (UID: \"d33b7312-dc19-402a-9c2e-12e0b7eee906\") "
Oct 30 19:00:22.214950 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:22.214496    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0d5a7945-c6aa-453a-ac8c-1b23cddc1642-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d5a7945-c6aa-453a-ac8c-1b23cddc1642-workload-socket\") pod \"0d5a7945-c6aa-453a-ac8c-1b23cddc1642\" (UID: \"0d5a7945-c6aa-453a-ac8c-1b23cddc1642\") "
Oct 30 19:00:22.214950 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:22.214612    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/a6c278cc-4cc3-4584-b83e-376f5af90380-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a6c278cc-4cc3-4584-b83e-376f5af90380-istio-podinfo\") pod \"a6c278cc-4cc3-4584-b83e-376f5af90380\" (UID: \"a6c278cc-4cc3-4584-b83e-376f5af90380\") "
Oct 30 19:00:22.214950 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:22.214693    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2ba277ac-44b2-4d12-b873-6b584085f0d1-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2ba277ac-44b2-4d12-b873-6b584085f0d1-workload-certs\") pod \"2ba277ac-44b2-4d12-b873-6b584085f0d1\" (UID: \"2ba277ac-44b2-4d12-b873-6b584085f0d1\") "
Oct 30 19:00:22.214950 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:22.214742    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0d5a7945-c6aa-453a-ac8c-1b23cddc1642-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0d5a7945-c6aa-453a-ac8c-1b23cddc1642-istiod-ca-cert\") pod \"0d5a7945-c6aa-453a-ac8c-1b23cddc1642\" (UID: \"0d5a7945-c6aa-453a-ac8c-1b23cddc1642\") "
Oct 30 19:00:22.215889 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:22.215850    4888 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6294bd8-9418-4c1a-99c6-cf52ba98edaa-workload-socket\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:00:22.215889 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:22.215878    4888 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ba277ac-44b2-4d12-b873-6b584085f0d1-workload-socket\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:00:22.216030 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:22.215894    4888 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a6c278cc-4cc3-4584-b83e-376f5af90380-credential-socket\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:00:22.217757 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:22.213480    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d33b7312-dc19-402a-9c2e-12e0b7eee906-credential-socket" pod "d33b7312-dc19-402a-9c2e-12e0b7eee906" (UID: "d33b7312-dc19-402a-9c2e-12e0b7eee906"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:00:22.217757 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:22.213664    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d6294bd8-9418-4c1a-99c6-cf52ba98edaa-workload-certs" pod "d6294bd8-9418-4c1a-99c6-cf52ba98edaa" (UID: "d6294bd8-9418-4c1a-99c6-cf52ba98edaa"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:00:22.217757 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:22.213838    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d5a7945-c6aa-453a-ac8c-1b23cddc1642-istio-data" pod "0d5a7945-c6aa-453a-ac8c-1b23cddc1642" (UID: "0d5a7945-c6aa-453a-ac8c-1b23cddc1642"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:00:22.217757 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:22.213849    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d33b7312-dc19-402a-9c2e-12e0b7eee906-workload-socket" pod "d33b7312-dc19-402a-9c2e-12e0b7eee906" (UID: "d33b7312-dc19-402a-9c2e-12e0b7eee906"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:00:22.217757 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:22.214020    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a6c278cc-4cc3-4584-b83e-376f5af90380-workload-socket" pod "a6c278cc-4cc3-4584-b83e-376f5af90380" (UID: "a6c278cc-4cc3-4584-b83e-376f5af90380"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:00:22.217757 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:22.214027    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d6294bd8-9418-4c1a-99c6-cf52ba98edaa-credential-socket" pod "d6294bd8-9418-4c1a-99c6-cf52ba98edaa" (UID: "d6294bd8-9418-4c1a-99c6-cf52ba98edaa"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:00:22.217757 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:22.214342    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d6294bd8-9418-4c1a-99c6-cf52ba98edaa-istiod-ca-cert" pod "d6294bd8-9418-4c1a-99c6-cf52ba98edaa" (UID: "d6294bd8-9418-4c1a-99c6-cf52ba98edaa"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:00:22.217757 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:22.214349    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a6c278cc-4cc3-4584-b83e-376f5af90380-istiod-ca-cert" pod "a6c278cc-4cc3-4584-b83e-376f5af90380" (UID: "a6c278cc-4cc3-4584-b83e-376f5af90380"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:00:22.217757 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:22.215838    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0d5a7945-c6aa-453a-ac8c-1b23cddc1642-kube-api-access-qfxlx" pod "0d5a7945-c6aa-453a-ac8c-1b23cddc1642" (UID: "0d5a7945-c6aa-453a-ac8c-1b23cddc1642"). InnerVolumeSpecName "kube-api-access-qfxlx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:00:22.217757 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:22.215855    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d33b7312-dc19-402a-9c2e-12e0b7eee906-istio-token" pod "d33b7312-dc19-402a-9c2e-12e0b7eee906" (UID: "d33b7312-dc19-402a-9c2e-12e0b7eee906"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:00:22.217757 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:22.215873    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a6c278cc-4cc3-4584-b83e-376f5af90380-istio-envoy" pod "a6c278cc-4cc3-4584-b83e-376f5af90380" (UID: "a6c278cc-4cc3-4584-b83e-376f5af90380"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:00:22.217757 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:22.216214    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0d5a7945-c6aa-453a-ac8c-1b23cddc1642-istiod-ca-cert" pod "0d5a7945-c6aa-453a-ac8c-1b23cddc1642" (UID: "0d5a7945-c6aa-453a-ac8c-1b23cddc1642"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:00:22.217757 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:22.216363    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0d5a7945-c6aa-453a-ac8c-1b23cddc1642-istio-podinfo" pod "0d5a7945-c6aa-453a-ac8c-1b23cddc1642" (UID: "0d5a7945-c6aa-453a-ac8c-1b23cddc1642"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:00:22.217757 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:22.216537    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d33b7312-dc19-402a-9c2e-12e0b7eee906-istiod-ca-cert" pod "d33b7312-dc19-402a-9c2e-12e0b7eee906" (UID: "d33b7312-dc19-402a-9c2e-12e0b7eee906"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:00:22.217757 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:22.217630    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ba277ac-44b2-4d12-b873-6b584085f0d1-istio-envoy" pod "2ba277ac-44b2-4d12-b873-6b584085f0d1" (UID: "2ba277ac-44b2-4d12-b873-6b584085f0d1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:00:22.218757 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:22.217859    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d6294bd8-9418-4c1a-99c6-cf52ba98edaa-kube-api-access-52g5c" pod "d6294bd8-9418-4c1a-99c6-cf52ba98edaa" (UID: "d6294bd8-9418-4c1a-99c6-cf52ba98edaa"). InnerVolumeSpecName "kube-api-access-52g5c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:00:22.218757 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:22.218073    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d33b7312-dc19-402a-9c2e-12e0b7eee906-kube-api-access-vdt4h" pod "d33b7312-dc19-402a-9c2e-12e0b7eee906" (UID: "d33b7312-dc19-402a-9c2e-12e0b7eee906"). InnerVolumeSpecName "kube-api-access-vdt4h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:00:22.218757 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:22.218357    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d33b7312-dc19-402a-9c2e-12e0b7eee906-workload-certs" pod "d33b7312-dc19-402a-9c2e-12e0b7eee906" (UID: "d33b7312-dc19-402a-9c2e-12e0b7eee906"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:00:22.218757 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:22.218416    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d6294bd8-9418-4c1a-99c6-cf52ba98edaa-istio-token" pod "d6294bd8-9418-4c1a-99c6-cf52ba98edaa" (UID: "d6294bd8-9418-4c1a-99c6-cf52ba98edaa"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:00:22.218757 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:22.218456    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a6c278cc-4cc3-4584-b83e-376f5af90380-istio-token" pod "a6c278cc-4cc3-4584-b83e-376f5af90380" (UID: "a6c278cc-4cc3-4584-b83e-376f5af90380"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:00:22.218757 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:22.218590    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a6c278cc-4cc3-4584-b83e-376f5af90380-istio-data" pod "a6c278cc-4cc3-4584-b83e-376f5af90380" (UID: "a6c278cc-4cc3-4584-b83e-376f5af90380"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:00:22.218757 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:22.218668    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ba277ac-44b2-4d12-b873-6b584085f0d1-istio-data" pod "2ba277ac-44b2-4d12-b873-6b584085f0d1" (UID: "2ba277ac-44b2-4d12-b873-6b584085f0d1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:00:22.218757 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:22.218679    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d5a7945-c6aa-453a-ac8c-1b23cddc1642-workload-certs" pod "0d5a7945-c6aa-453a-ac8c-1b23cddc1642" (UID: "0d5a7945-c6aa-453a-ac8c-1b23cddc1642"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:00:22.219207 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:22.218826    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2ba277ac-44b2-4d12-b873-6b584085f0d1-istio-podinfo" pod "2ba277ac-44b2-4d12-b873-6b584085f0d1" (UID: "2ba277ac-44b2-4d12-b873-6b584085f0d1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:00:22.219207 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:22.218828    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d5a7945-c6aa-453a-ac8c-1b23cddc1642-credential-socket" pod "0d5a7945-c6aa-453a-ac8c-1b23cddc1642" (UID: "0d5a7945-c6aa-453a-ac8c-1b23cddc1642"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:00:22.219207 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:22.218835    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ba277ac-44b2-4d12-b873-6b584085f0d1-credential-socket" pod "2ba277ac-44b2-4d12-b873-6b584085f0d1" (UID: "2ba277ac-44b2-4d12-b873-6b584085f0d1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:00:22.219207 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:22.218986    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d5a7945-c6aa-453a-ac8c-1b23cddc1642-workload-socket" pod "0d5a7945-c6aa-453a-ac8c-1b23cddc1642" (UID: "0d5a7945-c6aa-453a-ac8c-1b23cddc1642"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:00:22.219207 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:22.219185    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d33b7312-dc19-402a-9c2e-12e0b7eee906-istio-podinfo" pod "d33b7312-dc19-402a-9c2e-12e0b7eee906" (UID: "d33b7312-dc19-402a-9c2e-12e0b7eee906"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:00:22.219569 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:22.219166    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d6294bd8-9418-4c1a-99c6-cf52ba98edaa-istio-data" pod "d6294bd8-9418-4c1a-99c6-cf52ba98edaa" (UID: "d6294bd8-9418-4c1a-99c6-cf52ba98edaa"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:00:22.219569 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:22.219507    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a6c278cc-4cc3-4584-b83e-376f5af90380-workload-certs" pod "a6c278cc-4cc3-4584-b83e-376f5af90380" (UID: "a6c278cc-4cc3-4584-b83e-376f5af90380"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:00:22.219830 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:22.219805    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2ba277ac-44b2-4d12-b873-6b584085f0d1-istiod-ca-cert" pod "2ba277ac-44b2-4d12-b873-6b584085f0d1" (UID: "2ba277ac-44b2-4d12-b873-6b584085f0d1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:00:22.220208 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:22.220172    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d6294bd8-9418-4c1a-99c6-cf52ba98edaa-istio-envoy" pod "d6294bd8-9418-4c1a-99c6-cf52ba98edaa" (UID: "d6294bd8-9418-4c1a-99c6-cf52ba98edaa"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:00:22.220319 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:22.220251    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2ba277ac-44b2-4d12-b873-6b584085f0d1-istio-token" pod "2ba277ac-44b2-4d12-b873-6b584085f0d1" (UID: "2ba277ac-44b2-4d12-b873-6b584085f0d1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:00:22.220499 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:22.220480    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ba277ac-44b2-4d12-b873-6b584085f0d1-workload-certs" pod "2ba277ac-44b2-4d12-b873-6b584085f0d1" (UID: "2ba277ac-44b2-4d12-b873-6b584085f0d1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:00:22.220654 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:22.220638    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d33b7312-dc19-402a-9c2e-12e0b7eee906-istio-data" pod "d33b7312-dc19-402a-9c2e-12e0b7eee906" (UID: "d33b7312-dc19-402a-9c2e-12e0b7eee906"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:00:22.221502 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:22.221475    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a6c278cc-4cc3-4584-b83e-376f5af90380-kube-api-access-p5wb8" pod "a6c278cc-4cc3-4584-b83e-376f5af90380" (UID: "a6c278cc-4cc3-4584-b83e-376f5af90380"). InnerVolumeSpecName "kube-api-access-p5wb8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:00:22.221757 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:22.221731    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d33b7312-dc19-402a-9c2e-12e0b7eee906-istio-envoy" pod "d33b7312-dc19-402a-9c2e-12e0b7eee906" (UID: "d33b7312-dc19-402a-9c2e-12e0b7eee906"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:00:22.221844 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:22.221826    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d6294bd8-9418-4c1a-99c6-cf52ba98edaa-istio-podinfo" pod "d6294bd8-9418-4c1a-99c6-cf52ba98edaa" (UID: "d6294bd8-9418-4c1a-99c6-cf52ba98edaa"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:00:22.221922 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:22.221893    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a6c278cc-4cc3-4584-b83e-376f5af90380-istio-podinfo" pod "a6c278cc-4cc3-4584-b83e-376f5af90380" (UID: "a6c278cc-4cc3-4584-b83e-376f5af90380"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:00:22.222127 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:22.222109    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d5a7945-c6aa-453a-ac8c-1b23cddc1642-istio-envoy" pod "0d5a7945-c6aa-453a-ac8c-1b23cddc1642" (UID: "0d5a7945-c6aa-453a-ac8c-1b23cddc1642"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:00:22.222233 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:22.222219    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2ba277ac-44b2-4d12-b873-6b584085f0d1-kube-api-access-stnfd" pod "2ba277ac-44b2-4d12-b873-6b584085f0d1" (UID: "2ba277ac-44b2-4d12-b873-6b584085f0d1"). InnerVolumeSpecName "kube-api-access-stnfd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:00:22.222708 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:22.222691    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0d5a7945-c6aa-453a-ac8c-1b23cddc1642-istio-token" pod "0d5a7945-c6aa-453a-ac8c-1b23cddc1642" (UID: "0d5a7945-c6aa-453a-ac8c-1b23cddc1642"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:00:22.317711 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:22.317621    4888 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0d5a7945-c6aa-453a-ac8c-1b23cddc1642-istio-token\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:00:22.317711 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:22.317652    4888 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d33b7312-dc19-402a-9c2e-12e0b7eee906-istio-data\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:00:22.317711 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:22.317667    4888 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d33b7312-dc19-402a-9c2e-12e0b7eee906-workload-certs\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:00:22.317711 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:22.317680    4888 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d5a7945-c6aa-453a-ac8c-1b23cddc1642-credential-socket\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:00:22.317711 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:22.317695    4888 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2ba277ac-44b2-4d12-b873-6b584085f0d1-istio-data\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:00:22.317711 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:22.317707    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-stnfd\" (UniqueName: \"kubernetes.io/projected/2ba277ac-44b2-4d12-b873-6b584085f0d1-kube-api-access-stnfd\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:00:22.318146 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:22.317723    4888 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d6294bd8-9418-4c1a-99c6-cf52ba98edaa-istio-podinfo\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:00:22.318146 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:22.317735    4888 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0d5a7945-c6aa-453a-ac8c-1b23cddc1642-workload-certs\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:00:22.318146 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:22.317748    4888 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d33b7312-dc19-402a-9c2e-12e0b7eee906-istio-envoy\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:00:22.318146 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:22.317758    4888 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d5a7945-c6aa-453a-ac8c-1b23cddc1642-workload-socket\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:00:22.318146 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:22.317770    4888 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a6c278cc-4cc3-4584-b83e-376f5af90380-istio-podinfo\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:00:22.318146 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:22.317783    4888 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2ba277ac-44b2-4d12-b873-6b584085f0d1-workload-certs\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:00:22.318146 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:22.317795    4888 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0d5a7945-c6aa-453a-ac8c-1b23cddc1642-istiod-ca-cert\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:00:22.318146 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:22.317806    4888 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0d5a7945-c6aa-453a-ac8c-1b23cddc1642-istio-podinfo\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:00:22.318146 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:22.317820    4888 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d33b7312-dc19-402a-9c2e-12e0b7eee906-istio-token\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:00:22.318146 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:22.317833    4888 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d6294bd8-9418-4c1a-99c6-cf52ba98edaa-workload-certs\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:00:22.318146 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:22.317845    4888 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a6c278cc-4cc3-4584-b83e-376f5af90380-istio-envoy\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:00:22.318146 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:22.317857    4888 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d33b7312-dc19-402a-9c2e-12e0b7eee906-credential-socket\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:00:22.318146 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:22.317869    4888 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d6294bd8-9418-4c1a-99c6-cf52ba98edaa-istiod-ca-cert\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:00:22.318146 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:22.317881    4888 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2ba277ac-44b2-4d12-b873-6b584085f0d1-istio-podinfo\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:00:22.318146 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:22.317896    4888 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2ba277ac-44b2-4d12-b873-6b584085f0d1-istio-envoy\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:00:22.318146 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:22.317909    4888 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d33b7312-dc19-402a-9c2e-12e0b7eee906-workload-socket\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:00:22.318146 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:22.317927    4888 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d6294bd8-9418-4c1a-99c6-cf52ba98edaa-istio-envoy\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:00:22.318146 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:22.317941    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qfxlx\" (UniqueName: \"kubernetes.io/projected/0d5a7945-c6aa-453a-ac8c-1b23cddc1642-kube-api-access-qfxlx\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:00:22.318146 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:22.317955    4888 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a6c278cc-4cc3-4584-b83e-376f5af90380-workload-socket\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:00:22.318146 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:22.317970    4888 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6294bd8-9418-4c1a-99c6-cf52ba98edaa-credential-socket\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:00:22.318146 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:22.317985    4888 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0d5a7945-c6aa-453a-ac8c-1b23cddc1642-istio-data\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:00:22.318146 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:22.317998    4888 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2ba277ac-44b2-4d12-b873-6b584085f0d1-istio-token\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:00:22.318146 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:22.318012    4888 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d6294bd8-9418-4c1a-99c6-cf52ba98edaa-istio-token\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:00:22.318146 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:22.318026    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vdt4h\" (UniqueName: \"kubernetes.io/projected/d33b7312-dc19-402a-9c2e-12e0b7eee906-kube-api-access-vdt4h\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:00:22.318146 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:22.318041    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p5wb8\" (UniqueName: \"kubernetes.io/projected/a6c278cc-4cc3-4584-b83e-376f5af90380-kube-api-access-p5wb8\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:00:22.318146 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:22.318055    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-52g5c\" (UniqueName: \"kubernetes.io/projected/d6294bd8-9418-4c1a-99c6-cf52ba98edaa-kube-api-access-52g5c\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:00:22.318146 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:22.318070    4888 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a6c278cc-4cc3-4584-b83e-376f5af90380-istio-token\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:00:22.318146 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:22.318084    4888 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a6c278cc-4cc3-4584-b83e-376f5af90380-istiod-ca-cert\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:00:22.318146 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:22.318097    4888 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d33b7312-dc19-402a-9c2e-12e0b7eee906-istiod-ca-cert\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:00:22.318146 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:22.318111    4888 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d33b7312-dc19-402a-9c2e-12e0b7eee906-istio-podinfo\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:00:22.318146 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:22.318126    4888 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a6c278cc-4cc3-4584-b83e-376f5af90380-istio-data\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:00:22.318146 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:22.318140    4888 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2ba277ac-44b2-4d12-b873-6b584085f0d1-credential-socket\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:00:22.318146 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:22.318154    4888 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d6294bd8-9418-4c1a-99c6-cf52ba98edaa-istio-data\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:00:22.319189 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:22.318167    4888 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a6c278cc-4cc3-4584-b83e-376f5af90380-workload-certs\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:00:22.319189 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:22.318181    4888 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2ba277ac-44b2-4d12-b873-6b584085f0d1-istiod-ca-cert\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:00:22.319189 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:22.318195    4888 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0d5a7945-c6aa-453a-ac8c-1b23cddc1642-istio-envoy\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:00:22.937356 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:22.937329    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-hnndm"
Oct 30 19:00:22.937356 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:22.937336    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-hnndm" event={"ID":"d6294bd8-9418-4c1a-99c6-cf52ba98edaa","Type":"ContainerDied","Data":"61a09bcdd207ce8f295a456a5643432a0122b42fef3cd07d04189539dcde3632"}
Oct 30 19:00:22.937845 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:22.937393    4888 scope.go:122] "RemoveContainer" containerID="461c7337f2cbaa04e696b602018356667011c43f993fb2076a7082372dff2f93"
Oct 30 19:00:22.939069 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:22.938949    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9rstj" event={"ID":"0d5a7945-c6aa-453a-ac8c-1b23cddc1642","Type":"ContainerDied","Data":"01f81bcb2ae7b40ff48843868e0e28a2f1b1b90d25c435acd825d8852b967415"}
Oct 30 19:00:22.939069 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:22.938956    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9rstj"
Oct 30 19:00:22.940607 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:22.940589    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4xpmd"
Oct 30 19:00:22.940715 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:22.940637    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4xpmd" event={"ID":"a6c278cc-4cc3-4584-b83e-376f5af90380","Type":"ContainerDied","Data":"b163fe81cccc74dfb72891104e1bba7e5b453d1f2966fc752d7f2e9408901162"}
Oct 30 19:00:22.948462 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:22.947167    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4p2zp" event={"ID":"2ba277ac-44b2-4d12-b873-6b584085f0d1","Type":"ContainerDied","Data":"429dec9d2859989e15abb5415eea6dcf7cea0396957b7e3516500c3ab23fb664"}
Oct 30 19:00:22.948462 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:22.947256    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4p2zp"
Oct 30 19:00:22.951145 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:22.951084    4888 scope.go:122] "RemoveContainer" containerID="f0bc676121e67f034883592e084979545dbb256182eae9e11347cd66b32d7caa"
Oct 30 19:00:22.951961 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:22.951768    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-ngqb5" event={"ID":"d33b7312-dc19-402a-9c2e-12e0b7eee906","Type":"ContainerDied","Data":"04ac7af79bbf2806ade7e2746c66ab428c55a06cd254f8c3a2d19cbd262cb97f"}
Oct 30 19:00:22.951961 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:22.951779    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-ngqb5"
Oct 30 19:00:22.964965 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:22.964943    4888 scope.go:122] "RemoveContainer" containerID="7db3612372458c749c6ad16c7a0326ecb1f9259423087b1413072520e71655c7"
Oct 30 19:00:22.977140 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:22.977124    4888 scope.go:122] "RemoveContainer" containerID="5432460e6fed255408e2c642cd55b7ce9c4d906e7636c508abe5d5f10ea2df2b"
Oct 30 19:00:22.986075 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:22.986056    4888 scope.go:122] "RemoveContainer" containerID="d0fa2b0cac5d3e4044a8a2d1a39fa843e6d134086a59257db9c99b6cb7c6619f"
Oct 30 19:00:23.006361 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:23.006331    4888 patch_prober.go:29] interesting pod/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-hnndm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.89:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:00:23.006454 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:23.006381    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-hnndm" podUID="d6294bd8-9418-4c1a-99c6-cf52ba98edaa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.89:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:00:23.121086 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:23.121050    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-hnndm"]
Oct 30 19:00:23.124609 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:23.124583    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-hnndm"]
Oct 30 19:00:23.249178 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:23.249105    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4xpmd"]
Oct 30 19:00:23.256648 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:23.256622    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4xpmd"]
Oct 30 19:00:23.371832 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:23.371796    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9rstj"]
Oct 30 19:00:23.375303 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:23.375265    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-9rstj"]
Oct 30 19:00:23.490624 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:23.490578    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4p2zp"]
Oct 30 19:00:23.493960 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:23.493940    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-4p2zp"]
Oct 30 19:00:23.613400 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:23.613374    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-ngqb5"]
Oct 30 19:00:23.622659 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:23.622638    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-e7929d-openshift-default-7f6f56644f-ngqb5"]
Oct 30 19:00:23.893725 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:23.893643    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0d5a7945-c6aa-453a-ac8c-1b23cddc1642" path="/var/lib/kubelet/pods/0d5a7945-c6aa-453a-ac8c-1b23cddc1642/volumes"
Oct 30 19:00:23.894234 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:23.894213    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2ba277ac-44b2-4d12-b873-6b584085f0d1" path="/var/lib/kubelet/pods/2ba277ac-44b2-4d12-b873-6b584085f0d1/volumes"
Oct 30 19:00:23.894769 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:23.894749    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a6c278cc-4cc3-4584-b83e-376f5af90380" path="/var/lib/kubelet/pods/a6c278cc-4cc3-4584-b83e-376f5af90380/volumes"
Oct 30 19:00:23.895589 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:23.895569    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d33b7312-dc19-402a-9c2e-12e0b7eee906" path="/var/lib/kubelet/pods/d33b7312-dc19-402a-9c2e-12e0b7eee906/volumes"
Oct 30 19:00:23.896097 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:23.896075    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d6294bd8-9418-4c1a-99c6-cf52ba98edaa" path="/var/lib/kubelet/pods/d6294bd8-9418-4c1a-99c6-cf52ba98edaa/volumes"
Oct 30 19:00:29.464157 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:29.464122    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-rjmdz"]
Oct 30 19:00:35.621014 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:00:35.620979    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-jbsw6" containerName="extractor"
Oct 30 19:00:41.190399 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:41.190363    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-kgfbt"]
Oct 30 19:00:41.190829 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:41.190623    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-kgfbt" podUID="aa727ad6-0a18-4ce1-b48d-f2f01cd83677" containerName="istio-proxy" containerID="cri-o://e1994a709972cdd033ccff7397b1b7e1a34a7c132b3dc54cfc81feb6a3aab552" gracePeriod=30
Oct 30 19:00:41.249526 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:41.249494    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-ntnwm"]
Oct 30 19:00:41.249769 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:41.249747    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-ntnwm" podUID="73999d2f-99a7-43f6-99e9-de05ee01f6ce" containerName="istio-proxy" containerID="cri-o://ceb8b2d77237b854c9009c948cbf397b3de38c3588c00fa6d87fd12a69512bac" gracePeriod=30
Oct 30 19:00:46.655767 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:46.654786    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-kgfbt"
Oct 30 19:00:46.761064 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:46.761037    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/aa727ad6-0a18-4ce1-b48d-f2f01cd83677-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aa727ad6-0a18-4ce1-b48d-f2f01cd83677-istio-data\") pod \"aa727ad6-0a18-4ce1-b48d-f2f01cd83677\" (UID: \"aa727ad6-0a18-4ce1-b48d-f2f01cd83677\") "
Oct 30 19:00:46.761218 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:46.761093    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/aa727ad6-0a18-4ce1-b48d-f2f01cd83677-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aa727ad6-0a18-4ce1-b48d-f2f01cd83677-istiod-ca-cert\") pod \"aa727ad6-0a18-4ce1-b48d-f2f01cd83677\" (UID: \"aa727ad6-0a18-4ce1-b48d-f2f01cd83677\") "
Oct 30 19:00:46.761218 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:46.761143    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/aa727ad6-0a18-4ce1-b48d-f2f01cd83677-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aa727ad6-0a18-4ce1-b48d-f2f01cd83677-workload-certs\") pod \"aa727ad6-0a18-4ce1-b48d-f2f01cd83677\" (UID: \"aa727ad6-0a18-4ce1-b48d-f2f01cd83677\") "
Oct 30 19:00:46.761218 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:46.761214    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/aa727ad6-0a18-4ce1-b48d-f2f01cd83677-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aa727ad6-0a18-4ce1-b48d-f2f01cd83677-istio-podinfo\") pod \"aa727ad6-0a18-4ce1-b48d-f2f01cd83677\" (UID: \"aa727ad6-0a18-4ce1-b48d-f2f01cd83677\") "
Oct 30 19:00:46.761418 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:46.761346    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/aa727ad6-0a18-4ce1-b48d-f2f01cd83677-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa727ad6-0a18-4ce1-b48d-f2f01cd83677-workload-socket\") pod \"aa727ad6-0a18-4ce1-b48d-f2f01cd83677\" (UID: \"aa727ad6-0a18-4ce1-b48d-f2f01cd83677\") "
Oct 30 19:00:46.761418 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:46.761395    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/aa727ad6-0a18-4ce1-b48d-f2f01cd83677-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa727ad6-0a18-4ce1-b48d-f2f01cd83677-credential-socket\") pod \"aa727ad6-0a18-4ce1-b48d-f2f01cd83677\" (UID: \"aa727ad6-0a18-4ce1-b48d-f2f01cd83677\") "
Oct 30 19:00:46.761524 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:46.761490    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/aa727ad6-0a18-4ce1-b48d-f2f01cd83677-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aa727ad6-0a18-4ce1-b48d-f2f01cd83677-istio-envoy\") pod \"aa727ad6-0a18-4ce1-b48d-f2f01cd83677\" (UID: \"aa727ad6-0a18-4ce1-b48d-f2f01cd83677\") "
Oct 30 19:00:46.761577 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:46.761533    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/aa727ad6-0a18-4ce1-b48d-f2f01cd83677-kube-api-access-w6k7k\" (UniqueName: \"kubernetes.io/projected/aa727ad6-0a18-4ce1-b48d-f2f01cd83677-kube-api-access-w6k7k\") pod \"aa727ad6-0a18-4ce1-b48d-f2f01cd83677\" (UID: \"aa727ad6-0a18-4ce1-b48d-f2f01cd83677\") "
Oct 30 19:00:46.761640 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:46.761584    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/aa727ad6-0a18-4ce1-b48d-f2f01cd83677-istio-token\" (UniqueName: \"kubernetes.io/projected/aa727ad6-0a18-4ce1-b48d-f2f01cd83677-istio-token\") pod \"aa727ad6-0a18-4ce1-b48d-f2f01cd83677\" (UID: \"aa727ad6-0a18-4ce1-b48d-f2f01cd83677\") "
Oct 30 19:00:46.762868 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:46.761835    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-ntnwm"
Oct 30 19:00:46.763350 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:46.763125    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aa727ad6-0a18-4ce1-b48d-f2f01cd83677-istiod-ca-cert" pod "aa727ad6-0a18-4ce1-b48d-f2f01cd83677" (UID: "aa727ad6-0a18-4ce1-b48d-f2f01cd83677"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:00:46.763528 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:46.763483    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa727ad6-0a18-4ce1-b48d-f2f01cd83677-istio-data" pod "aa727ad6-0a18-4ce1-b48d-f2f01cd83677" (UID: "aa727ad6-0a18-4ce1-b48d-f2f01cd83677"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:00:46.763599 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:46.763518    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa727ad6-0a18-4ce1-b48d-f2f01cd83677-workload-socket" pod "aa727ad6-0a18-4ce1-b48d-f2f01cd83677" (UID: "aa727ad6-0a18-4ce1-b48d-f2f01cd83677"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:00:46.763758 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:46.763737    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa727ad6-0a18-4ce1-b48d-f2f01cd83677-credential-socket" pod "aa727ad6-0a18-4ce1-b48d-f2f01cd83677" (UID: "aa727ad6-0a18-4ce1-b48d-f2f01cd83677"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:00:46.764275 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:46.764251    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa727ad6-0a18-4ce1-b48d-f2f01cd83677-workload-certs" pod "aa727ad6-0a18-4ce1-b48d-f2f01cd83677" (UID: "aa727ad6-0a18-4ce1-b48d-f2f01cd83677"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:00:46.764930 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:46.764911    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/aa727ad6-0a18-4ce1-b48d-f2f01cd83677-istio-podinfo" pod "aa727ad6-0a18-4ce1-b48d-f2f01cd83677" (UID: "aa727ad6-0a18-4ce1-b48d-f2f01cd83677"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:00:46.765025 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:46.764941    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aa727ad6-0a18-4ce1-b48d-f2f01cd83677-istio-token" pod "aa727ad6-0a18-4ce1-b48d-f2f01cd83677" (UID: "aa727ad6-0a18-4ce1-b48d-f2f01cd83677"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:00:46.765853 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:46.765830    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aa727ad6-0a18-4ce1-b48d-f2f01cd83677-kube-api-access-w6k7k" pod "aa727ad6-0a18-4ce1-b48d-f2f01cd83677" (UID: "aa727ad6-0a18-4ce1-b48d-f2f01cd83677"). InnerVolumeSpecName "kube-api-access-w6k7k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:00:46.766219 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:46.766201    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa727ad6-0a18-4ce1-b48d-f2f01cd83677-istio-envoy" pod "aa727ad6-0a18-4ce1-b48d-f2f01cd83677" (UID: "aa727ad6-0a18-4ce1-b48d-f2f01cd83677"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:00:46.863249 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:46.863218    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/73999d2f-99a7-43f6-99e9-de05ee01f6ce-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/73999d2f-99a7-43f6-99e9-de05ee01f6ce-workload-certs\") pod \"73999d2f-99a7-43f6-99e9-de05ee01f6ce\" (UID: \"73999d2f-99a7-43f6-99e9-de05ee01f6ce\") "
Oct 30 19:00:46.863465 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:46.863359    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/73999d2f-99a7-43f6-99e9-de05ee01f6ce-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/73999d2f-99a7-43f6-99e9-de05ee01f6ce-istio-podinfo\") pod \"73999d2f-99a7-43f6-99e9-de05ee01f6ce\" (UID: \"73999d2f-99a7-43f6-99e9-de05ee01f6ce\") "
Oct 30 19:00:46.863465 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:46.863406    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/73999d2f-99a7-43f6-99e9-de05ee01f6ce-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/73999d2f-99a7-43f6-99e9-de05ee01f6ce-istio-data\") pod \"73999d2f-99a7-43f6-99e9-de05ee01f6ce\" (UID: \"73999d2f-99a7-43f6-99e9-de05ee01f6ce\") "
Oct 30 19:00:46.863465 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:46.863451    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/73999d2f-99a7-43f6-99e9-de05ee01f6ce-istio-token\" (UniqueName: \"kubernetes.io/projected/73999d2f-99a7-43f6-99e9-de05ee01f6ce-istio-token\") pod \"73999d2f-99a7-43f6-99e9-de05ee01f6ce\" (UID: \"73999d2f-99a7-43f6-99e9-de05ee01f6ce\") "
Oct 30 19:00:46.863648 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:46.863468    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/73999d2f-99a7-43f6-99e9-de05ee01f6ce-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/73999d2f-99a7-43f6-99e9-de05ee01f6ce-credential-socket\") pod \"73999d2f-99a7-43f6-99e9-de05ee01f6ce\" (UID: \"73999d2f-99a7-43f6-99e9-de05ee01f6ce\") "
Oct 30 19:00:46.863648 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:46.863492    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/73999d2f-99a7-43f6-99e9-de05ee01f6ce-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/73999d2f-99a7-43f6-99e9-de05ee01f6ce-istio-envoy\") pod \"73999d2f-99a7-43f6-99e9-de05ee01f6ce\" (UID: \"73999d2f-99a7-43f6-99e9-de05ee01f6ce\") "
Oct 30 19:00:46.863648 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:46.863509    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/73999d2f-99a7-43f6-99e9-de05ee01f6ce-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/73999d2f-99a7-43f6-99e9-de05ee01f6ce-workload-socket\") pod \"73999d2f-99a7-43f6-99e9-de05ee01f6ce\" (UID: \"73999d2f-99a7-43f6-99e9-de05ee01f6ce\") "
Oct 30 19:00:46.863648 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:46.863530    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/73999d2f-99a7-43f6-99e9-de05ee01f6ce-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/73999d2f-99a7-43f6-99e9-de05ee01f6ce-istiod-ca-cert\") pod \"73999d2f-99a7-43f6-99e9-de05ee01f6ce\" (UID: \"73999d2f-99a7-43f6-99e9-de05ee01f6ce\") "
Oct 30 19:00:46.863648 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:46.863644    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/73999d2f-99a7-43f6-99e9-de05ee01f6ce-kube-api-access-lmt4t\" (UniqueName: \"kubernetes.io/projected/73999d2f-99a7-43f6-99e9-de05ee01f6ce-kube-api-access-lmt4t\") pod \"73999d2f-99a7-43f6-99e9-de05ee01f6ce\" (UID: \"73999d2f-99a7-43f6-99e9-de05ee01f6ce\") "
Oct 30 19:00:46.863974 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:46.863960    4888 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa727ad6-0a18-4ce1-b48d-f2f01cd83677-workload-socket\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:00:46.864039 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:46.863977    4888 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa727ad6-0a18-4ce1-b48d-f2f01cd83677-credential-socket\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:00:46.864039 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:46.863985    4888 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aa727ad6-0a18-4ce1-b48d-f2f01cd83677-istio-envoy\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:00:46.864039 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:46.863994    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w6k7k\" (UniqueName: \"kubernetes.io/projected/aa727ad6-0a18-4ce1-b48d-f2f01cd83677-kube-api-access-w6k7k\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:00:46.864039 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:46.864003    4888 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aa727ad6-0a18-4ce1-b48d-f2f01cd83677-istio-token\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:00:46.864039 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:46.864011    4888 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aa727ad6-0a18-4ce1-b48d-f2f01cd83677-istio-data\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:00:46.864039 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:46.864021    4888 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aa727ad6-0a18-4ce1-b48d-f2f01cd83677-istiod-ca-cert\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:00:46.864039 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:46.864029    4888 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aa727ad6-0a18-4ce1-b48d-f2f01cd83677-workload-certs\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:00:46.864039 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:46.864036    4888 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aa727ad6-0a18-4ce1-b48d-f2f01cd83677-istio-podinfo\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:00:46.866473 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:46.866439    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73999d2f-99a7-43f6-99e9-de05ee01f6ce-credential-socket" pod "73999d2f-99a7-43f6-99e9-de05ee01f6ce" (UID: "73999d2f-99a7-43f6-99e9-de05ee01f6ce"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:00:46.866792 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:46.866689    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73999d2f-99a7-43f6-99e9-de05ee01f6ce-istio-data" pod "73999d2f-99a7-43f6-99e9-de05ee01f6ce" (UID: "73999d2f-99a7-43f6-99e9-de05ee01f6ce"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:00:46.866934 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:46.866912    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73999d2f-99a7-43f6-99e9-de05ee01f6ce-workload-certs" pod "73999d2f-99a7-43f6-99e9-de05ee01f6ce" (UID: "73999d2f-99a7-43f6-99e9-de05ee01f6ce"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:00:46.883349 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:46.867406    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/73999d2f-99a7-43f6-99e9-de05ee01f6ce-istio-token" pod "73999d2f-99a7-43f6-99e9-de05ee01f6ce" (UID: "73999d2f-99a7-43f6-99e9-de05ee01f6ce"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:00:46.883349 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:46.867442    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73999d2f-99a7-43f6-99e9-de05ee01f6ce-istio-envoy" pod "73999d2f-99a7-43f6-99e9-de05ee01f6ce" (UID: "73999d2f-99a7-43f6-99e9-de05ee01f6ce"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:00:46.883349 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:46.881600    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/73999d2f-99a7-43f6-99e9-de05ee01f6ce-istiod-ca-cert" pod "73999d2f-99a7-43f6-99e9-de05ee01f6ce" (UID: "73999d2f-99a7-43f6-99e9-de05ee01f6ce"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:00:46.893902 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:46.893867    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73999d2f-99a7-43f6-99e9-de05ee01f6ce-workload-socket" pod "73999d2f-99a7-43f6-99e9-de05ee01f6ce" (UID: "73999d2f-99a7-43f6-99e9-de05ee01f6ce"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:00:46.915781 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:46.911471    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/73999d2f-99a7-43f6-99e9-de05ee01f6ce-kube-api-access-lmt4t" pod "73999d2f-99a7-43f6-99e9-de05ee01f6ce" (UID: "73999d2f-99a7-43f6-99e9-de05ee01f6ce"). InnerVolumeSpecName "kube-api-access-lmt4t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:00:46.917520 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:46.917166    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/73999d2f-99a7-43f6-99e9-de05ee01f6ce-istio-podinfo" pod "73999d2f-99a7-43f6-99e9-de05ee01f6ce" (UID: "73999d2f-99a7-43f6-99e9-de05ee01f6ce"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:00:46.965727 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:46.965431    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lmt4t\" (UniqueName: \"kubernetes.io/projected/73999d2f-99a7-43f6-99e9-de05ee01f6ce-kube-api-access-lmt4t\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:00:46.965727 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:46.965456    4888 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/73999d2f-99a7-43f6-99e9-de05ee01f6ce-workload-certs\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:00:46.965727 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:46.965465    4888 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/73999d2f-99a7-43f6-99e9-de05ee01f6ce-istio-podinfo\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:00:46.965727 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:46.965474    4888 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/73999d2f-99a7-43f6-99e9-de05ee01f6ce-istio-data\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:00:46.965727 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:46.965483    4888 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/73999d2f-99a7-43f6-99e9-de05ee01f6ce-istio-token\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:00:46.965727 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:46.965491    4888 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/73999d2f-99a7-43f6-99e9-de05ee01f6ce-credential-socket\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:00:46.965727 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:46.965500    4888 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/73999d2f-99a7-43f6-99e9-de05ee01f6ce-istio-envoy\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:00:46.965727 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:46.965508    4888 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/73999d2f-99a7-43f6-99e9-de05ee01f6ce-workload-socket\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:00:46.965727 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:46.965518    4888 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/73999d2f-99a7-43f6-99e9-de05ee01f6ce-istiod-ca-cert\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:00:47.212306 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:47.208737    4888 generic.go:356] "Generic (PLEG): container finished" podID="aa727ad6-0a18-4ce1-b48d-f2f01cd83677" containerID="e1994a709972cdd033ccff7397b1b7e1a34a7c132b3dc54cfc81feb6a3aab552" exitCode=0
Oct 30 19:00:47.212306 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:47.208927    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-kgfbt" event={"ID":"aa727ad6-0a18-4ce1-b48d-f2f01cd83677","Type":"ContainerDied","Data":"e1994a709972cdd033ccff7397b1b7e1a34a7c132b3dc54cfc81feb6a3aab552"}
Oct 30 19:00:47.212306 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:47.209008    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-kgfbt" event={"ID":"aa727ad6-0a18-4ce1-b48d-f2f01cd83677","Type":"ContainerDied","Data":"295f35158c3a5062de68cad069aadb069c2f31643b7c3ead00214c2d33b7ef2c"}
Oct 30 19:00:47.212306 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:47.209043    4888 scope.go:122] "RemoveContainer" containerID="e1994a709972cdd033ccff7397b1b7e1a34a7c132b3dc54cfc81feb6a3aab552"
Oct 30 19:00:47.212306 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:47.209329    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-kgfbt"
Oct 30 19:00:47.231469 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:47.229225    4888 generic.go:356] "Generic (PLEG): container finished" podID="73999d2f-99a7-43f6-99e9-de05ee01f6ce" containerID="ceb8b2d77237b854c9009c948cbf397b3de38c3588c00fa6d87fd12a69512bac" exitCode=0
Oct 30 19:00:47.253322 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:47.242520    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-ntnwm" event={"ID":"73999d2f-99a7-43f6-99e9-de05ee01f6ce","Type":"ContainerDied","Data":"ceb8b2d77237b854c9009c948cbf397b3de38c3588c00fa6d87fd12a69512bac"}
Oct 30 19:00:47.253322 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:47.242603    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-ntnwm" event={"ID":"73999d2f-99a7-43f6-99e9-de05ee01f6ce","Type":"ContainerDied","Data":"db284f3aae7efee03d1b133be389244cdaf3aa2df19f7d523388ec2c367cd070"}
Oct 30 19:00:47.253322 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:47.242691    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-ntnwm"
Oct 30 19:00:47.340661 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:47.332534    4888 scope.go:122] "RemoveContainer" containerID="e1994a709972cdd033ccff7397b1b7e1a34a7c132b3dc54cfc81feb6a3aab552"
Oct 30 19:00:47.343915 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:00:47.343861    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e1994a709972cdd033ccff7397b1b7e1a34a7c132b3dc54cfc81feb6a3aab552\": container with ID starting with e1994a709972cdd033ccff7397b1b7e1a34a7c132b3dc54cfc81feb6a3aab552 not found: ID does not exist" containerID="e1994a709972cdd033ccff7397b1b7e1a34a7c132b3dc54cfc81feb6a3aab552"
Oct 30 19:00:47.344073 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:47.343925    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e1994a709972cdd033ccff7397b1b7e1a34a7c132b3dc54cfc81feb6a3aab552"} err="failed to get container status \"e1994a709972cdd033ccff7397b1b7e1a34a7c132b3dc54cfc81feb6a3aab552\": rpc error: code = NotFound desc = could not find container \"e1994a709972cdd033ccff7397b1b7e1a34a7c132b3dc54cfc81feb6a3aab552\": container with ID starting with e1994a709972cdd033ccff7397b1b7e1a34a7c132b3dc54cfc81feb6a3aab552 not found: ID does not exist"
Oct 30 19:00:47.344073 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:47.343966    4888 scope.go:122] "RemoveContainer" containerID="ceb8b2d77237b854c9009c948cbf397b3de38c3588c00fa6d87fd12a69512bac"
Oct 30 19:00:47.356614 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:47.356588    4888 scope.go:122] "RemoveContainer" containerID="ceb8b2d77237b854c9009c948cbf397b3de38c3588c00fa6d87fd12a69512bac"
Oct 30 19:00:47.357060 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:00:47.357027    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ceb8b2d77237b854c9009c948cbf397b3de38c3588c00fa6d87fd12a69512bac\": container with ID starting with ceb8b2d77237b854c9009c948cbf397b3de38c3588c00fa6d87fd12a69512bac not found: ID does not exist" containerID="ceb8b2d77237b854c9009c948cbf397b3de38c3588c00fa6d87fd12a69512bac"
Oct 30 19:00:47.357272 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:47.357242    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ceb8b2d77237b854c9009c948cbf397b3de38c3588c00fa6d87fd12a69512bac"} err="failed to get container status \"ceb8b2d77237b854c9009c948cbf397b3de38c3588c00fa6d87fd12a69512bac\": rpc error: code = NotFound desc = could not find container \"ceb8b2d77237b854c9009c948cbf397b3de38c3588c00fa6d87fd12a69512bac\": container with ID starting with ceb8b2d77237b854c9009c948cbf397b3de38c3588c00fa6d87fd12a69512bac not found: ID does not exist"
Oct 30 19:00:47.517443 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:47.517255    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-kgfbt"]
Oct 30 19:00:47.526064 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:47.526018    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-kgfbt"]
Oct 30 19:00:47.634309 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:47.633691    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-ntnwm"]
Oct 30 19:00:47.650328 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:47.648329    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-ntnwm"]
Oct 30 19:00:47.934229 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:47.933825    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="73999d2f-99a7-43f6-99e9-de05ee01f6ce" path="/var/lib/kubelet/pods/73999d2f-99a7-43f6-99e9-de05ee01f6ce/volumes"
Oct 30 19:00:47.934942 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:47.934921    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="aa727ad6-0a18-4ce1-b48d-f2f01cd83677" path="/var/lib/kubelet/pods/aa727ad6-0a18-4ce1-b48d-f2f01cd83677/volumes"
Oct 30 19:00:49.691312 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:49.688375    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-3b6013/distinct-2-w3-3b6013-kserve-7cc5454db-mqknj" podUID="e76c7b20-be2f-4ed2-8dc4-bef127ecc0c9" containerName="main" probeResult="failure" output="Get \"https://10.132.0.98:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:00:49.711620 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:49.711579    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-4a7a52/premium-w2-4a7a52-kserve-746d9579f6-hn7hl" podUID="3160b54f-1df9-48de-80d9-8d8a91d47f56" containerName="main" probeResult="failure" output="Get \"https://10.132.0.83:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:00:50.079309 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:50.078412    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e7929d/distinct-w4-e7929d-kserve-5bc9ff5bf5-kmrwv" podUID="abfd315d-a01a-42dd-9c06-2a80d3f8579a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.88:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:00:50.206861 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:50.206818    4888 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-3b6013/distinct-w3-3b6013-kserve-857f55d885-8fcrq" podUID="51927877-4d99-4797-82f7-1684f6ecda6b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.92:8000/health\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:00:50.340009 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:50.339902    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-rkm4t" podUID="1ab20f74-2c73-4d56-a604-aad4f8289450" containerName="main" probeResult="failure" output="Get \"https://10.132.0.53:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:00:55.696800 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:55.696766    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-nvlbr"]
Oct 30 19:00:55.697590 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:55.697570    4888 cpu_manager.go:414] "containerMap: removing container" podUID="2ba277ac-44b2-4d12-b873-6b584085f0d1" containerName="istio-proxy"
Oct 30 19:00:55.697700 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:55.697592    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2ba277ac-44b2-4d12-b873-6b584085f0d1" containerName="istio-proxy"
Oct 30 19:00:55.697700 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:55.697609    4888 cpu_manager.go:414] "containerMap: removing container" podUID="b40b1755-0472-48e2-be01-417743104be6" containerName="cleanup"
Oct 30 19:00:55.697700 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:55.697616    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b40b1755-0472-48e2-be01-417743104be6" containerName="cleanup"
Oct 30 19:00:55.697700 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:55.697632    4888 cpu_manager.go:414] "containerMap: removing container" podUID="3dc2dd63-b055-4a45-a14e-8854ab1243d7" containerName="cleanup"
Oct 30 19:00:55.697700 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:55.697639    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3dc2dd63-b055-4a45-a14e-8854ab1243d7" containerName="cleanup"
Oct 30 19:00:55.697700 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:55.697650    4888 cpu_manager.go:414] "containerMap: removing container" podUID="0d5a7945-c6aa-453a-ac8c-1b23cddc1642" containerName="istio-proxy"
Oct 30 19:00:55.697700 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:55.697657    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0d5a7945-c6aa-453a-ac8c-1b23cddc1642" containerName="istio-proxy"
Oct 30 19:00:55.697700 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:55.697668    4888 cpu_manager.go:414] "containerMap: removing container" podUID="aa727ad6-0a18-4ce1-b48d-f2f01cd83677" containerName="istio-proxy"
Oct 30 19:00:55.697700 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:55.697675    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="aa727ad6-0a18-4ce1-b48d-f2f01cd83677" containerName="istio-proxy"
Oct 30 19:00:55.697700 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:55.697692    4888 cpu_manager.go:414] "containerMap: removing container" podUID="d33b7312-dc19-402a-9c2e-12e0b7eee906" containerName="istio-proxy"
Oct 30 19:00:55.697700 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:55.697700    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d33b7312-dc19-402a-9c2e-12e0b7eee906" containerName="istio-proxy"
Oct 30 19:00:55.698231 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:55.697718    4888 cpu_manager.go:414] "containerMap: removing container" podUID="fea8eecd-4a56-48e1-b68f-a3a1418dc395" containerName="cleanup"
Oct 30 19:00:55.698231 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:55.697726    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fea8eecd-4a56-48e1-b68f-a3a1418dc395" containerName="cleanup"
Oct 30 19:00:55.698231 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:55.697741    4888 cpu_manager.go:414] "containerMap: removing container" podUID="b40b1755-0472-48e2-be01-417743104be6" containerName="cleanup"
Oct 30 19:00:55.698231 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:55.697748    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b40b1755-0472-48e2-be01-417743104be6" containerName="cleanup"
Oct 30 19:00:55.698231 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:55.697762    4888 cpu_manager.go:414] "containerMap: removing container" podUID="a6c278cc-4cc3-4584-b83e-376f5af90380" containerName="istio-proxy"
Oct 30 19:00:55.698231 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:55.697769    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a6c278cc-4cc3-4584-b83e-376f5af90380" containerName="istio-proxy"
Oct 30 19:00:55.698231 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:55.697785    4888 cpu_manager.go:414] "containerMap: removing container" podUID="fea8eecd-4a56-48e1-b68f-a3a1418dc395" containerName="cleanup"
Oct 30 19:00:55.698231 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:55.697793    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fea8eecd-4a56-48e1-b68f-a3a1418dc395" containerName="cleanup"
Oct 30 19:00:55.698231 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:55.697807    4888 cpu_manager.go:414] "containerMap: removing container" podUID="3dc2dd63-b055-4a45-a14e-8854ab1243d7" containerName="cleanup"
Oct 30 19:00:55.698231 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:55.697814    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3dc2dd63-b055-4a45-a14e-8854ab1243d7" containerName="cleanup"
Oct 30 19:00:55.698231 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:55.697835    4888 cpu_manager.go:414] "containerMap: removing container" podUID="b40b1755-0472-48e2-be01-417743104be6" containerName="cleanup"
Oct 30 19:00:55.698231 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:55.697842    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b40b1755-0472-48e2-be01-417743104be6" containerName="cleanup"
Oct 30 19:00:55.698231 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:55.697863    4888 cpu_manager.go:414] "containerMap: removing container" podUID="fea8eecd-4a56-48e1-b68f-a3a1418dc395" containerName="cleanup"
Oct 30 19:00:55.698231 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:55.697871    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fea8eecd-4a56-48e1-b68f-a3a1418dc395" containerName="cleanup"
Oct 30 19:00:55.698231 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:55.697884    4888 cpu_manager.go:414] "containerMap: removing container" podUID="73999d2f-99a7-43f6-99e9-de05ee01f6ce" containerName="istio-proxy"
Oct 30 19:00:55.698231 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:55.697892    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="73999d2f-99a7-43f6-99e9-de05ee01f6ce" containerName="istio-proxy"
Oct 30 19:00:55.698231 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:55.697904    4888 cpu_manager.go:414] "containerMap: removing container" podUID="d6294bd8-9418-4c1a-99c6-cf52ba98edaa" containerName="istio-proxy"
Oct 30 19:00:55.698231 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:55.697911    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d6294bd8-9418-4c1a-99c6-cf52ba98edaa" containerName="istio-proxy"
Oct 30 19:00:55.698231 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:55.698058    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="d33b7312-dc19-402a-9c2e-12e0b7eee906" containerName="istio-proxy"
Oct 30 19:00:55.698231 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:55.698074    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="fea8eecd-4a56-48e1-b68f-a3a1418dc395" containerName="cleanup"
Oct 30 19:00:55.698231 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:55.698084    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="b40b1755-0472-48e2-be01-417743104be6" containerName="cleanup"
Oct 30 19:00:55.698231 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:55.698093    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="a6c278cc-4cc3-4584-b83e-376f5af90380" containerName="istio-proxy"
Oct 30 19:00:55.698231 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:55.698102    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="3dc2dd63-b055-4a45-a14e-8854ab1243d7" containerName="cleanup"
Oct 30 19:00:55.698231 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:55.698111    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="fea8eecd-4a56-48e1-b68f-a3a1418dc395" containerName="cleanup"
Oct 30 19:00:55.698231 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:55.698121    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="3dc2dd63-b055-4a45-a14e-8854ab1243d7" containerName="cleanup"
Oct 30 19:00:55.698231 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:55.698133    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="73999d2f-99a7-43f6-99e9-de05ee01f6ce" containerName="istio-proxy"
Oct 30 19:00:55.698231 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:55.698144    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="d6294bd8-9418-4c1a-99c6-cf52ba98edaa" containerName="istio-proxy"
Oct 30 19:00:55.698231 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:55.698154    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="2ba277ac-44b2-4d12-b873-6b584085f0d1" containerName="istio-proxy"
Oct 30 19:00:55.698231 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:55.698165    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="b40b1755-0472-48e2-be01-417743104be6" containerName="cleanup"
Oct 30 19:00:55.698231 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:55.698173    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="0d5a7945-c6aa-453a-ac8c-1b23cddc1642" containerName="istio-proxy"
Oct 30 19:00:55.698231 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:55.698182    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="aa727ad6-0a18-4ce1-b48d-f2f01cd83677" containerName="istio-proxy"
Oct 30 19:00:55.703669 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:55.703649    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-nvlbr"
Oct 30 19:00:55.799959 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:55.799918    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/03fac346-2d8c-4d15-8e8a-c7e6571ad9b1-workload-certs\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-nvlbr\" (UID: \"03fac346-2d8c-4d15-8e8a-c7e6571ad9b1\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-nvlbr"
Oct 30 19:00:55.800167 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:55.799967    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/03fac346-2d8c-4d15-8e8a-c7e6571ad9b1-workload-socket\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-nvlbr\" (UID: \"03fac346-2d8c-4d15-8e8a-c7e6571ad9b1\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-nvlbr"
Oct 30 19:00:55.800167 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:55.800037    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/03fac346-2d8c-4d15-8e8a-c7e6571ad9b1-istio-envoy\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-nvlbr\" (UID: \"03fac346-2d8c-4d15-8e8a-c7e6571ad9b1\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-nvlbr"
Oct 30 19:00:55.800167 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:55.800071    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/03fac346-2d8c-4d15-8e8a-c7e6571ad9b1-istio-data\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-nvlbr\" (UID: \"03fac346-2d8c-4d15-8e8a-c7e6571ad9b1\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-nvlbr"
Oct 30 19:00:55.800167 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:55.800088    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-msnpp\" (UniqueName: \"kubernetes.io/projected/03fac346-2d8c-4d15-8e8a-c7e6571ad9b1-kube-api-access-msnpp\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-nvlbr\" (UID: \"03fac346-2d8c-4d15-8e8a-c7e6571ad9b1\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-nvlbr"
Oct 30 19:00:55.800167 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:55.800111    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/03fac346-2d8c-4d15-8e8a-c7e6571ad9b1-istiod-ca-cert\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-nvlbr\" (UID: \"03fac346-2d8c-4d15-8e8a-c7e6571ad9b1\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-nvlbr"
Oct 30 19:00:55.800488 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:55.800336    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/03fac346-2d8c-4d15-8e8a-c7e6571ad9b1-credential-socket\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-nvlbr\" (UID: \"03fac346-2d8c-4d15-8e8a-c7e6571ad9b1\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-nvlbr"
Oct 30 19:00:55.800488 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:55.800481    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/03fac346-2d8c-4d15-8e8a-c7e6571ad9b1-istio-token\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-nvlbr\" (UID: \"03fac346-2d8c-4d15-8e8a-c7e6571ad9b1\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-nvlbr"
Oct 30 19:00:55.800593 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:55.800513    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/03fac346-2d8c-4d15-8e8a-c7e6571ad9b1-istio-podinfo\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-nvlbr\" (UID: \"03fac346-2d8c-4d15-8e8a-c7e6571ad9b1\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-nvlbr"
Oct 30 19:00:55.832317 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:55.823823    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-nvlbr"]
Oct 30 19:00:55.901804 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:55.901772    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/03fac346-2d8c-4d15-8e8a-c7e6571ad9b1-istio-envoy\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-nvlbr\" (UID: \"03fac346-2d8c-4d15-8e8a-c7e6571ad9b1\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-nvlbr"
Oct 30 19:00:55.902010 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:55.901820    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/03fac346-2d8c-4d15-8e8a-c7e6571ad9b1-istio-data\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-nvlbr\" (UID: \"03fac346-2d8c-4d15-8e8a-c7e6571ad9b1\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-nvlbr"
Oct 30 19:00:55.902010 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:55.901845    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-msnpp\" (UniqueName: \"kubernetes.io/projected/03fac346-2d8c-4d15-8e8a-c7e6571ad9b1-kube-api-access-msnpp\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-nvlbr\" (UID: \"03fac346-2d8c-4d15-8e8a-c7e6571ad9b1\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-nvlbr"
Oct 30 19:00:55.902010 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:55.901877    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/03fac346-2d8c-4d15-8e8a-c7e6571ad9b1-istiod-ca-cert\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-nvlbr\" (UID: \"03fac346-2d8c-4d15-8e8a-c7e6571ad9b1\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-nvlbr"
Oct 30 19:00:55.902010 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:55.901925    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/03fac346-2d8c-4d15-8e8a-c7e6571ad9b1-credential-socket\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-nvlbr\" (UID: \"03fac346-2d8c-4d15-8e8a-c7e6571ad9b1\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-nvlbr"
Oct 30 19:00:55.902238 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:55.902055    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/03fac346-2d8c-4d15-8e8a-c7e6571ad9b1-istio-token\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-nvlbr\" (UID: \"03fac346-2d8c-4d15-8e8a-c7e6571ad9b1\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-nvlbr"
Oct 30 19:00:55.902238 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:55.902100    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/03fac346-2d8c-4d15-8e8a-c7e6571ad9b1-istio-podinfo\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-nvlbr\" (UID: \"03fac346-2d8c-4d15-8e8a-c7e6571ad9b1\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-nvlbr"
Oct 30 19:00:55.902238 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:55.902194    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/03fac346-2d8c-4d15-8e8a-c7e6571ad9b1-workload-certs\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-nvlbr\" (UID: \"03fac346-2d8c-4d15-8e8a-c7e6571ad9b1\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-nvlbr"
Oct 30 19:00:55.902238 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:55.902229    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/03fac346-2d8c-4d15-8e8a-c7e6571ad9b1-workload-socket\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-nvlbr\" (UID: \"03fac346-2d8c-4d15-8e8a-c7e6571ad9b1\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-nvlbr"
Oct 30 19:00:55.902516 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:55.902274    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/03fac346-2d8c-4d15-8e8a-c7e6571ad9b1-istio-data\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-nvlbr\" (UID: \"03fac346-2d8c-4d15-8e8a-c7e6571ad9b1\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-nvlbr"
Oct 30 19:00:55.902725 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:55.902695    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/03fac346-2d8c-4d15-8e8a-c7e6571ad9b1-istiod-ca-cert\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-nvlbr\" (UID: \"03fac346-2d8c-4d15-8e8a-c7e6571ad9b1\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-nvlbr"
Oct 30 19:00:55.902878 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:55.902857    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/03fac346-2d8c-4d15-8e8a-c7e6571ad9b1-workload-socket\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-nvlbr\" (UID: \"03fac346-2d8c-4d15-8e8a-c7e6571ad9b1\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-nvlbr"
Oct 30 19:00:55.903086 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:55.903062    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/03fac346-2d8c-4d15-8e8a-c7e6571ad9b1-credential-socket\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-nvlbr\" (UID: \"03fac346-2d8c-4d15-8e8a-c7e6571ad9b1\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-nvlbr"
Oct 30 19:00:55.903449 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:55.903427    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/03fac346-2d8c-4d15-8e8a-c7e6571ad9b1-workload-certs\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-nvlbr\" (UID: \"03fac346-2d8c-4d15-8e8a-c7e6571ad9b1\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-nvlbr"
Oct 30 19:00:55.904305 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:55.904269    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/03fac346-2d8c-4d15-8e8a-c7e6571ad9b1-istio-envoy\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-nvlbr\" (UID: \"03fac346-2d8c-4d15-8e8a-c7e6571ad9b1\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-nvlbr"
Oct 30 19:00:55.904927 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:55.904883    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/03fac346-2d8c-4d15-8e8a-c7e6571ad9b1-istio-podinfo\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-nvlbr\" (UID: \"03fac346-2d8c-4d15-8e8a-c7e6571ad9b1\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-nvlbr"
Oct 30 19:00:55.986726 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:55.986653    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/03fac346-2d8c-4d15-8e8a-c7e6571ad9b1-istio-token\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-nvlbr\" (UID: \"03fac346-2d8c-4d15-8e8a-c7e6571ad9b1\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-nvlbr"
Oct 30 19:00:55.998400 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:55.998374    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-msnpp\" (UniqueName: \"kubernetes.io/projected/03fac346-2d8c-4d15-8e8a-c7e6571ad9b1-kube-api-access-msnpp\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-nvlbr\" (UID: \"03fac346-2d8c-4d15-8e8a-c7e6571ad9b1\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-nvlbr"
Oct 30 19:00:56.019884 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:56.019862    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-nvlbr"
Oct 30 19:00:56.455516 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:56.455487    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-nvlbr"]
Oct 30 19:00:56.456500 ip-10-0-134-46 kubenswrapper[4888]: W1030 19:00:56.456472    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03fac346_2d8c_4d15_8e8a_c7e6571ad9b1.slice/crio-79618eac4c3da85208dac8a4e402b282d5b699785073696945e642bc02bd2e7c WatchSource:0}: Error finding container 79618eac4c3da85208dac8a4e402b282d5b699785073696945e642bc02bd2e7c: Status 404 returned error can't find the container with id 79618eac4c3da85208dac8a4e402b282d5b699785073696945e642bc02bd2e7c
Oct 30 19:00:56.459121 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:56.459089    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:00:56.459226 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:56.459153    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:00:56.459226 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:56.459180    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:00:57.405211 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:57.405181    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-nvlbr" event={"ID":"03fac346-2d8c-4d15-8e8a-c7e6571ad9b1","Type":"ContainerStarted","Data":"3e473820a56a96f987ec7165ff8d8938fdc360032470a99db092f1ebdbf31e8c"}
Oct 30 19:00:57.405748 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:57.405243    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-nvlbr" event={"ID":"03fac346-2d8c-4d15-8e8a-c7e6571ad9b1","Type":"ContainerStarted","Data":"79618eac4c3da85208dac8a4e402b282d5b699785073696945e642bc02bd2e7c"}
Oct 30 19:00:58.020673 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:58.020638    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-nvlbr"
Oct 30 19:00:58.022408 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:58.022383    4888 patch_prober.go:29] interesting pod/e2e-mt-66baf45c-openshift-default-974f9d7b9-nvlbr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.111:15021/healthz/ready\": dial tcp 10.132.0.111:15021: connect: connection refused" start-of-body=
Oct 30 19:00:58.022543 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:58.022435    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-nvlbr" podUID="03fac346-2d8c-4d15-8e8a-c7e6571ad9b1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.111:15021/healthz/ready\": dial tcp 10.132.0.111:15021: connect: connection refused"
Oct 30 19:00:59.020774 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:59.020736    4888 patch_prober.go:29] interesting pod/e2e-mt-66baf45c-openshift-default-974f9d7b9-nvlbr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.111:15021/healthz/ready\": dial tcp 10.132.0.111:15021: connect: connection refused" start-of-body=
Oct 30 19:00:59.021198 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:00:59.020803    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-nvlbr" podUID="03fac346-2d8c-4d15-8e8a-c7e6571ad9b1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.111:15021/healthz/ready\": dial tcp 10.132.0.111:15021: connect: connection refused"
Oct 30 19:01:00.021101 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:00.021068    4888 patch_prober.go:29] interesting pod/e2e-mt-66baf45c-openshift-default-974f9d7b9-nvlbr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.111:15021/healthz/ready\": dial tcp 10.132.0.111:15021: connect: connection refused" start-of-body=
Oct 30 19:01:00.021565 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:00.021141    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-nvlbr" podUID="03fac346-2d8c-4d15-8e8a-c7e6571ad9b1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.111:15021/healthz/ready\": dial tcp 10.132.0.111:15021: connect: connection refused"
Oct 30 19:01:01.020819 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:01.020779    4888 patch_prober.go:29] interesting pod/e2e-mt-66baf45c-openshift-default-974f9d7b9-nvlbr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.111:15021/healthz/ready\": dial tcp 10.132.0.111:15021: connect: connection refused" start-of-body=
Oct 30 19:01:01.021127 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:01.020850    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-nvlbr" podUID="03fac346-2d8c-4d15-8e8a-c7e6571ad9b1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.111:15021/healthz/ready\": dial tcp 10.132.0.111:15021: connect: connection refused"
Oct 30 19:01:02.021154 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:02.021123    4888 patch_prober.go:29] interesting pod/e2e-mt-66baf45c-openshift-default-974f9d7b9-nvlbr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.111:15021/healthz/ready\": dial tcp 10.132.0.111:15021: connect: connection refused" start-of-body=
Oct 30 19:01:02.021661 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:02.021175    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-nvlbr" podUID="03fac346-2d8c-4d15-8e8a-c7e6571ad9b1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.111:15021/healthz/ready\": dial tcp 10.132.0.111:15021: connect: connection refused"
Oct 30 19:01:03.024716 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:03.024689    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-nvlbr"
Oct 30 19:01:03.025146 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:03.025133    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-nvlbr"
Oct 30 19:01:03.025727 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:03.025707    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-nvlbr"
Oct 30 19:01:03.160604 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:03.160540    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-nvlbr" podStartSLOduration=8.160520972 podStartE2EDuration="8.160520972s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:00:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:00:57.592890275 +0000 UTC m=+1621.840296301" watchObservedRunningTime="2026-10-30 19:01:03.160520972 +0000 UTC m=+1627.407926997"
Oct 30 19:01:05.620995 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:01:05.620956    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-jbsw6" containerName="extractor"
Oct 30 19:01:10.803749 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:10.803712    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-v2sdh"]
Oct 30 19:01:10.804584 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:10.804561    4888 cpu_manager.go:414] "containerMap: removing container" podUID="3dc2dd63-b055-4a45-a14e-8854ab1243d7" containerName="cleanup"
Oct 30 19:01:10.804584 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:10.804583    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3dc2dd63-b055-4a45-a14e-8854ab1243d7" containerName="cleanup"
Oct 30 19:01:10.804769 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:10.804750    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="fea8eecd-4a56-48e1-b68f-a3a1418dc395" containerName="cleanup"
Oct 30 19:01:10.804769 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:10.804764    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="3dc2dd63-b055-4a45-a14e-8854ab1243d7" containerName="cleanup"
Oct 30 19:01:10.804868 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:10.804787    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="b40b1755-0472-48e2-be01-417743104be6" containerName="cleanup"
Oct 30 19:01:10.809695 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:10.809673    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-v2sdh"
Oct 30 19:01:10.926745 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:10.926690    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/687d01bd-f2ca-4fe3-8766-c8030df17da4-workload-certs\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-v2sdh\" (UID: \"687d01bd-f2ca-4fe3-8766-c8030df17da4\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-v2sdh"
Oct 30 19:01:10.927426 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:10.927397    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/687d01bd-f2ca-4fe3-8766-c8030df17da4-credential-socket\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-v2sdh\" (UID: \"687d01bd-f2ca-4fe3-8766-c8030df17da4\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-v2sdh"
Oct 30 19:01:10.927569 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:10.927463    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/687d01bd-f2ca-4fe3-8766-c8030df17da4-istio-envoy\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-v2sdh\" (UID: \"687d01bd-f2ca-4fe3-8766-c8030df17da4\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-v2sdh"
Oct 30 19:01:10.927569 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:10.927496    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/687d01bd-f2ca-4fe3-8766-c8030df17da4-workload-socket\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-v2sdh\" (UID: \"687d01bd-f2ca-4fe3-8766-c8030df17da4\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-v2sdh"
Oct 30 19:01:10.927569 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:10.927519    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/687d01bd-f2ca-4fe3-8766-c8030df17da4-istio-podinfo\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-v2sdh\" (UID: \"687d01bd-f2ca-4fe3-8766-c8030df17da4\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-v2sdh"
Oct 30 19:01:10.927732 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:10.927686    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/687d01bd-f2ca-4fe3-8766-c8030df17da4-istio-token\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-v2sdh\" (UID: \"687d01bd-f2ca-4fe3-8766-c8030df17da4\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-v2sdh"
Oct 30 19:01:10.927732 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:10.927719    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/687d01bd-f2ca-4fe3-8766-c8030df17da4-istiod-ca-cert\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-v2sdh\" (UID: \"687d01bd-f2ca-4fe3-8766-c8030df17da4\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-v2sdh"
Oct 30 19:01:10.927911 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:10.927789    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/687d01bd-f2ca-4fe3-8766-c8030df17da4-istio-data\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-v2sdh\" (UID: \"687d01bd-f2ca-4fe3-8766-c8030df17da4\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-v2sdh"
Oct 30 19:01:10.927911 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:10.927832    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d5qcs\" (UniqueName: \"kubernetes.io/projected/687d01bd-f2ca-4fe3-8766-c8030df17da4-kube-api-access-d5qcs\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-v2sdh\" (UID: \"687d01bd-f2ca-4fe3-8766-c8030df17da4\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-v2sdh"
Oct 30 19:01:10.933831 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:10.933805    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-v2sdh"]
Oct 30 19:01:11.028791 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:11.028752    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/687d01bd-f2ca-4fe3-8766-c8030df17da4-istio-data\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-v2sdh\" (UID: \"687d01bd-f2ca-4fe3-8766-c8030df17da4\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-v2sdh"
Oct 30 19:01:11.028983 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:11.028820    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d5qcs\" (UniqueName: \"kubernetes.io/projected/687d01bd-f2ca-4fe3-8766-c8030df17da4-kube-api-access-d5qcs\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-v2sdh\" (UID: \"687d01bd-f2ca-4fe3-8766-c8030df17da4\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-v2sdh"
Oct 30 19:01:11.028983 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:11.028895    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/687d01bd-f2ca-4fe3-8766-c8030df17da4-workload-certs\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-v2sdh\" (UID: \"687d01bd-f2ca-4fe3-8766-c8030df17da4\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-v2sdh"
Oct 30 19:01:11.028983 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:11.028954    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/687d01bd-f2ca-4fe3-8766-c8030df17da4-credential-socket\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-v2sdh\" (UID: \"687d01bd-f2ca-4fe3-8766-c8030df17da4\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-v2sdh"
Oct 30 19:01:11.029171 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:11.029008    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/687d01bd-f2ca-4fe3-8766-c8030df17da4-istio-envoy\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-v2sdh\" (UID: \"687d01bd-f2ca-4fe3-8766-c8030df17da4\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-v2sdh"
Oct 30 19:01:11.029171 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:11.029039    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/687d01bd-f2ca-4fe3-8766-c8030df17da4-workload-socket\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-v2sdh\" (UID: \"687d01bd-f2ca-4fe3-8766-c8030df17da4\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-v2sdh"
Oct 30 19:01:11.029171 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:11.029063    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/687d01bd-f2ca-4fe3-8766-c8030df17da4-istio-podinfo\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-v2sdh\" (UID: \"687d01bd-f2ca-4fe3-8766-c8030df17da4\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-v2sdh"
Oct 30 19:01:11.029355 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:11.029188    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/687d01bd-f2ca-4fe3-8766-c8030df17da4-istio-data\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-v2sdh\" (UID: \"687d01bd-f2ca-4fe3-8766-c8030df17da4\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-v2sdh"
Oct 30 19:01:11.029526 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:11.029507    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/687d01bd-f2ca-4fe3-8766-c8030df17da4-workload-socket\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-v2sdh\" (UID: \"687d01bd-f2ca-4fe3-8766-c8030df17da4\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-v2sdh"
Oct 30 19:01:11.029661 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:11.029577    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/687d01bd-f2ca-4fe3-8766-c8030df17da4-workload-certs\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-v2sdh\" (UID: \"687d01bd-f2ca-4fe3-8766-c8030df17da4\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-v2sdh"
Oct 30 19:01:11.029995 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:11.029714    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/687d01bd-f2ca-4fe3-8766-c8030df17da4-istio-token\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-v2sdh\" (UID: \"687d01bd-f2ca-4fe3-8766-c8030df17da4\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-v2sdh"
Oct 30 19:01:11.030160 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:11.030139    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/687d01bd-f2ca-4fe3-8766-c8030df17da4-istiod-ca-cert\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-v2sdh\" (UID: \"687d01bd-f2ca-4fe3-8766-c8030df17da4\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-v2sdh"
Oct 30 19:01:11.030762 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:11.030741    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/687d01bd-f2ca-4fe3-8766-c8030df17da4-istiod-ca-cert\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-v2sdh\" (UID: \"687d01bd-f2ca-4fe3-8766-c8030df17da4\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-v2sdh"
Oct 30 19:01:11.030856 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:11.030785    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/687d01bd-f2ca-4fe3-8766-c8030df17da4-credential-socket\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-v2sdh\" (UID: \"687d01bd-f2ca-4fe3-8766-c8030df17da4\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-v2sdh"
Oct 30 19:01:11.032166 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:11.032143    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/687d01bd-f2ca-4fe3-8766-c8030df17da4-istio-envoy\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-v2sdh\" (UID: \"687d01bd-f2ca-4fe3-8766-c8030df17da4\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-v2sdh"
Oct 30 19:01:11.032752 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:11.032730    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/687d01bd-f2ca-4fe3-8766-c8030df17da4-istio-podinfo\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-v2sdh\" (UID: \"687d01bd-f2ca-4fe3-8766-c8030df17da4\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-v2sdh"
Oct 30 19:01:11.094156 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:11.094129    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d5qcs\" (UniqueName: \"kubernetes.io/projected/687d01bd-f2ca-4fe3-8766-c8030df17da4-kube-api-access-d5qcs\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-v2sdh\" (UID: \"687d01bd-f2ca-4fe3-8766-c8030df17da4\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-v2sdh"
Oct 30 19:01:11.094381 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:11.094129    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/687d01bd-f2ca-4fe3-8766-c8030df17da4-istio-token\") pod \"e2e-mt-66baf45c-openshift-default-974f9d7b9-v2sdh\" (UID: \"687d01bd-f2ca-4fe3-8766-c8030df17da4\") " pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-v2sdh"
Oct 30 19:01:11.125154 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:11.125127    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-v2sdh"
Oct 30 19:01:11.563744 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:11.563703    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-v2sdh"]
Oct 30 19:01:11.564192 ip-10-0-134-46 kubenswrapper[4888]: W1030 19:01:11.564161    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod687d01bd_f2ca_4fe3_8766_c8030df17da4.slice/crio-86fb8239296aea1f9b0163c5f439c79b2decfb4ffe27abb4d0b883d3bdf99061 WatchSource:0}: Error finding container 86fb8239296aea1f9b0163c5f439c79b2decfb4ffe27abb4d0b883d3bdf99061: Status 404 returned error can't find the container with id 86fb8239296aea1f9b0163c5f439c79b2decfb4ffe27abb4d0b883d3bdf99061
Oct 30 19:01:11.566639 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:11.566606    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:01:11.566735 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:11.566673    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:01:11.566735 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:11.566703    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:01:12.523437 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:12.521404    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-v2sdh" event={"ID":"687d01bd-f2ca-4fe3-8766-c8030df17da4","Type":"ContainerStarted","Data":"d345dd999a7f21095883da557949bd436a24bad41ffe6c2690ae14f8da2219cf"}
Oct 30 19:01:12.523437 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:12.521470    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-v2sdh" event={"ID":"687d01bd-f2ca-4fe3-8766-c8030df17da4","Type":"ContainerStarted","Data":"86fb8239296aea1f9b0163c5f439c79b2decfb4ffe27abb4d0b883d3bdf99061"}
Oct 30 19:01:12.807051 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:12.806925    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-v2sdh" podStartSLOduration=2.806902334 podStartE2EDuration="2.806902334s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:01:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:01:12.714654913 +0000 UTC m=+1636.962060939" watchObservedRunningTime="2026-10-30 19:01:12.806902334 +0000 UTC m=+1637.054308360"
Oct 30 19:01:12.812357 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:12.810120    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-rjmdz"]
Oct 30 19:01:13.125855 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:13.125812    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-v2sdh"
Oct 30 19:01:13.128064 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:13.128027    4888 patch_prober.go:29] interesting pod/e2e-mt-66baf45c-openshift-default-974f9d7b9-v2sdh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.112:15021/healthz/ready\": dial tcp 10.132.0.112:15021: connect: connection refused" start-of-body=
Oct 30 19:01:13.128224 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:13.128085    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-v2sdh" podUID="687d01bd-f2ca-4fe3-8766-c8030df17da4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.112:15021/healthz/ready\": dial tcp 10.132.0.112:15021: connect: connection refused"
Oct 30 19:01:14.125816 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:14.125778    4888 patch_prober.go:29] interesting pod/e2e-mt-66baf45c-openshift-default-974f9d7b9-v2sdh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.112:15021/healthz/ready\": dial tcp 10.132.0.112:15021: connect: connection refused" start-of-body=
Oct 30 19:01:14.126313 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:14.125838    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-v2sdh" podUID="687d01bd-f2ca-4fe3-8766-c8030df17da4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.112:15021/healthz/ready\": dial tcp 10.132.0.112:15021: connect: connection refused"
Oct 30 19:01:15.126359 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:15.126326    4888 patch_prober.go:29] interesting pod/e2e-mt-66baf45c-openshift-default-974f9d7b9-v2sdh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.112:15021/healthz/ready\": dial tcp 10.132.0.112:15021: connect: connection refused" start-of-body=
Oct 30 19:01:15.126783 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:15.126388    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-v2sdh" podUID="687d01bd-f2ca-4fe3-8766-c8030df17da4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.112:15021/healthz/ready\": dial tcp 10.132.0.112:15021: connect: connection refused"
Oct 30 19:01:16.126174 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:16.126138    4888 patch_prober.go:29] interesting pod/e2e-mt-66baf45c-openshift-default-974f9d7b9-v2sdh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.112:15021/healthz/ready\": dial tcp 10.132.0.112:15021: connect: connection refused" start-of-body=
Oct 30 19:01:16.126392 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:16.126198    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-v2sdh" podUID="687d01bd-f2ca-4fe3-8766-c8030df17da4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.112:15021/healthz/ready\": dial tcp 10.132.0.112:15021: connect: connection refused"
Oct 30 19:01:17.125932 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:17.125897    4888 patch_prober.go:29] interesting pod/e2e-mt-66baf45c-openshift-default-974f9d7b9-v2sdh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.112:15021/healthz/ready\": dial tcp 10.132.0.112:15021: connect: connection refused" start-of-body=
Oct 30 19:01:17.126111 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:17.125954    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-v2sdh" podUID="687d01bd-f2ca-4fe3-8766-c8030df17da4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.112:15021/healthz/ready\": dial tcp 10.132.0.112:15021: connect: connection refused"
Oct 30 19:01:18.203242 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:18.203201    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-v2sdh"
Oct 30 19:01:18.203766 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:18.203723    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-v2sdh"
Oct 30 19:01:18.204636 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:18.204615    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-v2sdh"
Oct 30 19:01:24.208789 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:24.208752    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-6trhv"]
Oct 30 19:01:24.213926 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:24.213902    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-6trhv"
Oct 30 19:01:24.269473 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:24.269453    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-4ebcd2-openshift-default-dockercfg-2smq2\""
Oct 30 19:01:24.342712 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:24.342677    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/509acd5d-92ed-4366-881e-3181be309e13-istio-token\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-6trhv\" (UID: \"509acd5d-92ed-4366-881e-3181be309e13\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-6trhv"
Oct 30 19:01:24.342906 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:24.342730    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/509acd5d-92ed-4366-881e-3181be309e13-istiod-ca-cert\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-6trhv\" (UID: \"509acd5d-92ed-4366-881e-3181be309e13\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-6trhv"
Oct 30 19:01:24.342906 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:24.342823    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/509acd5d-92ed-4366-881e-3181be309e13-workload-socket\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-6trhv\" (UID: \"509acd5d-92ed-4366-881e-3181be309e13\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-6trhv"
Oct 30 19:01:24.342906 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:24.342847    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/509acd5d-92ed-4366-881e-3181be309e13-istio-data\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-6trhv\" (UID: \"509acd5d-92ed-4366-881e-3181be309e13\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-6trhv"
Oct 30 19:01:24.343102 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:24.342912    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/509acd5d-92ed-4366-881e-3181be309e13-istio-envoy\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-6trhv\" (UID: \"509acd5d-92ed-4366-881e-3181be309e13\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-6trhv"
Oct 30 19:01:24.343102 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:24.342947    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/509acd5d-92ed-4366-881e-3181be309e13-workload-certs\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-6trhv\" (UID: \"509acd5d-92ed-4366-881e-3181be309e13\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-6trhv"
Oct 30 19:01:24.343102 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:24.343042    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/509acd5d-92ed-4366-881e-3181be309e13-credential-socket\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-6trhv\" (UID: \"509acd5d-92ed-4366-881e-3181be309e13\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-6trhv"
Oct 30 19:01:24.343102 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:24.343086    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/509acd5d-92ed-4366-881e-3181be309e13-istio-podinfo\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-6trhv\" (UID: \"509acd5d-92ed-4366-881e-3181be309e13\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-6trhv"
Oct 30 19:01:24.343353 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:24.343158    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c9c89\" (UniqueName: \"kubernetes.io/projected/509acd5d-92ed-4366-881e-3181be309e13-kube-api-access-c9c89\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-6trhv\" (UID: \"509acd5d-92ed-4366-881e-3181be309e13\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-6trhv"
Oct 30 19:01:24.351720 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:24.351695    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-6trhv"]
Oct 30 19:01:24.444346 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:24.444315    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/509acd5d-92ed-4366-881e-3181be309e13-istio-envoy\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-6trhv\" (UID: \"509acd5d-92ed-4366-881e-3181be309e13\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-6trhv"
Oct 30 19:01:24.444536 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:24.444355    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/509acd5d-92ed-4366-881e-3181be309e13-workload-certs\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-6trhv\" (UID: \"509acd5d-92ed-4366-881e-3181be309e13\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-6trhv"
Oct 30 19:01:24.444536 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:24.444411    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/509acd5d-92ed-4366-881e-3181be309e13-credential-socket\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-6trhv\" (UID: \"509acd5d-92ed-4366-881e-3181be309e13\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-6trhv"
Oct 30 19:01:24.444536 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:24.444438    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/509acd5d-92ed-4366-881e-3181be309e13-istio-podinfo\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-6trhv\" (UID: \"509acd5d-92ed-4366-881e-3181be309e13\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-6trhv"
Oct 30 19:01:24.444536 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:24.444476    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c9c89\" (UniqueName: \"kubernetes.io/projected/509acd5d-92ed-4366-881e-3181be309e13-kube-api-access-c9c89\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-6trhv\" (UID: \"509acd5d-92ed-4366-881e-3181be309e13\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-6trhv"
Oct 30 19:01:24.444763 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:24.444544    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/509acd5d-92ed-4366-881e-3181be309e13-istio-token\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-6trhv\" (UID: \"509acd5d-92ed-4366-881e-3181be309e13\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-6trhv"
Oct 30 19:01:24.444763 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:24.444572    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/509acd5d-92ed-4366-881e-3181be309e13-istiod-ca-cert\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-6trhv\" (UID: \"509acd5d-92ed-4366-881e-3181be309e13\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-6trhv"
Oct 30 19:01:24.444763 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:24.444654    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/509acd5d-92ed-4366-881e-3181be309e13-workload-socket\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-6trhv\" (UID: \"509acd5d-92ed-4366-881e-3181be309e13\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-6trhv"
Oct 30 19:01:24.444763 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:24.444676    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/509acd5d-92ed-4366-881e-3181be309e13-istio-data\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-6trhv\" (UID: \"509acd5d-92ed-4366-881e-3181be309e13\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-6trhv"
Oct 30 19:01:24.444988 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:24.444814    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/509acd5d-92ed-4366-881e-3181be309e13-credential-socket\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-6trhv\" (UID: \"509acd5d-92ed-4366-881e-3181be309e13\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-6trhv"
Oct 30 19:01:24.444988 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:24.444855    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/509acd5d-92ed-4366-881e-3181be309e13-workload-certs\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-6trhv\" (UID: \"509acd5d-92ed-4366-881e-3181be309e13\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-6trhv"
Oct 30 19:01:24.445088 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:24.445027    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/509acd5d-92ed-4366-881e-3181be309e13-istio-data\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-6trhv\" (UID: \"509acd5d-92ed-4366-881e-3181be309e13\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-6trhv"
Oct 30 19:01:24.445144 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:24.445101    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/509acd5d-92ed-4366-881e-3181be309e13-workload-socket\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-6trhv\" (UID: \"509acd5d-92ed-4366-881e-3181be309e13\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-6trhv"
Oct 30 19:01:24.445408 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:24.445383    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/509acd5d-92ed-4366-881e-3181be309e13-istiod-ca-cert\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-6trhv\" (UID: \"509acd5d-92ed-4366-881e-3181be309e13\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-6trhv"
Oct 30 19:01:24.447036 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:24.447011    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/509acd5d-92ed-4366-881e-3181be309e13-istio-envoy\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-6trhv\" (UID: \"509acd5d-92ed-4366-881e-3181be309e13\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-6trhv"
Oct 30 19:01:24.447250 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:24.447179    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/509acd5d-92ed-4366-881e-3181be309e13-istio-podinfo\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-6trhv\" (UID: \"509acd5d-92ed-4366-881e-3181be309e13\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-6trhv"
Oct 30 19:01:24.512602 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:24.512220    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c9c89\" (UniqueName: \"kubernetes.io/projected/509acd5d-92ed-4366-881e-3181be309e13-kube-api-access-c9c89\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-6trhv\" (UID: \"509acd5d-92ed-4366-881e-3181be309e13\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-6trhv"
Oct 30 19:01:24.512953 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:24.512911    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/509acd5d-92ed-4366-881e-3181be309e13-istio-token\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-6trhv\" (UID: \"509acd5d-92ed-4366-881e-3181be309e13\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-6trhv"
Oct 30 19:01:24.527811 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:24.527789    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-6trhv"
Oct 30 19:01:24.974771 ip-10-0-134-46 kubenswrapper[4888]: W1030 19:01:24.974745    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod509acd5d_92ed_4366_881e_3181be309e13.slice/crio-73bd163a437a92086ffc58a21804ed6dd6263817a64ab10d2a42473404e3af7f WatchSource:0}: Error finding container 73bd163a437a92086ffc58a21804ed6dd6263817a64ab10d2a42473404e3af7f: Status 404 returned error can't find the container with id 73bd163a437a92086ffc58a21804ed6dd6263817a64ab10d2a42473404e3af7f
Oct 30 19:01:24.977153 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:24.977126    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-6trhv"]
Oct 30 19:01:24.977459 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:24.977431    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:01:24.977508 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:24.977497    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:01:24.977542 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:24.977532    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:01:25.652362 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:25.652325    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-6trhv" event={"ID":"509acd5d-92ed-4366-881e-3181be309e13","Type":"ContainerStarted","Data":"9f865bc4a0bfbe50ae203eda8a1a4f3dd3ea4695ea82c1003076c6676e818c84"}
Oct 30 19:01:25.652861 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:25.652426    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-6trhv" event={"ID":"509acd5d-92ed-4366-881e-3181be309e13","Type":"ContainerStarted","Data":"73bd163a437a92086ffc58a21804ed6dd6263817a64ab10d2a42473404e3af7f"}
Oct 30 19:01:25.791309 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:25.791218    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-6trhv" podStartSLOduration=1.791199038 podStartE2EDuration="1.791199038s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:01:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:01:25.790627694 +0000 UTC m=+1650.038033722" watchObservedRunningTime="2026-10-30 19:01:25.791199038 +0000 UTC m=+1650.038605065"
Oct 30 19:01:26.528296 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:26.528253    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-6trhv"
Oct 30 19:01:26.529545 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:26.529524    4888 patch_prober.go:29] interesting pod/e2e-api-b-4ebcd2-openshift-default-59d766c94c-6trhv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.113:15021/healthz/ready\": dial tcp 10.132.0.113:15021: connect: connection refused" start-of-body=
Oct 30 19:01:26.529609 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:26.529573    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-6trhv" podUID="509acd5d-92ed-4366-881e-3181be309e13" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.113:15021/healthz/ready\": dial tcp 10.132.0.113:15021: connect: connection refused"
Oct 30 19:01:27.528929 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:27.528891    4888 patch_prober.go:29] interesting pod/e2e-api-b-4ebcd2-openshift-default-59d766c94c-6trhv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.113:15021/healthz/ready\": dial tcp 10.132.0.113:15021: connect: connection refused" start-of-body=
Oct 30 19:01:27.529358 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:27.528947    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-6trhv" podUID="509acd5d-92ed-4366-881e-3181be309e13" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.113:15021/healthz/ready\": dial tcp 10.132.0.113:15021: connect: connection refused"
Oct 30 19:01:28.528489 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:28.528457    4888 patch_prober.go:29] interesting pod/e2e-api-b-4ebcd2-openshift-default-59d766c94c-6trhv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.113:15021/healthz/ready\": dial tcp 10.132.0.113:15021: connect: connection refused" start-of-body=
Oct 30 19:01:28.528675 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:28.528516    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-6trhv" podUID="509acd5d-92ed-4366-881e-3181be309e13" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.113:15021/healthz/ready\": dial tcp 10.132.0.113:15021: connect: connection refused"
Oct 30 19:01:29.528626 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:29.528586    4888 patch_prober.go:29] interesting pod/e2e-api-b-4ebcd2-openshift-default-59d766c94c-6trhv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.113:15021/healthz/ready\": dial tcp 10.132.0.113:15021: connect: connection refused" start-of-body=
Oct 30 19:01:29.529025 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:29.528665    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-6trhv" podUID="509acd5d-92ed-4366-881e-3181be309e13" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.113:15021/healthz/ready\": dial tcp 10.132.0.113:15021: connect: connection refused"
Oct 30 19:01:30.541354 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:30.535869    4888 patch_prober.go:29] interesting pod/e2e-api-b-4ebcd2-openshift-default-59d766c94c-6trhv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.113:15021/healthz/ready\": dial tcp 10.132.0.113:15021: connect: connection refused" start-of-body=
Oct 30 19:01:30.541354 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:30.535930    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-6trhv" podUID="509acd5d-92ed-4366-881e-3181be309e13" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.113:15021/healthz/ready\": dial tcp 10.132.0.113:15021: connect: connection refused"
Oct 30 19:01:30.599985 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:30.599947    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-a-82305a-openshift-default-7cdd8f9877-sf7w8"]
Oct 30 19:01:30.614637 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:30.614593    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-82305a-openshift-default-7cdd8f9877-sf7w8"
Oct 30 19:01:30.659835 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:30.659741    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f9c90752-5508-445c-823e-3d9ea37f54b2-istio-token\") pod \"e2e-ipp-a-82305a-openshift-default-7cdd8f9877-sf7w8\" (UID: \"f9c90752-5508-445c-823e-3d9ea37f54b2\") " pod="openshift-ingress/e2e-ipp-a-82305a-openshift-default-7cdd8f9877-sf7w8"
Oct 30 19:01:30.660017 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:30.659873    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f9c90752-5508-445c-823e-3d9ea37f54b2-istiod-ca-cert\") pod \"e2e-ipp-a-82305a-openshift-default-7cdd8f9877-sf7w8\" (UID: \"f9c90752-5508-445c-823e-3d9ea37f54b2\") " pod="openshift-ingress/e2e-ipp-a-82305a-openshift-default-7cdd8f9877-sf7w8"
Oct 30 19:01:30.660017 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:30.659933    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f9c90752-5508-445c-823e-3d9ea37f54b2-istio-data\") pod \"e2e-ipp-a-82305a-openshift-default-7cdd8f9877-sf7w8\" (UID: \"f9c90752-5508-445c-823e-3d9ea37f54b2\") " pod="openshift-ingress/e2e-ipp-a-82305a-openshift-default-7cdd8f9877-sf7w8"
Oct 30 19:01:30.660017 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:30.659998    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f9c90752-5508-445c-823e-3d9ea37f54b2-istio-envoy\") pod \"e2e-ipp-a-82305a-openshift-default-7cdd8f9877-sf7w8\" (UID: \"f9c90752-5508-445c-823e-3d9ea37f54b2\") " pod="openshift-ingress/e2e-ipp-a-82305a-openshift-default-7cdd8f9877-sf7w8"
Oct 30 19:01:30.660204 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:30.660076    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9c90752-5508-445c-823e-3d9ea37f54b2-credential-socket\") pod \"e2e-ipp-a-82305a-openshift-default-7cdd8f9877-sf7w8\" (UID: \"f9c90752-5508-445c-823e-3d9ea37f54b2\") " pod="openshift-ingress/e2e-ipp-a-82305a-openshift-default-7cdd8f9877-sf7w8"
Oct 30 19:01:30.660204 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:30.660101    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f9c90752-5508-445c-823e-3d9ea37f54b2-istio-podinfo\") pod \"e2e-ipp-a-82305a-openshift-default-7cdd8f9877-sf7w8\" (UID: \"f9c90752-5508-445c-823e-3d9ea37f54b2\") " pod="openshift-ingress/e2e-ipp-a-82305a-openshift-default-7cdd8f9877-sf7w8"
Oct 30 19:01:30.660204 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:30.660138    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9c90752-5508-445c-823e-3d9ea37f54b2-workload-socket\") pod \"e2e-ipp-a-82305a-openshift-default-7cdd8f9877-sf7w8\" (UID: \"f9c90752-5508-445c-823e-3d9ea37f54b2\") " pod="openshift-ingress/e2e-ipp-a-82305a-openshift-default-7cdd8f9877-sf7w8"
Oct 30 19:01:30.660397 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:30.660255    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xtjhn\" (UniqueName: \"kubernetes.io/projected/f9c90752-5508-445c-823e-3d9ea37f54b2-kube-api-access-xtjhn\") pod \"e2e-ipp-a-82305a-openshift-default-7cdd8f9877-sf7w8\" (UID: \"f9c90752-5508-445c-823e-3d9ea37f54b2\") " pod="openshift-ingress/e2e-ipp-a-82305a-openshift-default-7cdd8f9877-sf7w8"
Oct 30 19:01:30.660397 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:30.660329    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f9c90752-5508-445c-823e-3d9ea37f54b2-workload-certs\") pod \"e2e-ipp-a-82305a-openshift-default-7cdd8f9877-sf7w8\" (UID: \"f9c90752-5508-445c-823e-3d9ea37f54b2\") " pod="openshift-ingress/e2e-ipp-a-82305a-openshift-default-7cdd8f9877-sf7w8"
Oct 30 19:01:30.720195 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:30.720164    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-82305a-openshift-default-7cdd8f9877-sf7w8"]
Oct 30 19:01:30.741257 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:30.741232    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-a-82305a-openshift-default-dockercfg-8xhg6\""
Oct 30 19:01:30.818370 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:30.802113    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f9c90752-5508-445c-823e-3d9ea37f54b2-istiod-ca-cert\") pod \"e2e-ipp-a-82305a-openshift-default-7cdd8f9877-sf7w8\" (UID: \"f9c90752-5508-445c-823e-3d9ea37f54b2\") " pod="openshift-ingress/e2e-ipp-a-82305a-openshift-default-7cdd8f9877-sf7w8"
Oct 30 19:01:30.818690 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:30.818656    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f9c90752-5508-445c-823e-3d9ea37f54b2-istio-data\") pod \"e2e-ipp-a-82305a-openshift-default-7cdd8f9877-sf7w8\" (UID: \"f9c90752-5508-445c-823e-3d9ea37f54b2\") " pod="openshift-ingress/e2e-ipp-a-82305a-openshift-default-7cdd8f9877-sf7w8"
Oct 30 19:01:30.818916 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:30.818897    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f9c90752-5508-445c-823e-3d9ea37f54b2-istio-envoy\") pod \"e2e-ipp-a-82305a-openshift-default-7cdd8f9877-sf7w8\" (UID: \"f9c90752-5508-445c-823e-3d9ea37f54b2\") " pod="openshift-ingress/e2e-ipp-a-82305a-openshift-default-7cdd8f9877-sf7w8"
Oct 30 19:01:30.819108 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:30.819091    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9c90752-5508-445c-823e-3d9ea37f54b2-credential-socket\") pod \"e2e-ipp-a-82305a-openshift-default-7cdd8f9877-sf7w8\" (UID: \"f9c90752-5508-445c-823e-3d9ea37f54b2\") " pod="openshift-ingress/e2e-ipp-a-82305a-openshift-default-7cdd8f9877-sf7w8"
Oct 30 19:01:30.819227 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:30.819213    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f9c90752-5508-445c-823e-3d9ea37f54b2-istio-podinfo\") pod \"e2e-ipp-a-82305a-openshift-default-7cdd8f9877-sf7w8\" (UID: \"f9c90752-5508-445c-823e-3d9ea37f54b2\") " pod="openshift-ingress/e2e-ipp-a-82305a-openshift-default-7cdd8f9877-sf7w8"
Oct 30 19:01:30.828098 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:30.828069    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9c90752-5508-445c-823e-3d9ea37f54b2-workload-socket\") pod \"e2e-ipp-a-82305a-openshift-default-7cdd8f9877-sf7w8\" (UID: \"f9c90752-5508-445c-823e-3d9ea37f54b2\") " pod="openshift-ingress/e2e-ipp-a-82305a-openshift-default-7cdd8f9877-sf7w8"
Oct 30 19:01:30.828260 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:30.828225    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xtjhn\" (UniqueName: \"kubernetes.io/projected/f9c90752-5508-445c-823e-3d9ea37f54b2-kube-api-access-xtjhn\") pod \"e2e-ipp-a-82305a-openshift-default-7cdd8f9877-sf7w8\" (UID: \"f9c90752-5508-445c-823e-3d9ea37f54b2\") " pod="openshift-ingress/e2e-ipp-a-82305a-openshift-default-7cdd8f9877-sf7w8"
Oct 30 19:01:30.828349 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:30.828272    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f9c90752-5508-445c-823e-3d9ea37f54b2-workload-certs\") pod \"e2e-ipp-a-82305a-openshift-default-7cdd8f9877-sf7w8\" (UID: \"f9c90752-5508-445c-823e-3d9ea37f54b2\") " pod="openshift-ingress/e2e-ipp-a-82305a-openshift-default-7cdd8f9877-sf7w8"
Oct 30 19:01:30.828400 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:30.823470    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f9c90752-5508-445c-823e-3d9ea37f54b2-istiod-ca-cert\") pod \"e2e-ipp-a-82305a-openshift-default-7cdd8f9877-sf7w8\" (UID: \"f9c90752-5508-445c-823e-3d9ea37f54b2\") " pod="openshift-ingress/e2e-ipp-a-82305a-openshift-default-7cdd8f9877-sf7w8"
Oct 30 19:01:30.828453 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:30.822729    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f9c90752-5508-445c-823e-3d9ea37f54b2-istio-data\") pod \"e2e-ipp-a-82305a-openshift-default-7cdd8f9877-sf7w8\" (UID: \"f9c90752-5508-445c-823e-3d9ea37f54b2\") " pod="openshift-ingress/e2e-ipp-a-82305a-openshift-default-7cdd8f9877-sf7w8"
Oct 30 19:01:30.828453 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:30.828420    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f9c90752-5508-445c-823e-3d9ea37f54b2-istio-token\") pod \"e2e-ipp-a-82305a-openshift-default-7cdd8f9877-sf7w8\" (UID: \"f9c90752-5508-445c-823e-3d9ea37f54b2\") " pod="openshift-ingress/e2e-ipp-a-82305a-openshift-default-7cdd8f9877-sf7w8"
Oct 30 19:01:30.828713 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:30.828688    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9c90752-5508-445c-823e-3d9ea37f54b2-workload-socket\") pod \"e2e-ipp-a-82305a-openshift-default-7cdd8f9877-sf7w8\" (UID: \"f9c90752-5508-445c-823e-3d9ea37f54b2\") " pod="openshift-ingress/e2e-ipp-a-82305a-openshift-default-7cdd8f9877-sf7w8"
Oct 30 19:01:30.828796 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:30.824128    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9c90752-5508-445c-823e-3d9ea37f54b2-credential-socket\") pod \"e2e-ipp-a-82305a-openshift-default-7cdd8f9877-sf7w8\" (UID: \"f9c90752-5508-445c-823e-3d9ea37f54b2\") " pod="openshift-ingress/e2e-ipp-a-82305a-openshift-default-7cdd8f9877-sf7w8"
Oct 30 19:01:30.828962 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:30.827218    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f9c90752-5508-445c-823e-3d9ea37f54b2-istio-envoy\") pod \"e2e-ipp-a-82305a-openshift-default-7cdd8f9877-sf7w8\" (UID: \"f9c90752-5508-445c-823e-3d9ea37f54b2\") " pod="openshift-ingress/e2e-ipp-a-82305a-openshift-default-7cdd8f9877-sf7w8"
Oct 30 19:01:30.829100 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:30.829075    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f9c90752-5508-445c-823e-3d9ea37f54b2-workload-certs\") pod \"e2e-ipp-a-82305a-openshift-default-7cdd8f9877-sf7w8\" (UID: \"f9c90752-5508-445c-823e-3d9ea37f54b2\") " pod="openshift-ingress/e2e-ipp-a-82305a-openshift-default-7cdd8f9877-sf7w8"
Oct 30 19:01:30.830744 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:30.829670    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f9c90752-5508-445c-823e-3d9ea37f54b2-istio-podinfo\") pod \"e2e-ipp-a-82305a-openshift-default-7cdd8f9877-sf7w8\" (UID: \"f9c90752-5508-445c-823e-3d9ea37f54b2\") " pod="openshift-ingress/e2e-ipp-a-82305a-openshift-default-7cdd8f9877-sf7w8"
Oct 30 19:01:30.907642 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:30.907609    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xtjhn\" (UniqueName: \"kubernetes.io/projected/f9c90752-5508-445c-823e-3d9ea37f54b2-kube-api-access-xtjhn\") pod \"e2e-ipp-a-82305a-openshift-default-7cdd8f9877-sf7w8\" (UID: \"f9c90752-5508-445c-823e-3d9ea37f54b2\") " pod="openshift-ingress/e2e-ipp-a-82305a-openshift-default-7cdd8f9877-sf7w8"
Oct 30 19:01:30.908547 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:30.908503    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f9c90752-5508-445c-823e-3d9ea37f54b2-istio-token\") pod \"e2e-ipp-a-82305a-openshift-default-7cdd8f9877-sf7w8\" (UID: \"f9c90752-5508-445c-823e-3d9ea37f54b2\") " pod="openshift-ingress/e2e-ipp-a-82305a-openshift-default-7cdd8f9877-sf7w8"
Oct 30 19:01:30.932997 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:30.932374    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-82305a-openshift-default-7cdd8f9877-sf7w8"
Oct 30 19:01:31.389508 ip-10-0-134-46 kubenswrapper[4888]: W1030 19:01:31.389471    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf9c90752_5508_445c_823e_3d9ea37f54b2.slice/crio-1055798aa2d74b89965a965515468e5d5af8c9f481bcb6c4bd8acc1a13166aa7 WatchSource:0}: Error finding container 1055798aa2d74b89965a965515468e5d5af8c9f481bcb6c4bd8acc1a13166aa7: Status 404 returned error can't find the container with id 1055798aa2d74b89965a965515468e5d5af8c9f481bcb6c4bd8acc1a13166aa7
Oct 30 19:01:31.391379 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:31.391355    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-82305a-openshift-default-7cdd8f9877-sf7w8"]
Oct 30 19:01:31.392377 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:31.392341    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:01:31.392444 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:31.392417    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:01:31.392498 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:31.392448    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:01:31.534804 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:31.534770    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-6trhv"
Oct 30 19:01:31.536375 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:31.536352    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-6trhv"
Oct 30 19:01:31.537358 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:31.537335    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-6trhv"
Oct 30 19:01:31.717686 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:31.717592    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-82305a-openshift-default-7cdd8f9877-sf7w8" event={"ID":"f9c90752-5508-445c-823e-3d9ea37f54b2","Type":"ContainerStarted","Data":"657ccac8b2eaa14fb21da70f9b5bf1b4f53aced0226bff2c13e5c64eb9e8ca8b"}
Oct 30 19:01:31.717686 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:31.717662    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-82305a-openshift-default-7cdd8f9877-sf7w8" event={"ID":"f9c90752-5508-445c-823e-3d9ea37f54b2","Type":"ContainerStarted","Data":"1055798aa2d74b89965a965515468e5d5af8c9f481bcb6c4bd8acc1a13166aa7"}
Oct 30 19:01:31.911995 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:31.911946    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-a-82305a-openshift-default-7cdd8f9877-sf7w8" podStartSLOduration=1.9119324789999999 podStartE2EDuration="1.911932479s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:01:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:01:31.906615957 +0000 UTC m=+1656.154021984" watchObservedRunningTime="2026-10-30 19:01:31.911932479 +0000 UTC m=+1656.159338503"
Oct 30 19:01:31.933127 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:31.933100    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-82305a-openshift-default-7cdd8f9877-sf7w8"
Oct 30 19:01:32.937906 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:32.937880    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-82305a-openshift-default-7cdd8f9877-sf7w8"
Oct 30 19:01:33.733014 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:33.732982    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-82305a-openshift-default-7cdd8f9877-sf7w8"
Oct 30 19:01:33.733970 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:01:33.733954    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-82305a-openshift-default-7cdd8f9877-sf7w8"
Oct 30 19:01:35.622517 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:01:35.622419    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-jbsw6" containerName="extractor"
Oct 30 19:02:05.620452 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:02:05.620418    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-jbsw6" containerName="extractor"
Oct 30 19:02:08.531544 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:08.531514    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-rjmdz"]
Oct 30 19:02:09.498274 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:09.498242    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-jlp7l"]
Oct 30 19:02:09.503947 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:09.503909    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-jlp7l"
Oct 30 19:02:09.548851 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:09.548813    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-2h9dw"]
Oct 30 19:02:09.553340 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:09.553320    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-2h9dw"
Oct 30 19:02:09.598610 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:09.598580    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6wjtz\" (UniqueName: \"kubernetes.io/projected/c9b213cb-4804-4bf8-a9d4-657bef3a11ba-kube-api-access-6wjtz\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-jlp7l\" (UID: \"c9b213cb-4804-4bf8-a9d4-657bef3a11ba\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-jlp7l"
Oct 30 19:02:09.598794 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:09.598639    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c9b213cb-4804-4bf8-a9d4-657bef3a11ba-workload-socket\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-jlp7l\" (UID: \"c9b213cb-4804-4bf8-a9d4-657bef3a11ba\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-jlp7l"
Oct 30 19:02:09.598794 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:09.598736    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c9b213cb-4804-4bf8-a9d4-657bef3a11ba-istio-envoy\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-jlp7l\" (UID: \"c9b213cb-4804-4bf8-a9d4-657bef3a11ba\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-jlp7l"
Oct 30 19:02:09.598794 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:09.598762    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c9b213cb-4804-4bf8-a9d4-657bef3a11ba-istio-podinfo\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-jlp7l\" (UID: \"c9b213cb-4804-4bf8-a9d4-657bef3a11ba\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-jlp7l"
Oct 30 19:02:09.598794 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:09.598790    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cea29cd6-99e6-4c4b-b232-6f2ca2ac5b91-istiod-ca-cert\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-2h9dw\" (UID: \"cea29cd6-99e6-4c4b-b232-6f2ca2ac5b91\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-2h9dw"
Oct 30 19:02:09.599006 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:09.598850    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cea29cd6-99e6-4c4b-b232-6f2ca2ac5b91-istio-data\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-2h9dw\" (UID: \"cea29cd6-99e6-4c4b-b232-6f2ca2ac5b91\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-2h9dw"
Oct 30 19:02:09.599006 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:09.598954    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gnjn2\" (UniqueName: \"kubernetes.io/projected/cea29cd6-99e6-4c4b-b232-6f2ca2ac5b91-kube-api-access-gnjn2\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-2h9dw\" (UID: \"cea29cd6-99e6-4c4b-b232-6f2ca2ac5b91\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-2h9dw"
Oct 30 19:02:09.599006 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:09.599003    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c9b213cb-4804-4bf8-a9d4-657bef3a11ba-istio-data\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-jlp7l\" (UID: \"c9b213cb-4804-4bf8-a9d4-657bef3a11ba\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-jlp7l"
Oct 30 19:02:09.599149 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:09.599029    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c9b213cb-4804-4bf8-a9d4-657bef3a11ba-credential-socket\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-jlp7l\" (UID: \"c9b213cb-4804-4bf8-a9d4-657bef3a11ba\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-jlp7l"
Oct 30 19:02:09.599149 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:09.599054    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c9b213cb-4804-4bf8-a9d4-657bef3a11ba-workload-certs\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-jlp7l\" (UID: \"c9b213cb-4804-4bf8-a9d4-657bef3a11ba\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-jlp7l"
Oct 30 19:02:09.599252 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:09.599183    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cea29cd6-99e6-4c4b-b232-6f2ca2ac5b91-workload-certs\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-2h9dw\" (UID: \"cea29cd6-99e6-4c4b-b232-6f2ca2ac5b91\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-2h9dw"
Oct 30 19:02:09.599319 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:09.599294    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cea29cd6-99e6-4c4b-b232-6f2ca2ac5b91-istio-envoy\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-2h9dw\" (UID: \"cea29cd6-99e6-4c4b-b232-6f2ca2ac5b91\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-2h9dw"
Oct 30 19:02:09.599319 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:09.599316    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c9b213cb-4804-4bf8-a9d4-657bef3a11ba-istio-token\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-jlp7l\" (UID: \"c9b213cb-4804-4bf8-a9d4-657bef3a11ba\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-jlp7l"
Oct 30 19:02:09.599439 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:09.599337    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cea29cd6-99e6-4c4b-b232-6f2ca2ac5b91-credential-socket\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-2h9dw\" (UID: \"cea29cd6-99e6-4c4b-b232-6f2ca2ac5b91\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-2h9dw"
Oct 30 19:02:09.599491 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:09.599450    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cea29cd6-99e6-4c4b-b232-6f2ca2ac5b91-istio-token\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-2h9dw\" (UID: \"cea29cd6-99e6-4c4b-b232-6f2ca2ac5b91\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-2h9dw"
Oct 30 19:02:09.599554 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:09.599490    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c9b213cb-4804-4bf8-a9d4-657bef3a11ba-istiod-ca-cert\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-jlp7l\" (UID: \"c9b213cb-4804-4bf8-a9d4-657bef3a11ba\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-jlp7l"
Oct 30 19:02:09.599608 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:09.599569    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cea29cd6-99e6-4c4b-b232-6f2ca2ac5b91-istio-podinfo\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-2h9dw\" (UID: \"cea29cd6-99e6-4c4b-b232-6f2ca2ac5b91\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-2h9dw"
Oct 30 19:02:09.599660 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:09.599625    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cea29cd6-99e6-4c4b-b232-6f2ca2ac5b91-workload-socket\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-2h9dw\" (UID: \"cea29cd6-99e6-4c4b-b232-6f2ca2ac5b91\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-2h9dw"
Oct 30 19:02:09.607720 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:09.607694    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-jlp7l"]
Oct 30 19:02:09.658740 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:09.658705    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-2h9dw"]
Oct 30 19:02:09.700936 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:09.700889    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cea29cd6-99e6-4c4b-b232-6f2ca2ac5b91-istio-podinfo\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-2h9dw\" (UID: \"cea29cd6-99e6-4c4b-b232-6f2ca2ac5b91\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-2h9dw"
Oct 30 19:02:09.701133 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:09.700954    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cea29cd6-99e6-4c4b-b232-6f2ca2ac5b91-workload-socket\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-2h9dw\" (UID: \"cea29cd6-99e6-4c4b-b232-6f2ca2ac5b91\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-2h9dw"
Oct 30 19:02:09.701133 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:09.701020    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6wjtz\" (UniqueName: \"kubernetes.io/projected/c9b213cb-4804-4bf8-a9d4-657bef3a11ba-kube-api-access-6wjtz\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-jlp7l\" (UID: \"c9b213cb-4804-4bf8-a9d4-657bef3a11ba\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-jlp7l"
Oct 30 19:02:09.701133 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:09.701071    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c9b213cb-4804-4bf8-a9d4-657bef3a11ba-workload-socket\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-jlp7l\" (UID: \"c9b213cb-4804-4bf8-a9d4-657bef3a11ba\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-jlp7l"
Oct 30 19:02:09.701133 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:09.701111    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c9b213cb-4804-4bf8-a9d4-657bef3a11ba-istio-envoy\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-jlp7l\" (UID: \"c9b213cb-4804-4bf8-a9d4-657bef3a11ba\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-jlp7l"
Oct 30 19:02:09.701417 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:09.701139    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c9b213cb-4804-4bf8-a9d4-657bef3a11ba-istio-podinfo\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-jlp7l\" (UID: \"c9b213cb-4804-4bf8-a9d4-657bef3a11ba\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-jlp7l"
Oct 30 19:02:09.701417 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:09.701167    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cea29cd6-99e6-4c4b-b232-6f2ca2ac5b91-istiod-ca-cert\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-2h9dw\" (UID: \"cea29cd6-99e6-4c4b-b232-6f2ca2ac5b91\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-2h9dw"
Oct 30 19:02:09.701417 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:09.701195    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cea29cd6-99e6-4c4b-b232-6f2ca2ac5b91-istio-data\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-2h9dw\" (UID: \"cea29cd6-99e6-4c4b-b232-6f2ca2ac5b91\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-2h9dw"
Oct 30 19:02:09.701417 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:09.701226    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gnjn2\" (UniqueName: \"kubernetes.io/projected/cea29cd6-99e6-4c4b-b232-6f2ca2ac5b91-kube-api-access-gnjn2\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-2h9dw\" (UID: \"cea29cd6-99e6-4c4b-b232-6f2ca2ac5b91\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-2h9dw"
Oct 30 19:02:09.701417 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:09.701259    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c9b213cb-4804-4bf8-a9d4-657bef3a11ba-istio-data\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-jlp7l\" (UID: \"c9b213cb-4804-4bf8-a9d4-657bef3a11ba\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-jlp7l"
Oct 30 19:02:09.701417 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:09.701341    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c9b213cb-4804-4bf8-a9d4-657bef3a11ba-credential-socket\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-jlp7l\" (UID: \"c9b213cb-4804-4bf8-a9d4-657bef3a11ba\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-jlp7l"
Oct 30 19:02:09.701417 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:09.701373    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c9b213cb-4804-4bf8-a9d4-657bef3a11ba-workload-certs\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-jlp7l\" (UID: \"c9b213cb-4804-4bf8-a9d4-657bef3a11ba\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-jlp7l"
Oct 30 19:02:09.701782 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:09.701462    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cea29cd6-99e6-4c4b-b232-6f2ca2ac5b91-workload-certs\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-2h9dw\" (UID: \"cea29cd6-99e6-4c4b-b232-6f2ca2ac5b91\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-2h9dw"
Oct 30 19:02:09.701782 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:09.701553    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cea29cd6-99e6-4c4b-b232-6f2ca2ac5b91-istio-envoy\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-2h9dw\" (UID: \"cea29cd6-99e6-4c4b-b232-6f2ca2ac5b91\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-2h9dw"
Oct 30 19:02:09.702096 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:09.702035    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c9b213cb-4804-4bf8-a9d4-657bef3a11ba-istio-data\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-jlp7l\" (UID: \"c9b213cb-4804-4bf8-a9d4-657bef3a11ba\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-jlp7l"
Oct 30 19:02:09.702326 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:09.702216    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c9b213cb-4804-4bf8-a9d4-657bef3a11ba-workload-socket\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-jlp7l\" (UID: \"c9b213cb-4804-4bf8-a9d4-657bef3a11ba\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-jlp7l"
Oct 30 19:02:09.703171 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:09.702578    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cea29cd6-99e6-4c4b-b232-6f2ca2ac5b91-istio-data\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-2h9dw\" (UID: \"cea29cd6-99e6-4c4b-b232-6f2ca2ac5b91\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-2h9dw"
Oct 30 19:02:09.703171 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:09.702833    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c9b213cb-4804-4bf8-a9d4-657bef3a11ba-workload-certs\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-jlp7l\" (UID: \"c9b213cb-4804-4bf8-a9d4-657bef3a11ba\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-jlp7l"
Oct 30 19:02:09.703171 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:09.703099    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c9b213cb-4804-4bf8-a9d4-657bef3a11ba-credential-socket\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-jlp7l\" (UID: \"c9b213cb-4804-4bf8-a9d4-657bef3a11ba\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-jlp7l"
Oct 30 19:02:09.703171 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:09.703120    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cea29cd6-99e6-4c4b-b232-6f2ca2ac5b91-workload-certs\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-2h9dw\" (UID: \"cea29cd6-99e6-4c4b-b232-6f2ca2ac5b91\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-2h9dw"
Oct 30 19:02:09.703171 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:09.703161    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c9b213cb-4804-4bf8-a9d4-657bef3a11ba-istio-token\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-jlp7l\" (UID: \"c9b213cb-4804-4bf8-a9d4-657bef3a11ba\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-jlp7l"
Oct 30 19:02:09.703545 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:09.703192    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cea29cd6-99e6-4c4b-b232-6f2ca2ac5b91-credential-socket\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-2h9dw\" (UID: \"cea29cd6-99e6-4c4b-b232-6f2ca2ac5b91\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-2h9dw"
Oct 30 19:02:09.703545 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:09.703238    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cea29cd6-99e6-4c4b-b232-6f2ca2ac5b91-istio-token\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-2h9dw\" (UID: \"cea29cd6-99e6-4c4b-b232-6f2ca2ac5b91\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-2h9dw"
Oct 30 19:02:09.703545 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:09.703318    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c9b213cb-4804-4bf8-a9d4-657bef3a11ba-istiod-ca-cert\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-jlp7l\" (UID: \"c9b213cb-4804-4bf8-a9d4-657bef3a11ba\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-jlp7l"
Oct 30 19:02:09.703827 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:09.703791    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cea29cd6-99e6-4c4b-b232-6f2ca2ac5b91-credential-socket\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-2h9dw\" (UID: \"cea29cd6-99e6-4c4b-b232-6f2ca2ac5b91\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-2h9dw"
Oct 30 19:02:09.704150 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:09.704122    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c9b213cb-4804-4bf8-a9d4-657bef3a11ba-istiod-ca-cert\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-jlp7l\" (UID: \"c9b213cb-4804-4bf8-a9d4-657bef3a11ba\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-jlp7l"
Oct 30 19:02:09.704362 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:09.704339    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cea29cd6-99e6-4c4b-b232-6f2ca2ac5b91-istiod-ca-cert\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-2h9dw\" (UID: \"cea29cd6-99e6-4c4b-b232-6f2ca2ac5b91\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-2h9dw"
Oct 30 19:02:09.704630 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:09.704591    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cea29cd6-99e6-4c4b-b232-6f2ca2ac5b91-istio-podinfo\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-2h9dw\" (UID: \"cea29cd6-99e6-4c4b-b232-6f2ca2ac5b91\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-2h9dw"
Oct 30 19:02:09.704772 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:09.704749    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cea29cd6-99e6-4c4b-b232-6f2ca2ac5b91-workload-socket\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-2h9dw\" (UID: \"cea29cd6-99e6-4c4b-b232-6f2ca2ac5b91\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-2h9dw"
Oct 30 19:02:09.704838 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:09.704788    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c9b213cb-4804-4bf8-a9d4-657bef3a11ba-istio-envoy\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-jlp7l\" (UID: \"c9b213cb-4804-4bf8-a9d4-657bef3a11ba\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-jlp7l"
Oct 30 19:02:09.705846 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:09.705815    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c9b213cb-4804-4bf8-a9d4-657bef3a11ba-istio-podinfo\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-jlp7l\" (UID: \"c9b213cb-4804-4bf8-a9d4-657bef3a11ba\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-jlp7l"
Oct 30 19:02:09.706250 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:09.706227    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cea29cd6-99e6-4c4b-b232-6f2ca2ac5b91-istio-envoy\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-2h9dw\" (UID: \"cea29cd6-99e6-4c4b-b232-6f2ca2ac5b91\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-2h9dw"
Oct 30 19:02:09.764940 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:09.764869    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6wjtz\" (UniqueName: \"kubernetes.io/projected/c9b213cb-4804-4bf8-a9d4-657bef3a11ba-kube-api-access-6wjtz\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-jlp7l\" (UID: \"c9b213cb-4804-4bf8-a9d4-657bef3a11ba\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-jlp7l"
Oct 30 19:02:09.765869 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:09.765837    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c9b213cb-4804-4bf8-a9d4-657bef3a11ba-istio-token\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-jlp7l\" (UID: \"c9b213cb-4804-4bf8-a9d4-657bef3a11ba\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-jlp7l"
Oct 30 19:02:09.766693 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:09.766668    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gnjn2\" (UniqueName: \"kubernetes.io/projected/cea29cd6-99e6-4c4b-b232-6f2ca2ac5b91-kube-api-access-gnjn2\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-2h9dw\" (UID: \"cea29cd6-99e6-4c4b-b232-6f2ca2ac5b91\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-2h9dw"
Oct 30 19:02:09.767947 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:09.767925    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cea29cd6-99e6-4c4b-b232-6f2ca2ac5b91-istio-token\") pod \"e2e-api-b-4ebcd2-openshift-default-59d766c94c-2h9dw\" (UID: \"cea29cd6-99e6-4c4b-b232-6f2ca2ac5b91\") " pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-2h9dw"
Oct 30 19:02:09.818559 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:09.818533    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-jlp7l"
Oct 30 19:02:09.868688 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:09.868655    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-2h9dw"
Oct 30 19:02:10.256822 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:10.256778    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-jlp7l"]
Oct 30 19:02:10.291391 ip-10-0-134-46 kubenswrapper[4888]: W1030 19:02:10.291366    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9b213cb_4804_4bf8_a9d4_657bef3a11ba.slice/crio-d05c6af5baec3ab4713037030e2bb6500fb126d622d8ff4e743bc499d2da2922 WatchSource:0}: Error finding container d05c6af5baec3ab4713037030e2bb6500fb126d622d8ff4e743bc499d2da2922: Status 404 returned error can't find the container with id d05c6af5baec3ab4713037030e2bb6500fb126d622d8ff4e743bc499d2da2922
Oct 30 19:02:10.293759 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:10.293725    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:02:10.293851 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:10.293791    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:02:10.293851 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:10.293817    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:02:10.311689 ip-10-0-134-46 kubenswrapper[4888]: W1030 19:02:10.311661    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcea29cd6_99e6_4c4b_b232_6f2ca2ac5b91.slice/crio-95d9fb377e7869ee76d3c7631b6fef4572896b286e33ef97a3050cfc6418fe14 WatchSource:0}: Error finding container 95d9fb377e7869ee76d3c7631b6fef4572896b286e33ef97a3050cfc6418fe14: Status 404 returned error can't find the container with id 95d9fb377e7869ee76d3c7631b6fef4572896b286e33ef97a3050cfc6418fe14
Oct 30 19:02:10.312710 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:10.312665    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-2h9dw"]
Oct 30 19:02:10.314178 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:10.314150    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:02:10.314315 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:10.314209    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:02:10.314315 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:10.314239    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:02:11.132134 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:11.132094    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-jlp7l" event={"ID":"c9b213cb-4804-4bf8-a9d4-657bef3a11ba","Type":"ContainerStarted","Data":"e7b2ddb4be2750affca10dc6c0b3a60f7866a4330759d9d8c4e9793494e47803"}
Oct 30 19:02:11.132732 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:11.132167    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-jlp7l" event={"ID":"c9b213cb-4804-4bf8-a9d4-657bef3a11ba","Type":"ContainerStarted","Data":"d05c6af5baec3ab4713037030e2bb6500fb126d622d8ff4e743bc499d2da2922"}
Oct 30 19:02:11.135193 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:11.135166    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-2h9dw" event={"ID":"cea29cd6-99e6-4c4b-b232-6f2ca2ac5b91","Type":"ContainerStarted","Data":"c9d73471a5244580b90cb1dcb49bb798fa16f9da9dc49b4b8d969b46a17f00d0"}
Oct 30 19:02:11.135367 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:11.135214    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-2h9dw" event={"ID":"cea29cd6-99e6-4c4b-b232-6f2ca2ac5b91","Type":"ContainerStarted","Data":"95d9fb377e7869ee76d3c7631b6fef4572896b286e33ef97a3050cfc6418fe14"}
Oct 30 19:02:11.259688 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:11.259637    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-jlp7l" podStartSLOduration=2.259622963 podStartE2EDuration="2.259622963s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:02:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:02:11.255599349 +0000 UTC m=+1695.503005375" watchObservedRunningTime="2026-10-30 19:02:11.259622963 +0000 UTC m=+1695.507028988"
Oct 30 19:02:11.386031 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:11.385823    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-2h9dw" podStartSLOduration=2.385801213 podStartE2EDuration="2.385801213s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:02:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:02:11.382799206 +0000 UTC m=+1695.630205245" watchObservedRunningTime="2026-10-30 19:02:11.385801213 +0000 UTC m=+1695.633207240"
Oct 30 19:02:11.824351 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:11.824055    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-jlp7l"
Oct 30 19:02:11.827775 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:11.827746    4888 patch_prober.go:29] interesting pod/e2e-api-b-4ebcd2-openshift-default-59d766c94c-jlp7l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.115:15021/healthz/ready\": dial tcp 10.132.0.115:15021: connect: connection refused" start-of-body=
Oct 30 19:02:11.827946 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:11.827814    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-jlp7l" podUID="c9b213cb-4804-4bf8-a9d4-657bef3a11ba" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.115:15021/healthz/ready\": dial tcp 10.132.0.115:15021: connect: connection refused"
Oct 30 19:02:11.875312 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:11.873003    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-2h9dw"
Oct 30 19:02:11.918359 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:11.911502    4888 patch_prober.go:29] interesting pod/e2e-api-b-4ebcd2-openshift-default-59d766c94c-2h9dw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.116:15021/healthz/ready\": dial tcp 10.132.0.116:15021: connect: connection refused" start-of-body=
Oct 30 19:02:11.918359 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:11.911566    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-2h9dw" podUID="cea29cd6-99e6-4c4b-b232-6f2ca2ac5b91" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.116:15021/healthz/ready\": dial tcp 10.132.0.116:15021: connect: connection refused"
Oct 30 19:02:12.824144 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:12.823670    4888 patch_prober.go:29] interesting pod/e2e-api-b-4ebcd2-openshift-default-59d766c94c-jlp7l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.115:15021/healthz/ready\": dial tcp 10.132.0.115:15021: connect: connection refused" start-of-body=
Oct 30 19:02:12.824144 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:12.823728    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-jlp7l" podUID="c9b213cb-4804-4bf8-a9d4-657bef3a11ba" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.115:15021/healthz/ready\": dial tcp 10.132.0.115:15021: connect: connection refused"
Oct 30 19:02:12.881310 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:12.879679    4888 patch_prober.go:29] interesting pod/e2e-api-b-4ebcd2-openshift-default-59d766c94c-2h9dw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.116:15021/healthz/ready\": dial tcp 10.132.0.116:15021: connect: connection refused" start-of-body=
Oct 30 19:02:12.881310 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:12.879742    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-2h9dw" podUID="cea29cd6-99e6-4c4b-b232-6f2ca2ac5b91" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.116:15021/healthz/ready\": dial tcp 10.132.0.116:15021: connect: connection refused"
Oct 30 19:02:13.819655 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:13.819617    4888 patch_prober.go:29] interesting pod/e2e-api-b-4ebcd2-openshift-default-59d766c94c-jlp7l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.115:15021/healthz/ready\": dial tcp 10.132.0.115:15021: connect: connection refused" start-of-body=
Oct 30 19:02:13.819940 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:13.819911    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-jlp7l" podUID="c9b213cb-4804-4bf8-a9d4-657bef3a11ba" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.115:15021/healthz/ready\": dial tcp 10.132.0.115:15021: connect: connection refused"
Oct 30 19:02:13.871638 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:13.871602    4888 patch_prober.go:29] interesting pod/e2e-api-b-4ebcd2-openshift-default-59d766c94c-2h9dw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.116:15021/healthz/ready\": dial tcp 10.132.0.116:15021: connect: connection refused" start-of-body=
Oct 30 19:02:13.872216 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:13.872185    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-2h9dw" podUID="cea29cd6-99e6-4c4b-b232-6f2ca2ac5b91" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.116:15021/healthz/ready\": dial tcp 10.132.0.116:15021: connect: connection refused"
Oct 30 19:02:14.819857 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:14.819823    4888 patch_prober.go:29] interesting pod/e2e-api-b-4ebcd2-openshift-default-59d766c94c-jlp7l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.115:15021/healthz/ready\": dial tcp 10.132.0.115:15021: connect: connection refused" start-of-body=
Oct 30 19:02:14.820151 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:14.820122    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-jlp7l" podUID="c9b213cb-4804-4bf8-a9d4-657bef3a11ba" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.115:15021/healthz/ready\": dial tcp 10.132.0.115:15021: connect: connection refused"
Oct 30 19:02:14.887336 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:14.887262    4888 patch_prober.go:29] interesting pod/e2e-api-b-4ebcd2-openshift-default-59d766c94c-2h9dw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.116:15021/healthz/ready\": dial tcp 10.132.0.116:15021: connect: connection refused" start-of-body=
Oct 30 19:02:14.888037 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:14.888007    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-2h9dw" podUID="cea29cd6-99e6-4c4b-b232-6f2ca2ac5b91" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.116:15021/healthz/ready\": dial tcp 10.132.0.116:15021: connect: connection refused"
Oct 30 19:02:15.819028 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:15.818978    4888 patch_prober.go:29] interesting pod/e2e-api-b-4ebcd2-openshift-default-59d766c94c-jlp7l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.115:15021/healthz/ready\": dial tcp 10.132.0.115:15021: connect: connection refused" start-of-body=
Oct 30 19:02:15.819243 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:15.819046    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-jlp7l" podUID="c9b213cb-4804-4bf8-a9d4-657bef3a11ba" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.115:15021/healthz/ready\": dial tcp 10.132.0.115:15021: connect: connection refused"
Oct 30 19:02:15.871603 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:15.871568    4888 patch_prober.go:29] interesting pod/e2e-api-b-4ebcd2-openshift-default-59d766c94c-2h9dw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.116:15021/healthz/ready\": dial tcp 10.132.0.116:15021: connect: connection refused" start-of-body=
Oct 30 19:02:15.871868 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:15.871839    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-2h9dw" podUID="cea29cd6-99e6-4c4b-b232-6f2ca2ac5b91" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.116:15021/healthz/ready\": dial tcp 10.132.0.116:15021: connect: connection refused"
Oct 30 19:02:16.824255 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:16.824219    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-jlp7l"
Oct 30 19:02:16.824715 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:16.824694    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-jlp7l"
Oct 30 19:02:16.825494 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:16.825474    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-jlp7l"
Oct 30 19:02:16.874028 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:16.874005    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-2h9dw"
Oct 30 19:02:16.874309 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:16.874274    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-2h9dw"
Oct 30 19:02:16.875080 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:16.875061    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-2h9dw"
Oct 30 19:02:24.905636 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:24.905604    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-85ab6a-57bf4bc848-2lrb7"]
Oct 30 19:02:24.906422 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:24.906374    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-85ab6a-57bf4bc848-2lrb7" podUID="3ba1c3a8-aa3d-45d6-bb89-c00b9993596e" containerName="maas-api" containerID="cri-o://32cc03c5edf83adbe0b8f775a7c9700ef088fecb7e4ad976f6c80d88ef041e09" gracePeriod=30
Oct 30 19:02:25.261127 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:25.261095    4888 generic.go:356] "Generic (PLEG): container finished" podID="3ba1c3a8-aa3d-45d6-bb89-c00b9993596e" containerID="32cc03c5edf83adbe0b8f775a7c9700ef088fecb7e4ad976f6c80d88ef041e09" exitCode=0
Oct 30 19:02:25.261273 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:25.261176    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-85ab6a-57bf4bc848-2lrb7" event={"ID":"3ba1c3a8-aa3d-45d6-bb89-c00b9993596e","Type":"ContainerDied","Data":"32cc03c5edf83adbe0b8f775a7c9700ef088fecb7e4ad976f6c80d88ef041e09"}
Oct 30 19:02:25.261273 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:25.261242    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-85ab6a-57bf4bc848-2lrb7" event={"ID":"3ba1c3a8-aa3d-45d6-bb89-c00b9993596e","Type":"ContainerDied","Data":"28e710fc9477c5974ebfea96abe3bc0fa27df6fa11295f5bf71ddc1482a9d337"}
Oct 30 19:02:25.261273 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:25.261267    4888 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="28e710fc9477c5974ebfea96abe3bc0fa27df6fa11295f5bf71ddc1482a9d337"
Oct 30 19:02:25.266103 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:25.266082    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-85ab6a-57bf4bc848-2lrb7"
Oct 30 19:02:25.362542 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:25.361393    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3ba1c3a8-aa3d-45d6-bb89-c00b9993596e-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3ba1c3a8-aa3d-45d6-bb89-c00b9993596e-openshift-service-ca\") pod \"3ba1c3a8-aa3d-45d6-bb89-c00b9993596e\" (UID: \"3ba1c3a8-aa3d-45d6-bb89-c00b9993596e\") "
Oct 30 19:02:25.362542 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:25.361516    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3ba1c3a8-aa3d-45d6-bb89-c00b9993596e-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3ba1c3a8-aa3d-45d6-bb89-c00b9993596e-maas-api-tls\") pod \"3ba1c3a8-aa3d-45d6-bb89-c00b9993596e\" (UID: \"3ba1c3a8-aa3d-45d6-bb89-c00b9993596e\") "
Oct 30 19:02:25.362542 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:25.361571    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3ba1c3a8-aa3d-45d6-bb89-c00b9993596e-kube-api-access-fm7ph\" (UniqueName: \"kubernetes.io/projected/3ba1c3a8-aa3d-45d6-bb89-c00b9993596e-kube-api-access-fm7ph\") pod \"3ba1c3a8-aa3d-45d6-bb89-c00b9993596e\" (UID: \"3ba1c3a8-aa3d-45d6-bb89-c00b9993596e\") "
Oct 30 19:02:25.362542 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:25.361676    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3ba1c3a8-aa3d-45d6-bb89-c00b9993596e-metrics-certs\" (UniqueName: \"kubernetes.io/secret/3ba1c3a8-aa3d-45d6-bb89-c00b9993596e-metrics-certs\") pod \"3ba1c3a8-aa3d-45d6-bb89-c00b9993596e\" (UID: \"3ba1c3a8-aa3d-45d6-bb89-c00b9993596e\") "
Oct 30 19:02:25.366695 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:25.366622    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3ba1c3a8-aa3d-45d6-bb89-c00b9993596e-metrics-certs" pod "3ba1c3a8-aa3d-45d6-bb89-c00b9993596e" (UID: "3ba1c3a8-aa3d-45d6-bb89-c00b9993596e"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:02:25.367310 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:25.367209    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3ba1c3a8-aa3d-45d6-bb89-c00b9993596e-openshift-service-ca" pod "3ba1c3a8-aa3d-45d6-bb89-c00b9993596e" (UID: "3ba1c3a8-aa3d-45d6-bb89-c00b9993596e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:02:25.370692 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:25.370655    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3ba1c3a8-aa3d-45d6-bb89-c00b9993596e-maas-api-tls" pod "3ba1c3a8-aa3d-45d6-bb89-c00b9993596e" (UID: "3ba1c3a8-aa3d-45d6-bb89-c00b9993596e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:02:25.376767 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:25.376736    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3ba1c3a8-aa3d-45d6-bb89-c00b9993596e-kube-api-access-fm7ph" pod "3ba1c3a8-aa3d-45d6-bb89-c00b9993596e" (UID: "3ba1c3a8-aa3d-45d6-bb89-c00b9993596e"). InnerVolumeSpecName "kube-api-access-fm7ph". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:02:25.463209 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:25.463180    4888 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3ba1c3a8-aa3d-45d6-bb89-c00b9993596e-maas-api-tls\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:02:25.463209 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:25.463206    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fm7ph\" (UniqueName: \"kubernetes.io/projected/3ba1c3a8-aa3d-45d6-bb89-c00b9993596e-kube-api-access-fm7ph\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:02:25.463437 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:25.463218    4888 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3ba1c3a8-aa3d-45d6-bb89-c00b9993596e-metrics-certs\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:02:25.463437 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:25.463229    4888 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3ba1c3a8-aa3d-45d6-bb89-c00b9993596e-openshift-service-ca\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:02:26.305857 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:26.305769    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-85ab6a-57bf4bc848-2lrb7"
Oct 30 19:02:26.486819 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:26.486783    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-85ab6a-57bf4bc848-2lrb7"]
Oct 30 19:02:26.537389 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:26.537347    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-85ab6a-57bf4bc848-2lrb7"]
Oct 30 19:02:27.892183 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:27.892151    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3ba1c3a8-aa3d-45d6-bb89-c00b9993596e" path="/var/lib/kubelet/pods/3ba1c3a8-aa3d-45d6-bb89-c00b9993596e/volumes"
Oct 30 19:02:35.621814 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:02:35.621759    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-jbsw6" containerName="extractor"
Oct 30 19:02:43.790865 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:43.790831    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-8sg49"]
Oct 30 19:02:43.791443 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:43.791117    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-8sg49" podUID="863c528e-6d99-4867-bbff-02bdc504c923" containerName="istio-proxy" containerID="cri-o://ae9b4220c08c1c6b8b0ad2f8eb31f638f52579d81c08ffc2d0a288a2c2f6dd87" gracePeriod=30
Oct 30 19:02:45.696757 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:45.696725    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-rjmdz"]
Oct 30 19:02:49.127225 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:49.127200    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-8sg49"
Oct 30 19:02:49.244456 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:49.244427    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/863c528e-6d99-4867-bbff-02bdc504c923-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/863c528e-6d99-4867-bbff-02bdc504c923-istiod-ca-cert\") pod \"863c528e-6d99-4867-bbff-02bdc504c923\" (UID: \"863c528e-6d99-4867-bbff-02bdc504c923\") "
Oct 30 19:02:49.244653 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:49.244483    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/863c528e-6d99-4867-bbff-02bdc504c923-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/863c528e-6d99-4867-bbff-02bdc504c923-istio-data\") pod \"863c528e-6d99-4867-bbff-02bdc504c923\" (UID: \"863c528e-6d99-4867-bbff-02bdc504c923\") "
Oct 30 19:02:49.244653 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:49.244572    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/863c528e-6d99-4867-bbff-02bdc504c923-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/863c528e-6d99-4867-bbff-02bdc504c923-credential-socket\") pod \"863c528e-6d99-4867-bbff-02bdc504c923\" (UID: \"863c528e-6d99-4867-bbff-02bdc504c923\") "
Oct 30 19:02:49.244653 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:49.244612    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/863c528e-6d99-4867-bbff-02bdc504c923-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/863c528e-6d99-4867-bbff-02bdc504c923-istio-envoy\") pod \"863c528e-6d99-4867-bbff-02bdc504c923\" (UID: \"863c528e-6d99-4867-bbff-02bdc504c923\") "
Oct 30 19:02:49.244834 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:49.244642    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/863c528e-6d99-4867-bbff-02bdc504c923-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/863c528e-6d99-4867-bbff-02bdc504c923-workload-socket\") pod \"863c528e-6d99-4867-bbff-02bdc504c923\" (UID: \"863c528e-6d99-4867-bbff-02bdc504c923\") "
Oct 30 19:02:49.244834 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:49.244718    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/863c528e-6d99-4867-bbff-02bdc504c923-istio-token\" (UniqueName: \"kubernetes.io/projected/863c528e-6d99-4867-bbff-02bdc504c923-istio-token\") pod \"863c528e-6d99-4867-bbff-02bdc504c923\" (UID: \"863c528e-6d99-4867-bbff-02bdc504c923\") "
Oct 30 19:02:49.244834 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:49.244800    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/863c528e-6d99-4867-bbff-02bdc504c923-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/863c528e-6d99-4867-bbff-02bdc504c923-istio-podinfo\") pod \"863c528e-6d99-4867-bbff-02bdc504c923\" (UID: \"863c528e-6d99-4867-bbff-02bdc504c923\") "
Oct 30 19:02:49.244834 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:49.244819    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/863c528e-6d99-4867-bbff-02bdc504c923-istio-data" pod "863c528e-6d99-4867-bbff-02bdc504c923" (UID: "863c528e-6d99-4867-bbff-02bdc504c923"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:02:49.245037 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:49.244861    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/863c528e-6d99-4867-bbff-02bdc504c923-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/863c528e-6d99-4867-bbff-02bdc504c923-workload-certs\") pod \"863c528e-6d99-4867-bbff-02bdc504c923\" (UID: \"863c528e-6d99-4867-bbff-02bdc504c923\") "
Oct 30 19:02:49.245037 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:49.244932    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/863c528e-6d99-4867-bbff-02bdc504c923-credential-socket" pod "863c528e-6d99-4867-bbff-02bdc504c923" (UID: "863c528e-6d99-4867-bbff-02bdc504c923"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:02:49.245037 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:49.244939    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/863c528e-6d99-4867-bbff-02bdc504c923-workload-socket" pod "863c528e-6d99-4867-bbff-02bdc504c923" (UID: "863c528e-6d99-4867-bbff-02bdc504c923"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:02:49.245037 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:49.244974    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/863c528e-6d99-4867-bbff-02bdc504c923-istiod-ca-cert" pod "863c528e-6d99-4867-bbff-02bdc504c923" (UID: "863c528e-6d99-4867-bbff-02bdc504c923"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:02:49.245037 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:49.244995    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/863c528e-6d99-4867-bbff-02bdc504c923-kube-api-access-kdk88\" (UniqueName: \"kubernetes.io/projected/863c528e-6d99-4867-bbff-02bdc504c923-kube-api-access-kdk88\") pod \"863c528e-6d99-4867-bbff-02bdc504c923\" (UID: \"863c528e-6d99-4867-bbff-02bdc504c923\") "
Oct 30 19:02:49.245307 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:49.245229    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/863c528e-6d99-4867-bbff-02bdc504c923-workload-certs" pod "863c528e-6d99-4867-bbff-02bdc504c923" (UID: "863c528e-6d99-4867-bbff-02bdc504c923"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:02:49.245597 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:49.245574    4888 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/863c528e-6d99-4867-bbff-02bdc504c923-istiod-ca-cert\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:02:49.245679 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:49.245600    4888 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/863c528e-6d99-4867-bbff-02bdc504c923-istio-data\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:02:49.245679 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:49.245616    4888 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/863c528e-6d99-4867-bbff-02bdc504c923-credential-socket\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:02:49.245679 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:49.245629    4888 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/863c528e-6d99-4867-bbff-02bdc504c923-workload-socket\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:02:49.245679 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:49.245643    4888 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/863c528e-6d99-4867-bbff-02bdc504c923-workload-certs\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:02:49.247443 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:49.247409    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/863c528e-6d99-4867-bbff-02bdc504c923-istio-envoy" pod "863c528e-6d99-4867-bbff-02bdc504c923" (UID: "863c528e-6d99-4867-bbff-02bdc504c923"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:02:49.247546 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:49.247514    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/863c528e-6d99-4867-bbff-02bdc504c923-istio-podinfo" pod "863c528e-6d99-4867-bbff-02bdc504c923" (UID: "863c528e-6d99-4867-bbff-02bdc504c923"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:02:49.247677 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:49.247659    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/863c528e-6d99-4867-bbff-02bdc504c923-istio-token" pod "863c528e-6d99-4867-bbff-02bdc504c923" (UID: "863c528e-6d99-4867-bbff-02bdc504c923"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:02:49.247726 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:49.247688    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/863c528e-6d99-4867-bbff-02bdc504c923-kube-api-access-kdk88" pod "863c528e-6d99-4867-bbff-02bdc504c923" (UID: "863c528e-6d99-4867-bbff-02bdc504c923"). InnerVolumeSpecName "kube-api-access-kdk88". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:02:49.347473 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:49.347441    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kdk88\" (UniqueName: \"kubernetes.io/projected/863c528e-6d99-4867-bbff-02bdc504c923-kube-api-access-kdk88\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:02:49.347473 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:49.347471    4888 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/863c528e-6d99-4867-bbff-02bdc504c923-istio-envoy\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:02:49.347681 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:49.347482    4888 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/863c528e-6d99-4867-bbff-02bdc504c923-istio-token\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:02:49.347681 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:49.347491    4888 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/863c528e-6d99-4867-bbff-02bdc504c923-istio-podinfo\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:02:49.546660 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:49.546575    4888 generic.go:356] "Generic (PLEG): container finished" podID="863c528e-6d99-4867-bbff-02bdc504c923" containerID="ae9b4220c08c1c6b8b0ad2f8eb31f638f52579d81c08ffc2d0a288a2c2f6dd87" exitCode=0
Oct 30 19:02:49.546660 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:49.546633    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-8sg49"
Oct 30 19:02:49.546895 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:49.546666    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-8sg49" event={"ID":"863c528e-6d99-4867-bbff-02bdc504c923","Type":"ContainerDied","Data":"ae9b4220c08c1c6b8b0ad2f8eb31f638f52579d81c08ffc2d0a288a2c2f6dd87"}
Oct 30 19:02:49.546895 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:49.546733    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-8sg49" event={"ID":"863c528e-6d99-4867-bbff-02bdc504c923","Type":"ContainerDied","Data":"52a92e271aa6358e9c96015bda2272c78c7a88ab5f3726f0501a82792b6e48d3"}
Oct 30 19:02:49.546895 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:49.546747    4888 scope.go:122] "RemoveContainer" containerID="ae9b4220c08c1c6b8b0ad2f8eb31f638f52579d81c08ffc2d0a288a2c2f6dd87"
Oct 30 19:02:49.566320 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:49.566273    4888 scope.go:122] "RemoveContainer" containerID="ae9b4220c08c1c6b8b0ad2f8eb31f638f52579d81c08ffc2d0a288a2c2f6dd87"
Oct 30 19:02:49.572560 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:02:49.572478    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ae9b4220c08c1c6b8b0ad2f8eb31f638f52579d81c08ffc2d0a288a2c2f6dd87\": container with ID starting with ae9b4220c08c1c6b8b0ad2f8eb31f638f52579d81c08ffc2d0a288a2c2f6dd87 not found: ID does not exist" containerID="ae9b4220c08c1c6b8b0ad2f8eb31f638f52579d81c08ffc2d0a288a2c2f6dd87"
Oct 30 19:02:49.572560 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:49.572523    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ae9b4220c08c1c6b8b0ad2f8eb31f638f52579d81c08ffc2d0a288a2c2f6dd87"} err="failed to get container status \"ae9b4220c08c1c6b8b0ad2f8eb31f638f52579d81c08ffc2d0a288a2c2f6dd87\": rpc error: code = NotFound desc = could not find container \"ae9b4220c08c1c6b8b0ad2f8eb31f638f52579d81c08ffc2d0a288a2c2f6dd87\": container with ID starting with ae9b4220c08c1c6b8b0ad2f8eb31f638f52579d81c08ffc2d0a288a2c2f6dd87 not found: ID does not exist"
Oct 30 19:02:49.726095 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:49.726063    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-8sg49"]
Oct 30 19:02:49.729959 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:49.729931    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-8sg49"]
Oct 30 19:02:49.907303 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:49.907261    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="863c528e-6d99-4867-bbff-02bdc504c923" path="/var/lib/kubelet/pods/863c528e-6d99-4867-bbff-02bdc504c923/volumes"
Oct 30 19:02:56.614983 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:56.614894    4888 generic.go:356] "Generic (PLEG): container finished" podID="f9c90752-5508-445c-823e-3d9ea37f54b2" containerID="657ccac8b2eaa14fb21da70f9b5bf1b4f53aced0226bff2c13e5c64eb9e8ca8b" exitCode=0
Oct 30 19:02:56.615532 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:56.614984    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-82305a-openshift-default-7cdd8f9877-sf7w8" event={"ID":"f9c90752-5508-445c-823e-3d9ea37f54b2","Type":"ContainerDied","Data":"657ccac8b2eaa14fb21da70f9b5bf1b4f53aced0226bff2c13e5c64eb9e8ca8b"}
Oct 30 19:02:56.615602 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:56.615540    4888 scope.go:122] "RemoveContainer" containerID="657ccac8b2eaa14fb21da70f9b5bf1b4f53aced0226bff2c13e5c64eb9e8ca8b"
Oct 30 19:02:56.617219 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:56.617194    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:02:56.617354 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:56.617302    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:02:56.617354 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:56.617338    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:02:57.625517 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:57.625477    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-82305a-openshift-default-7cdd8f9877-sf7w8" event={"ID":"f9c90752-5508-445c-823e-3d9ea37f54b2","Type":"ContainerStarted","Data":"ded22ec4462b0c07cf005e7b484f8f79e2938b72652d3f2c232932dd98c96b7b"}
Oct 30 19:02:57.933455 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:57.933368    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-82305a-openshift-default-7cdd8f9877-sf7w8"
Oct 30 19:02:57.933636 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:57.933614    4888 patch_prober.go:29] interesting pod/e2e-ipp-a-82305a-openshift-default-7cdd8f9877-sf7w8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.114:15021/healthz/ready\": dial tcp 10.132.0.114:15021: connect: connection refused" start-of-body=
Oct 30 19:02:57.933689 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:57.933666    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-82305a-openshift-default-7cdd8f9877-sf7w8" podUID="f9c90752-5508-445c-823e-3d9ea37f54b2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.114:15021/healthz/ready\": dial tcp 10.132.0.114:15021: connect: connection refused"
Oct 30 19:02:58.933439 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:58.933403    4888 patch_prober.go:29] interesting pod/e2e-ipp-a-82305a-openshift-default-7cdd8f9877-sf7w8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.114:15021/healthz/ready\": dial tcp 10.132.0.114:15021: connect: connection refused" start-of-body=
Oct 30 19:02:58.933927 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:58.933468    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-82305a-openshift-default-7cdd8f9877-sf7w8" podUID="f9c90752-5508-445c-823e-3d9ea37f54b2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.114:15021/healthz/ready\": dial tcp 10.132.0.114:15021: connect: connection refused"
Oct 30 19:02:59.933435 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:59.933395    4888 patch_prober.go:29] interesting pod/e2e-ipp-a-82305a-openshift-default-7cdd8f9877-sf7w8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.114:15021/healthz/ready\": dial tcp 10.132.0.114:15021: connect: connection refused" start-of-body=
Oct 30 19:02:59.933621 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:02:59.933450    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-82305a-openshift-default-7cdd8f9877-sf7w8" podUID="f9c90752-5508-445c-823e-3d9ea37f54b2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.114:15021/healthz/ready\": dial tcp 10.132.0.114:15021: connect: connection refused"
Oct 30 19:03:00.933109 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:00.933073    4888 patch_prober.go:29] interesting pod/e2e-ipp-a-82305a-openshift-default-7cdd8f9877-sf7w8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.114:15021/healthz/ready\": dial tcp 10.132.0.114:15021: connect: connection refused" start-of-body=
Oct 30 19:03:00.933325 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:00.933138    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-82305a-openshift-default-7cdd8f9877-sf7w8" podUID="f9c90752-5508-445c-823e-3d9ea37f54b2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.114:15021/healthz/ready\": dial tcp 10.132.0.114:15021: connect: connection refused"
Oct 30 19:03:01.933520 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:01.933489    4888 patch_prober.go:29] interesting pod/e2e-ipp-a-82305a-openshift-default-7cdd8f9877-sf7w8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.114:15021/healthz/ready\": dial tcp 10.132.0.114:15021: connect: connection refused" start-of-body=
Oct 30 19:03:01.934026 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:01.933546    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-82305a-openshift-default-7cdd8f9877-sf7w8" podUID="f9c90752-5508-445c-823e-3d9ea37f54b2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.114:15021/healthz/ready\": dial tcp 10.132.0.114:15021: connect: connection refused"
Oct 30 19:03:02.937792 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:02.937761    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-82305a-openshift-default-7cdd8f9877-sf7w8"
Oct 30 19:03:02.938311 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:02.938272    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-82305a-openshift-default-7cdd8f9877-sf7w8"
Oct 30 19:03:02.939053 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:02.939034    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-82305a-openshift-default-7cdd8f9877-sf7w8"
Oct 30 19:03:05.623305 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:03:05.623199    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-jbsw6" containerName="extractor"
Oct 30 19:03:12.092745 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:12.092704    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-83d5bd-64886885df-zj2nz"]
Oct 30 19:03:12.093262 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:12.092945    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-83d5bd-64886885df-zj2nz" podUID="998d1b1d-e656-49b0-8763-540ef647899d" containerName="maas-api" containerID="cri-o://0a65bab8677ec9b71030c536b117e43b5405b079b9005c88898ca5640f0a7182" gracePeriod=30
Oct 30 19:03:12.569851 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:12.569516    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-83d5bd-64886885df-zj2nz"
Oct 30 19:03:12.607817 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:12.606298    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/998d1b1d-e656-49b0-8763-540ef647899d-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/998d1b1d-e656-49b0-8763-540ef647899d-openshift-service-ca\") pod \"998d1b1d-e656-49b0-8763-540ef647899d\" (UID: \"998d1b1d-e656-49b0-8763-540ef647899d\") "
Oct 30 19:03:12.607817 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:12.606370    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/998d1b1d-e656-49b0-8763-540ef647899d-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/998d1b1d-e656-49b0-8763-540ef647899d-maas-api-tls\") pod \"998d1b1d-e656-49b0-8763-540ef647899d\" (UID: \"998d1b1d-e656-49b0-8763-540ef647899d\") "
Oct 30 19:03:12.607817 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:12.606425    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/998d1b1d-e656-49b0-8763-540ef647899d-kube-api-access-fpdss\" (UniqueName: \"kubernetes.io/projected/998d1b1d-e656-49b0-8763-540ef647899d-kube-api-access-fpdss\") pod \"998d1b1d-e656-49b0-8763-540ef647899d\" (UID: \"998d1b1d-e656-49b0-8763-540ef647899d\") "
Oct 30 19:03:12.607817 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:12.606716    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/998d1b1d-e656-49b0-8763-540ef647899d-metrics-certs\" (UniqueName: \"kubernetes.io/secret/998d1b1d-e656-49b0-8763-540ef647899d-metrics-certs\") pod \"998d1b1d-e656-49b0-8763-540ef647899d\" (UID: \"998d1b1d-e656-49b0-8763-540ef647899d\") "
Oct 30 19:03:12.609338 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:12.609251    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/998d1b1d-e656-49b0-8763-540ef647899d-openshift-service-ca" pod "998d1b1d-e656-49b0-8763-540ef647899d" (UID: "998d1b1d-e656-49b0-8763-540ef647899d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:03:12.610203 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:12.610132    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/998d1b1d-e656-49b0-8763-540ef647899d-metrics-certs" pod "998d1b1d-e656-49b0-8763-540ef647899d" (UID: "998d1b1d-e656-49b0-8763-540ef647899d"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:03:12.611508 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:12.611461    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/998d1b1d-e656-49b0-8763-540ef647899d-maas-api-tls" pod "998d1b1d-e656-49b0-8763-540ef647899d" (UID: "998d1b1d-e656-49b0-8763-540ef647899d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:03:12.612230 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:12.612196    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/998d1b1d-e656-49b0-8763-540ef647899d-kube-api-access-fpdss" pod "998d1b1d-e656-49b0-8763-540ef647899d" (UID: "998d1b1d-e656-49b0-8763-540ef647899d"). InnerVolumeSpecName "kube-api-access-fpdss". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:03:12.709211 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:12.709138    4888 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/998d1b1d-e656-49b0-8763-540ef647899d-metrics-certs\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:03:12.709211 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:12.709176    4888 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/998d1b1d-e656-49b0-8763-540ef647899d-openshift-service-ca\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:03:12.709211 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:12.709190    4888 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/998d1b1d-e656-49b0-8763-540ef647899d-maas-api-tls\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:03:12.709211 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:12.709203    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fpdss\" (UniqueName: \"kubernetes.io/projected/998d1b1d-e656-49b0-8763-540ef647899d-kube-api-access-fpdss\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:03:12.731186 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:12.731152    4888 generic.go:356] "Generic (PLEG): container finished" podID="998d1b1d-e656-49b0-8763-540ef647899d" containerID="0a65bab8677ec9b71030c536b117e43b5405b079b9005c88898ca5640f0a7182" exitCode=0
Oct 30 19:03:12.731384 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:12.731201    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-83d5bd-64886885df-zj2nz"
Oct 30 19:03:12.731384 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:12.731230    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-83d5bd-64886885df-zj2nz" event={"ID":"998d1b1d-e656-49b0-8763-540ef647899d","Type":"ContainerDied","Data":"0a65bab8677ec9b71030c536b117e43b5405b079b9005c88898ca5640f0a7182"}
Oct 30 19:03:12.731384 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:12.731319    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-83d5bd-64886885df-zj2nz" event={"ID":"998d1b1d-e656-49b0-8763-540ef647899d","Type":"ContainerDied","Data":"da0ef95b80f29ba342578b597185f930ef1df84b97eb2ab3be729d8ac81168f7"}
Oct 30 19:03:12.731384 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:12.731345    4888 scope.go:122] "RemoveContainer" containerID="0a65bab8677ec9b71030c536b117e43b5405b079b9005c88898ca5640f0a7182"
Oct 30 19:03:12.753523 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:12.753500    4888 scope.go:122] "RemoveContainer" containerID="0a65bab8677ec9b71030c536b117e43b5405b079b9005c88898ca5640f0a7182"
Oct 30 19:03:12.753856 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:03:12.753827    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0a65bab8677ec9b71030c536b117e43b5405b079b9005c88898ca5640f0a7182\": container with ID starting with 0a65bab8677ec9b71030c536b117e43b5405b079b9005c88898ca5640f0a7182 not found: ID does not exist" containerID="0a65bab8677ec9b71030c536b117e43b5405b079b9005c88898ca5640f0a7182"
Oct 30 19:03:12.753955 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:12.753864    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0a65bab8677ec9b71030c536b117e43b5405b079b9005c88898ca5640f0a7182"} err="failed to get container status \"0a65bab8677ec9b71030c536b117e43b5405b079b9005c88898ca5640f0a7182\": rpc error: code = NotFound desc = could not find container \"0a65bab8677ec9b71030c536b117e43b5405b079b9005c88898ca5640f0a7182\": container with ID starting with 0a65bab8677ec9b71030c536b117e43b5405b079b9005c88898ca5640f0a7182 not found: ID does not exist"
Oct 30 19:03:12.917524 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:12.917462    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-83d5bd-64886885df-zj2nz"]
Oct 30 19:03:12.925765 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:12.924068    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-83d5bd-64886885df-zj2nz"]
Oct 30 19:03:13.893049 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:13.893009    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="998d1b1d-e656-49b0-8763-540ef647899d" path="/var/lib/kubelet/pods/998d1b1d-e656-49b0-8763-540ef647899d/volumes"
Oct 30 19:03:16.574074 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:16.574037    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-nvlbr"]
Oct 30 19:03:16.574795 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:16.574310    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-nvlbr" podUID="03fac346-2d8c-4d15-8e8a-c7e6571ad9b1" containerName="istio-proxy" containerID="cri-o://3e473820a56a96f987ec7165ff8d8938fdc360032470a99db092f1ebdbf31e8c" gracePeriod=30
Oct 30 19:03:16.583325 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:16.583271    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-v2sdh"]
Oct 30 19:03:16.583523 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:16.583500    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-v2sdh" podUID="687d01bd-f2ca-4fe3-8766-c8030df17da4" containerName="istio-proxy" containerID="cri-o://d345dd999a7f21095883da557949bd436a24bad41ffe6c2690ae14f8da2219cf" gracePeriod=30
Oct 30 19:03:16.614581 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:16.614541    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-9k7xv"]
Oct 30 19:03:16.614796 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:16.614760    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-9k7xv" podUID="427c5128-a124-4dd6-9748-a4dca0a85a4f" containerName="istio-proxy" containerID="cri-o://c5011ae29c57bbfe0a14858cc96b1fe1836fcf5bab80c336877452bfa7ab26b2" gracePeriod=30
Oct 30 19:03:18.028505 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:18.028461    4888 patch_prober.go:29] interesting pod/e2e-mt-66baf45c-openshift-default-974f9d7b9-nvlbr container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 19:03:18.029007 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:18.028524    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-nvlbr" podUID="03fac346-2d8c-4d15-8e8a-c7e6571ad9b1" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 19:03:18.207091 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:18.207052    4888 patch_prober.go:29] interesting pod/e2e-mt-66baf45c-openshift-default-974f9d7b9-v2sdh container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 19:03:18.207319 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:18.207110    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-v2sdh" podUID="687d01bd-f2ca-4fe3-8766-c8030df17da4" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 19:03:21.734275 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:21.733985    4888 patch_prober.go:29] interesting pod/e2e-mt-66baf45c-openshift-default-974f9d7b9-9k7xv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.100:15021/healthz/ready\": dial tcp 10.132.0.100:15021: connect: connection refused" start-of-body=
Oct 30 19:03:21.734275 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:21.734036    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-9k7xv" podUID="427c5128-a124-4dd6-9748-a4dca0a85a4f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.100:15021/healthz/ready\": dial tcp 10.132.0.100:15021: connect: connection refused"
Oct 30 19:03:21.930587 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:21.930549    4888 generic.go:356] "Generic (PLEG): container finished" podID="427c5128-a124-4dd6-9748-a4dca0a85a4f" containerID="c5011ae29c57bbfe0a14858cc96b1fe1836fcf5bab80c336877452bfa7ab26b2" exitCode=0
Oct 30 19:03:21.930837 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:21.930678    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-9k7xv" event={"ID":"427c5128-a124-4dd6-9748-a4dca0a85a4f","Type":"ContainerDied","Data":"c5011ae29c57bbfe0a14858cc96b1fe1836fcf5bab80c336877452bfa7ab26b2"}
Oct 30 19:03:21.930837 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:21.930740    4888 scope.go:122] "RemoveContainer" containerID="096083c6914a7bf98e5b8b3f42b9e4584ee0f3434a1d4dbf182585688b1ff6ef"
Oct 30 19:03:21.931324 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:21.931264    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-v2sdh"
Oct 30 19:03:21.934088 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:21.934070    4888 generic.go:356] "Generic (PLEG): container finished" podID="03fac346-2d8c-4d15-8e8a-c7e6571ad9b1" containerID="3e473820a56a96f987ec7165ff8d8938fdc360032470a99db092f1ebdbf31e8c" exitCode=0
Oct 30 19:03:21.934199 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:21.934184    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-nvlbr" event={"ID":"03fac346-2d8c-4d15-8e8a-c7e6571ad9b1","Type":"ContainerDied","Data":"3e473820a56a96f987ec7165ff8d8938fdc360032470a99db092f1ebdbf31e8c"}
Oct 30 19:03:21.935925 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:21.935899    4888 generic.go:356] "Generic (PLEG): container finished" podID="687d01bd-f2ca-4fe3-8766-c8030df17da4" containerID="d345dd999a7f21095883da557949bd436a24bad41ffe6c2690ae14f8da2219cf" exitCode=0
Oct 30 19:03:21.936040 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:21.935935    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-v2sdh"
Oct 30 19:03:21.936040 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:21.935954    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-v2sdh" event={"ID":"687d01bd-f2ca-4fe3-8766-c8030df17da4","Type":"ContainerDied","Data":"d345dd999a7f21095883da557949bd436a24bad41ffe6c2690ae14f8da2219cf"}
Oct 30 19:03:21.936040 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:21.935993    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-v2sdh" event={"ID":"687d01bd-f2ca-4fe3-8766-c8030df17da4","Type":"ContainerDied","Data":"86fb8239296aea1f9b0163c5f439c79b2decfb4ffe27abb4d0b883d3bdf99061"}
Oct 30 19:03:21.962330 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:21.948875    4888 scope.go:122] "RemoveContainer" containerID="d345dd999a7f21095883da557949bd436a24bad41ffe6c2690ae14f8da2219cf"
Oct 30 19:03:21.985610 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:21.985581    4888 scope.go:122] "RemoveContainer" containerID="d345dd999a7f21095883da557949bd436a24bad41ffe6c2690ae14f8da2219cf"
Oct 30 19:03:21.986017 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:03:21.985959    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d345dd999a7f21095883da557949bd436a24bad41ffe6c2690ae14f8da2219cf\": container with ID starting with d345dd999a7f21095883da557949bd436a24bad41ffe6c2690ae14f8da2219cf not found: ID does not exist" containerID="d345dd999a7f21095883da557949bd436a24bad41ffe6c2690ae14f8da2219cf"
Oct 30 19:03:21.986123 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:21.986028    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d345dd999a7f21095883da557949bd436a24bad41ffe6c2690ae14f8da2219cf"} err="failed to get container status \"d345dd999a7f21095883da557949bd436a24bad41ffe6c2690ae14f8da2219cf\": rpc error: code = NotFound desc = could not find container \"d345dd999a7f21095883da557949bd436a24bad41ffe6c2690ae14f8da2219cf\": container with ID starting with d345dd999a7f21095883da557949bd436a24bad41ffe6c2690ae14f8da2219cf not found: ID does not exist"
Oct 30 19:03:21.987258 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:21.987240    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-nvlbr"
Oct 30 19:03:22.001847 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:22.001824    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-9k7xv"
Oct 30 19:03:22.019840 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:22.019810    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/427c5128-a124-4dd6-9748-a4dca0a85a4f-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/427c5128-a124-4dd6-9748-a4dca0a85a4f-istiod-ca-cert\") pod \"427c5128-a124-4dd6-9748-a4dca0a85a4f\" (UID: \"427c5128-a124-4dd6-9748-a4dca0a85a4f\") "
Oct 30 19:03:22.019984 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:22.019914    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/427c5128-a124-4dd6-9748-a4dca0a85a4f-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/427c5128-a124-4dd6-9748-a4dca0a85a4f-workload-certs\") pod \"427c5128-a124-4dd6-9748-a4dca0a85a4f\" (UID: \"427c5128-a124-4dd6-9748-a4dca0a85a4f\") "
Oct 30 19:03:22.019984 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:22.019954    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/687d01bd-f2ca-4fe3-8766-c8030df17da4-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/687d01bd-f2ca-4fe3-8766-c8030df17da4-istio-envoy\") pod \"687d01bd-f2ca-4fe3-8766-c8030df17da4\" (UID: \"687d01bd-f2ca-4fe3-8766-c8030df17da4\") "
Oct 30 19:03:22.020116 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:22.019990    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/427c5128-a124-4dd6-9748-a4dca0a85a4f-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/427c5128-a124-4dd6-9748-a4dca0a85a4f-istio-podinfo\") pod \"427c5128-a124-4dd6-9748-a4dca0a85a4f\" (UID: \"427c5128-a124-4dd6-9748-a4dca0a85a4f\") "
Oct 30 19:03:22.020116 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:22.020019    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/427c5128-a124-4dd6-9748-a4dca0a85a4f-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/427c5128-a124-4dd6-9748-a4dca0a85a4f-istio-envoy\") pod \"427c5128-a124-4dd6-9748-a4dca0a85a4f\" (UID: \"427c5128-a124-4dd6-9748-a4dca0a85a4f\") "
Oct 30 19:03:22.020116 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:22.020049    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/687d01bd-f2ca-4fe3-8766-c8030df17da4-kube-api-access-d5qcs\" (UniqueName: \"kubernetes.io/projected/687d01bd-f2ca-4fe3-8766-c8030df17da4-kube-api-access-d5qcs\") pod \"687d01bd-f2ca-4fe3-8766-c8030df17da4\" (UID: \"687d01bd-f2ca-4fe3-8766-c8030df17da4\") "
Oct 30 19:03:22.020116 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:22.020080    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/687d01bd-f2ca-4fe3-8766-c8030df17da4-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/687d01bd-f2ca-4fe3-8766-c8030df17da4-istio-data\") pod \"687d01bd-f2ca-4fe3-8766-c8030df17da4\" (UID: \"687d01bd-f2ca-4fe3-8766-c8030df17da4\") "
Oct 30 19:03:22.020356 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:22.020116    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/03fac346-2d8c-4d15-8e8a-c7e6571ad9b1-istio-token\" (UniqueName: \"kubernetes.io/projected/03fac346-2d8c-4d15-8e8a-c7e6571ad9b1-istio-token\") pod \"03fac346-2d8c-4d15-8e8a-c7e6571ad9b1\" (UID: \"03fac346-2d8c-4d15-8e8a-c7e6571ad9b1\") "
Oct 30 19:03:22.020356 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:22.020159    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/427c5128-a124-4dd6-9748-a4dca0a85a4f-kube-api-access-8nfjh\" (UniqueName: \"kubernetes.io/projected/427c5128-a124-4dd6-9748-a4dca0a85a4f-kube-api-access-8nfjh\") pod \"427c5128-a124-4dd6-9748-a4dca0a85a4f\" (UID: \"427c5128-a124-4dd6-9748-a4dca0a85a4f\") "
Oct 30 19:03:22.020356 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:22.020179    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/427c5128-a124-4dd6-9748-a4dca0a85a4f-istiod-ca-cert" pod "427c5128-a124-4dd6-9748-a4dca0a85a4f" (UID: "427c5128-a124-4dd6-9748-a4dca0a85a4f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:03:22.020356 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:22.020217    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/03fac346-2d8c-4d15-8e8a-c7e6571ad9b1-kube-api-access-msnpp\" (UniqueName: \"kubernetes.io/projected/03fac346-2d8c-4d15-8e8a-c7e6571ad9b1-kube-api-access-msnpp\") pod \"03fac346-2d8c-4d15-8e8a-c7e6571ad9b1\" (UID: \"03fac346-2d8c-4d15-8e8a-c7e6571ad9b1\") "
Oct 30 19:03:22.020356 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:22.020269    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/427c5128-a124-4dd6-9748-a4dca0a85a4f-istio-token\" (UniqueName: \"kubernetes.io/projected/427c5128-a124-4dd6-9748-a4dca0a85a4f-istio-token\") pod \"427c5128-a124-4dd6-9748-a4dca0a85a4f\" (UID: \"427c5128-a124-4dd6-9748-a4dca0a85a4f\") "
Oct 30 19:03:22.020356 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:22.020329    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/687d01bd-f2ca-4fe3-8766-c8030df17da4-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/687d01bd-f2ca-4fe3-8766-c8030df17da4-credential-socket\") pod \"687d01bd-f2ca-4fe3-8766-c8030df17da4\" (UID: \"687d01bd-f2ca-4fe3-8766-c8030df17da4\") "
Oct 30 19:03:22.020599 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:22.020383    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/03fac346-2d8c-4d15-8e8a-c7e6571ad9b1-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/03fac346-2d8c-4d15-8e8a-c7e6571ad9b1-istio-podinfo\") pod \"03fac346-2d8c-4d15-8e8a-c7e6571ad9b1\" (UID: \"03fac346-2d8c-4d15-8e8a-c7e6571ad9b1\") "
Oct 30 19:03:22.020599 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:22.020438    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/427c5128-a124-4dd6-9748-a4dca0a85a4f-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/427c5128-a124-4dd6-9748-a4dca0a85a4f-workload-socket\") pod \"427c5128-a124-4dd6-9748-a4dca0a85a4f\" (UID: \"427c5128-a124-4dd6-9748-a4dca0a85a4f\") "
Oct 30 19:03:22.020599 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:22.020468    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/427c5128-a124-4dd6-9748-a4dca0a85a4f-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/427c5128-a124-4dd6-9748-a4dca0a85a4f-credential-socket\") pod \"427c5128-a124-4dd6-9748-a4dca0a85a4f\" (UID: \"427c5128-a124-4dd6-9748-a4dca0a85a4f\") "
Oct 30 19:03:22.020599 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:22.020555    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/687d01bd-f2ca-4fe3-8766-c8030df17da4-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/687d01bd-f2ca-4fe3-8766-c8030df17da4-istiod-ca-cert\") pod \"687d01bd-f2ca-4fe3-8766-c8030df17da4\" (UID: \"687d01bd-f2ca-4fe3-8766-c8030df17da4\") "
Oct 30 19:03:22.020599 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:22.020590    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/03fac346-2d8c-4d15-8e8a-c7e6571ad9b1-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/03fac346-2d8c-4d15-8e8a-c7e6571ad9b1-workload-socket\") pod \"03fac346-2d8c-4d15-8e8a-c7e6571ad9b1\" (UID: \"03fac346-2d8c-4d15-8e8a-c7e6571ad9b1\") "
Oct 30 19:03:22.020752 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:22.020659    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/03fac346-2d8c-4d15-8e8a-c7e6571ad9b1-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/03fac346-2d8c-4d15-8e8a-c7e6571ad9b1-istio-data\") pod \"03fac346-2d8c-4d15-8e8a-c7e6571ad9b1\" (UID: \"03fac346-2d8c-4d15-8e8a-c7e6571ad9b1\") "
Oct 30 19:03:22.020752 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:22.020722    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/03fac346-2d8c-4d15-8e8a-c7e6571ad9b1-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/03fac346-2d8c-4d15-8e8a-c7e6571ad9b1-credential-socket\") pod \"03fac346-2d8c-4d15-8e8a-c7e6571ad9b1\" (UID: \"03fac346-2d8c-4d15-8e8a-c7e6571ad9b1\") "
Oct 30 19:03:22.020817 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:22.020755    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/687d01bd-f2ca-4fe3-8766-c8030df17da4-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/687d01bd-f2ca-4fe3-8766-c8030df17da4-workload-socket\") pod \"687d01bd-f2ca-4fe3-8766-c8030df17da4\" (UID: \"687d01bd-f2ca-4fe3-8766-c8030df17da4\") "
Oct 30 19:03:22.020817 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:22.020791    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/687d01bd-f2ca-4fe3-8766-c8030df17da4-istio-token\" (UniqueName: \"kubernetes.io/projected/687d01bd-f2ca-4fe3-8766-c8030df17da4-istio-token\") pod \"687d01bd-f2ca-4fe3-8766-c8030df17da4\" (UID: \"687d01bd-f2ca-4fe3-8766-c8030df17da4\") "
Oct 30 19:03:22.020880 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:22.020834    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/03fac346-2d8c-4d15-8e8a-c7e6571ad9b1-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/03fac346-2d8c-4d15-8e8a-c7e6571ad9b1-workload-certs\") pod \"03fac346-2d8c-4d15-8e8a-c7e6571ad9b1\" (UID: \"03fac346-2d8c-4d15-8e8a-c7e6571ad9b1\") "
Oct 30 19:03:22.020880 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:22.020869    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/427c5128-a124-4dd6-9748-a4dca0a85a4f-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/427c5128-a124-4dd6-9748-a4dca0a85a4f-istio-data\") pod \"427c5128-a124-4dd6-9748-a4dca0a85a4f\" (UID: \"427c5128-a124-4dd6-9748-a4dca0a85a4f\") "
Oct 30 19:03:22.020958 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:22.020922    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/03fac346-2d8c-4d15-8e8a-c7e6571ad9b1-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/03fac346-2d8c-4d15-8e8a-c7e6571ad9b1-istio-envoy\") pod \"03fac346-2d8c-4d15-8e8a-c7e6571ad9b1\" (UID: \"03fac346-2d8c-4d15-8e8a-c7e6571ad9b1\") "
Oct 30 19:03:22.021065 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:22.020987    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/03fac346-2d8c-4d15-8e8a-c7e6571ad9b1-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/03fac346-2d8c-4d15-8e8a-c7e6571ad9b1-istiod-ca-cert\") pod \"03fac346-2d8c-4d15-8e8a-c7e6571ad9b1\" (UID: \"03fac346-2d8c-4d15-8e8a-c7e6571ad9b1\") "
Oct 30 19:03:22.021212 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:22.021134    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/427c5128-a124-4dd6-9748-a4dca0a85a4f-workload-socket" pod "427c5128-a124-4dd6-9748-a4dca0a85a4f" (UID: "427c5128-a124-4dd6-9748-a4dca0a85a4f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:03:22.021769 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:22.021743    4888 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/427c5128-a124-4dd6-9748-a4dca0a85a4f-workload-socket\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:03:22.021870 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:22.021774    4888 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/427c5128-a124-4dd6-9748-a4dca0a85a4f-istiod-ca-cert\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:03:22.022841 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:22.021986    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/687d01bd-f2ca-4fe3-8766-c8030df17da4-credential-socket" pod "687d01bd-f2ca-4fe3-8766-c8030df17da4" (UID: "687d01bd-f2ca-4fe3-8766-c8030df17da4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:03:22.022841 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:22.022154    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/03fac346-2d8c-4d15-8e8a-c7e6571ad9b1-istiod-ca-cert" pod "03fac346-2d8c-4d15-8e8a-c7e6571ad9b1" (UID: "03fac346-2d8c-4d15-8e8a-c7e6571ad9b1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:03:22.022841 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:22.022662    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/427c5128-a124-4dd6-9748-a4dca0a85a4f-credential-socket" pod "427c5128-a124-4dd6-9748-a4dca0a85a4f" (UID: "427c5128-a124-4dd6-9748-a4dca0a85a4f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:03:22.023745 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:22.023586    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/03fac346-2d8c-4d15-8e8a-c7e6571ad9b1-workload-socket" pod "03fac346-2d8c-4d15-8e8a-c7e6571ad9b1" (UID: "03fac346-2d8c-4d15-8e8a-c7e6571ad9b1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:03:22.024168 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:22.023912    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/03fac346-2d8c-4d15-8e8a-c7e6571ad9b1-istio-data" pod "03fac346-2d8c-4d15-8e8a-c7e6571ad9b1" (UID: "03fac346-2d8c-4d15-8e8a-c7e6571ad9b1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:03:22.024168 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:22.023952    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/03fac346-2d8c-4d15-8e8a-c7e6571ad9b1-credential-socket" pod "03fac346-2d8c-4d15-8e8a-c7e6571ad9b1" (UID: "03fac346-2d8c-4d15-8e8a-c7e6571ad9b1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:03:22.024168 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:22.023983    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/687d01bd-f2ca-4fe3-8766-c8030df17da4-istiod-ca-cert" pod "687d01bd-f2ca-4fe3-8766-c8030df17da4" (UID: "687d01bd-f2ca-4fe3-8766-c8030df17da4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:03:22.024399 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:22.024227    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/03fac346-2d8c-4d15-8e8a-c7e6571ad9b1-workload-certs" pod "03fac346-2d8c-4d15-8e8a-c7e6571ad9b1" (UID: "03fac346-2d8c-4d15-8e8a-c7e6571ad9b1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:03:22.024806 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:22.024781    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/687d01bd-f2ca-4fe3-8766-c8030df17da4-workload-socket" pod "687d01bd-f2ca-4fe3-8766-c8030df17da4" (UID: "687d01bd-f2ca-4fe3-8766-c8030df17da4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:03:22.025892 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:22.025268    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/427c5128-a124-4dd6-9748-a4dca0a85a4f-istio-data" pod "427c5128-a124-4dd6-9748-a4dca0a85a4f" (UID: "427c5128-a124-4dd6-9748-a4dca0a85a4f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:03:22.025892 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:22.025700    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/687d01bd-f2ca-4fe3-8766-c8030df17da4-istio-data" pod "687d01bd-f2ca-4fe3-8766-c8030df17da4" (UID: "687d01bd-f2ca-4fe3-8766-c8030df17da4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:03:22.026159 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:22.026111    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/427c5128-a124-4dd6-9748-a4dca0a85a4f-workload-certs" pod "427c5128-a124-4dd6-9748-a4dca0a85a4f" (UID: "427c5128-a124-4dd6-9748-a4dca0a85a4f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:03:22.027563 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:22.027537    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/687d01bd-f2ca-4fe3-8766-c8030df17da4-kube-api-access-d5qcs" pod "687d01bd-f2ca-4fe3-8766-c8030df17da4" (UID: "687d01bd-f2ca-4fe3-8766-c8030df17da4"). InnerVolumeSpecName "kube-api-access-d5qcs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:03:22.028794 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:22.028757    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/03fac346-2d8c-4d15-8e8a-c7e6571ad9b1-istio-podinfo" pod "03fac346-2d8c-4d15-8e8a-c7e6571ad9b1" (UID: "03fac346-2d8c-4d15-8e8a-c7e6571ad9b1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:03:22.028883 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:22.028823    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/427c5128-a124-4dd6-9748-a4dca0a85a4f-kube-api-access-8nfjh" pod "427c5128-a124-4dd6-9748-a4dca0a85a4f" (UID: "427c5128-a124-4dd6-9748-a4dca0a85a4f"). InnerVolumeSpecName "kube-api-access-8nfjh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:03:22.029160 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:22.029075    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/03fac346-2d8c-4d15-8e8a-c7e6571ad9b1-kube-api-access-msnpp" pod "03fac346-2d8c-4d15-8e8a-c7e6571ad9b1" (UID: "03fac346-2d8c-4d15-8e8a-c7e6571ad9b1"). InnerVolumeSpecName "kube-api-access-msnpp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:03:22.029160 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:22.029110    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/03fac346-2d8c-4d15-8e8a-c7e6571ad9b1-istio-envoy" pod "03fac346-2d8c-4d15-8e8a-c7e6571ad9b1" (UID: "03fac346-2d8c-4d15-8e8a-c7e6571ad9b1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:03:22.029160 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:22.029148    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/427c5128-a124-4dd6-9748-a4dca0a85a4f-istio-envoy" pod "427c5128-a124-4dd6-9748-a4dca0a85a4f" (UID: "427c5128-a124-4dd6-9748-a4dca0a85a4f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:03:22.029160 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:22.029193    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/427c5128-a124-4dd6-9748-a4dca0a85a4f-istio-podinfo" pod "427c5128-a124-4dd6-9748-a4dca0a85a4f" (UID: "427c5128-a124-4dd6-9748-a4dca0a85a4f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:03:22.029966 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:22.029947    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/687d01bd-f2ca-4fe3-8766-c8030df17da4-istio-token" pod "687d01bd-f2ca-4fe3-8766-c8030df17da4" (UID: "687d01bd-f2ca-4fe3-8766-c8030df17da4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:03:22.030801 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:22.030781    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/03fac346-2d8c-4d15-8e8a-c7e6571ad9b1-istio-token" pod "03fac346-2d8c-4d15-8e8a-c7e6571ad9b1" (UID: "03fac346-2d8c-4d15-8e8a-c7e6571ad9b1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:03:22.030878 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:22.030848    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/687d01bd-f2ca-4fe3-8766-c8030df17da4-istio-envoy" pod "687d01bd-f2ca-4fe3-8766-c8030df17da4" (UID: "687d01bd-f2ca-4fe3-8766-c8030df17da4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:03:22.031150 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:22.031136    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/427c5128-a124-4dd6-9748-a4dca0a85a4f-istio-token" pod "427c5128-a124-4dd6-9748-a4dca0a85a4f" (UID: "427c5128-a124-4dd6-9748-a4dca0a85a4f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:03:22.124228 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:22.123994    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/687d01bd-f2ca-4fe3-8766-c8030df17da4-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/687d01bd-f2ca-4fe3-8766-c8030df17da4-workload-certs\") pod \"687d01bd-f2ca-4fe3-8766-c8030df17da4\" (UID: \"687d01bd-f2ca-4fe3-8766-c8030df17da4\") "
Oct 30 19:03:22.124228 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:22.124044    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/687d01bd-f2ca-4fe3-8766-c8030df17da4-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/687d01bd-f2ca-4fe3-8766-c8030df17da4-istio-podinfo\") pod \"687d01bd-f2ca-4fe3-8766-c8030df17da4\" (UID: \"687d01bd-f2ca-4fe3-8766-c8030df17da4\") "
Oct 30 19:03:22.128589 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:22.126863    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d5qcs\" (UniqueName: \"kubernetes.io/projected/687d01bd-f2ca-4fe3-8766-c8030df17da4-kube-api-access-d5qcs\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:03:22.128589 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:22.126894    4888 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/687d01bd-f2ca-4fe3-8766-c8030df17da4-istio-data\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:03:22.128589 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:22.126909    4888 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/03fac346-2d8c-4d15-8e8a-c7e6571ad9b1-istio-token\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:03:22.128589 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:22.126923    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8nfjh\" (UniqueName: \"kubernetes.io/projected/427c5128-a124-4dd6-9748-a4dca0a85a4f-kube-api-access-8nfjh\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:03:22.128589 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:22.126935    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-msnpp\" (UniqueName: \"kubernetes.io/projected/03fac346-2d8c-4d15-8e8a-c7e6571ad9b1-kube-api-access-msnpp\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:03:22.128589 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:22.126947    4888 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/427c5128-a124-4dd6-9748-a4dca0a85a4f-istio-token\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:03:22.128589 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:22.126961    4888 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/687d01bd-f2ca-4fe3-8766-c8030df17da4-credential-socket\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:03:22.128589 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:22.126974    4888 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/03fac346-2d8c-4d15-8e8a-c7e6571ad9b1-istio-podinfo\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:03:22.128589 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:22.126988    4888 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/427c5128-a124-4dd6-9748-a4dca0a85a4f-credential-socket\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:03:22.128589 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:22.127000    4888 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/687d01bd-f2ca-4fe3-8766-c8030df17da4-istiod-ca-cert\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:03:22.128589 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:22.127012    4888 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/03fac346-2d8c-4d15-8e8a-c7e6571ad9b1-workload-socket\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:03:22.128589 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:22.127026    4888 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/03fac346-2d8c-4d15-8e8a-c7e6571ad9b1-istio-data\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:03:22.128589 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:22.127039    4888 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/03fac346-2d8c-4d15-8e8a-c7e6571ad9b1-credential-socket\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:03:22.128589 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:22.127051    4888 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/687d01bd-f2ca-4fe3-8766-c8030df17da4-workload-socket\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:03:22.128589 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:22.127066    4888 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/687d01bd-f2ca-4fe3-8766-c8030df17da4-istio-token\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:03:22.128589 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:22.127079    4888 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/03fac346-2d8c-4d15-8e8a-c7e6571ad9b1-workload-certs\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:03:22.128589 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:22.127092    4888 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/427c5128-a124-4dd6-9748-a4dca0a85a4f-istio-data\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:03:22.128589 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:22.127103    4888 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/03fac346-2d8c-4d15-8e8a-c7e6571ad9b1-istio-envoy\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:03:22.128589 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:22.127117    4888 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/03fac346-2d8c-4d15-8e8a-c7e6571ad9b1-istiod-ca-cert\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:03:22.128589 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:22.127130    4888 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/427c5128-a124-4dd6-9748-a4dca0a85a4f-workload-certs\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:03:22.128589 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:22.127144    4888 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/687d01bd-f2ca-4fe3-8766-c8030df17da4-istio-envoy\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:03:22.128589 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:22.127156    4888 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/427c5128-a124-4dd6-9748-a4dca0a85a4f-istio-podinfo\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:03:22.128589 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:22.127170    4888 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/427c5128-a124-4dd6-9748-a4dca0a85a4f-istio-envoy\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:03:22.128589 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:22.127433    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/687d01bd-f2ca-4fe3-8766-c8030df17da4-workload-certs" pod "687d01bd-f2ca-4fe3-8766-c8030df17da4" (UID: "687d01bd-f2ca-4fe3-8766-c8030df17da4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:03:22.133744 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:22.133658    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/687d01bd-f2ca-4fe3-8766-c8030df17da4-istio-podinfo" pod "687d01bd-f2ca-4fe3-8766-c8030df17da4" (UID: "687d01bd-f2ca-4fe3-8766-c8030df17da4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:03:22.227904 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:22.227873    4888 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/687d01bd-f2ca-4fe3-8766-c8030df17da4-workload-certs\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:03:22.227904 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:22.227900    4888 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/687d01bd-f2ca-4fe3-8766-c8030df17da4-istio-podinfo\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:03:22.598336 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:22.596339    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-v2sdh"]
Oct 30 19:03:22.602310 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:22.602171    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-v2sdh"]
Oct 30 19:03:22.948050 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:22.947979    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-9k7xv"
Oct 30 19:03:22.948050 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:22.948019    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-9k7xv" event={"ID":"427c5128-a124-4dd6-9748-a4dca0a85a4f","Type":"ContainerDied","Data":"597061b1625dbe99704d710f120fd053a1f27013a69d0f79d325c36791974e24"}
Oct 30 19:03:22.948578 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:22.948082    4888 scope.go:122] "RemoveContainer" containerID="c5011ae29c57bbfe0a14858cc96b1fe1836fcf5bab80c336877452bfa7ab26b2"
Oct 30 19:03:22.949941 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:22.949924    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-nvlbr"
Oct 30 19:03:22.950062 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:22.949961    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-nvlbr" event={"ID":"03fac346-2d8c-4d15-8e8a-c7e6571ad9b1","Type":"ContainerDied","Data":"79618eac4c3da85208dac8a4e402b282d5b699785073696945e642bc02bd2e7c"}
Oct 30 19:03:22.959721 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:22.959695    4888 scope.go:122] "RemoveContainer" containerID="3e473820a56a96f987ec7165ff8d8938fdc360032470a99db092f1ebdbf31e8c"
Oct 30 19:03:23.166936 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:23.166906    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-9k7xv"]
Oct 30 19:03:23.181576 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:23.181499    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-9k7xv"]
Oct 30 19:03:23.247798 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:23.247728    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-nvlbr"]
Oct 30 19:03:23.259124 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:23.259098    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-66baf45c-openshift-default-974f9d7b9-nvlbr"]
Oct 30 19:03:23.891426 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:23.891395    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="03fac346-2d8c-4d15-8e8a-c7e6571ad9b1" path="/var/lib/kubelet/pods/03fac346-2d8c-4d15-8e8a-c7e6571ad9b1/volumes"
Oct 30 19:03:23.891864 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:23.891850    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="427c5128-a124-4dd6-9748-a4dca0a85a4f" path="/var/lib/kubelet/pods/427c5128-a124-4dd6-9748-a4dca0a85a4f/volumes"
Oct 30 19:03:23.892325 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:23.892276    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="687d01bd-f2ca-4fe3-8766-c8030df17da4" path="/var/lib/kubelet/pods/687d01bd-f2ca-4fe3-8766-c8030df17da4/volumes"
Oct 30 19:03:35.655990 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:03:35.655948    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-jbsw6" containerName="extractor"
Oct 30 19:03:43.967945 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:43.967906    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-rjmdz"]
Oct 30 19:03:56.490529 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:56.490496    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-7x4rb_382e8b31-a7fa-43f3-9f15-4c61627b0c04/console-operator/2.log"
Oct 30 19:03:56.491014 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:56.490498    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-7x4rb_382e8b31-a7fa-43f3-9f15-4c61627b0c04/console-operator/2.log"
Oct 30 19:03:56.500178 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:56.500149    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xfcdl_3f3fe764-2652-46c9-8e54-7882c0e6bcf1/ovn-acl-logging/0.log"
Oct 30 19:03:56.500368 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:56.500150    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xfcdl_3f3fe764-2652-46c9-8e54-7882c0e6bcf1/ovn-acl-logging/0.log"
Oct 30 19:03:56.577027 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:03:56.576988    4888 scope.go:122] "RemoveContainer" containerID="32cc03c5edf83adbe0b8f775a7c9700ef088fecb7e4ad976f6c80d88ef041e09"
Oct 30 19:04:05.618469 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:04:05.618434    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-jbsw6" containerName="extractor"
Oct 30 19:04:15.563537 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:04:15.561908    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-rjmdz"]
Oct 30 19:04:15.808724 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:04:15.808688    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-b-fee9f0-openshift-default-68c7499775-l68kt"]
Oct 30 19:04:15.809307 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:04:15.809269    4888 cpu_manager.go:414] "containerMap: removing container" podUID="998d1b1d-e656-49b0-8763-540ef647899d" containerName="maas-api"
Oct 30 19:04:15.809307 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:04:15.809300    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="998d1b1d-e656-49b0-8763-540ef647899d" containerName="maas-api"
Oct 30 19:04:15.809307 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:04:15.809314    4888 cpu_manager.go:414] "containerMap: removing container" podUID="427c5128-a124-4dd6-9748-a4dca0a85a4f" containerName="istio-proxy"
Oct 30 19:04:15.809487 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:04:15.809319    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="427c5128-a124-4dd6-9748-a4dca0a85a4f" containerName="istio-proxy"
Oct 30 19:04:15.809487 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:04:15.809331    4888 cpu_manager.go:414] "containerMap: removing container" podUID="687d01bd-f2ca-4fe3-8766-c8030df17da4" containerName="istio-proxy"
Oct 30 19:04:15.809487 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:04:15.809336    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="687d01bd-f2ca-4fe3-8766-c8030df17da4" containerName="istio-proxy"
Oct 30 19:04:15.809487 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:04:15.809350    4888 cpu_manager.go:414] "containerMap: removing container" podUID="03fac346-2d8c-4d15-8e8a-c7e6571ad9b1" containerName="istio-proxy"
Oct 30 19:04:15.809487 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:04:15.809355    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="03fac346-2d8c-4d15-8e8a-c7e6571ad9b1" containerName="istio-proxy"
Oct 30 19:04:15.809487 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:04:15.809366    4888 cpu_manager.go:414] "containerMap: removing container" podUID="427c5128-a124-4dd6-9748-a4dca0a85a4f" containerName="istio-proxy"
Oct 30 19:04:15.809487 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:04:15.809371    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="427c5128-a124-4dd6-9748-a4dca0a85a4f" containerName="istio-proxy"
Oct 30 19:04:15.809487 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:04:15.809385    4888 cpu_manager.go:414] "containerMap: removing container" podUID="863c528e-6d99-4867-bbff-02bdc504c923" containerName="istio-proxy"
Oct 30 19:04:15.809487 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:04:15.809389    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="863c528e-6d99-4867-bbff-02bdc504c923" containerName="istio-proxy"
Oct 30 19:04:15.809487 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:04:15.809407    4888 cpu_manager.go:414] "containerMap: removing container" podUID="3ba1c3a8-aa3d-45d6-bb89-c00b9993596e" containerName="maas-api"
Oct 30 19:04:15.809487 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:04:15.809414    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3ba1c3a8-aa3d-45d6-bb89-c00b9993596e" containerName="maas-api"
Oct 30 19:04:15.809804 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:04:15.809516    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="998d1b1d-e656-49b0-8763-540ef647899d" containerName="maas-api"
Oct 30 19:04:15.809804 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:04:15.809526    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="427c5128-a124-4dd6-9748-a4dca0a85a4f" containerName="istio-proxy"
Oct 30 19:04:15.809804 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:04:15.809534    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="687d01bd-f2ca-4fe3-8766-c8030df17da4" containerName="istio-proxy"
Oct 30 19:04:15.809804 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:04:15.809541    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="03fac346-2d8c-4d15-8e8a-c7e6571ad9b1" containerName="istio-proxy"
Oct 30 19:04:15.809804 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:04:15.809546    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="427c5128-a124-4dd6-9748-a4dca0a85a4f" containerName="istio-proxy"
Oct 30 19:04:15.809804 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:04:15.809554    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="863c528e-6d99-4867-bbff-02bdc504c923" containerName="istio-proxy"
Oct 30 19:04:15.809804 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:04:15.809561    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="3ba1c3a8-aa3d-45d6-bb89-c00b9993596e" containerName="maas-api"
Oct 30 19:04:15.812975 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:04:15.812953    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-fee9f0-openshift-default-68c7499775-l68kt"
Oct 30 19:04:15.860968 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:04:15.860934    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/94364f5b-4740-430f-87df-51e9b656d963-istio-podinfo\") pod \"e2e-ipp-b-fee9f0-openshift-default-68c7499775-l68kt\" (UID: \"94364f5b-4740-430f-87df-51e9b656d963\") " pod="openshift-ingress/e2e-ipp-b-fee9f0-openshift-default-68c7499775-l68kt"
Oct 30 19:04:15.861321 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:04:15.861299    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/94364f5b-4740-430f-87df-51e9b656d963-workload-socket\") pod \"e2e-ipp-b-fee9f0-openshift-default-68c7499775-l68kt\" (UID: \"94364f5b-4740-430f-87df-51e9b656d963\") " pod="openshift-ingress/e2e-ipp-b-fee9f0-openshift-default-68c7499775-l68kt"
Oct 30 19:04:15.861517 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:04:15.861502    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/94364f5b-4740-430f-87df-51e9b656d963-istio-data\") pod \"e2e-ipp-b-fee9f0-openshift-default-68c7499775-l68kt\" (UID: \"94364f5b-4740-430f-87df-51e9b656d963\") " pod="openshift-ingress/e2e-ipp-b-fee9f0-openshift-default-68c7499775-l68kt"
Oct 30 19:04:15.861750 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:04:15.861687    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/94364f5b-4740-430f-87df-51e9b656d963-istio-envoy\") pod \"e2e-ipp-b-fee9f0-openshift-default-68c7499775-l68kt\" (UID: \"94364f5b-4740-430f-87df-51e9b656d963\") " pod="openshift-ingress/e2e-ipp-b-fee9f0-openshift-default-68c7499775-l68kt"
Oct 30 19:04:15.861886 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:04:15.861870    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/94364f5b-4740-430f-87df-51e9b656d963-istio-token\") pod \"e2e-ipp-b-fee9f0-openshift-default-68c7499775-l68kt\" (UID: \"94364f5b-4740-430f-87df-51e9b656d963\") " pod="openshift-ingress/e2e-ipp-b-fee9f0-openshift-default-68c7499775-l68kt"
Oct 30 19:04:15.861988 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:04:15.861975    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4czzj\" (UniqueName: \"kubernetes.io/projected/94364f5b-4740-430f-87df-51e9b656d963-kube-api-access-4czzj\") pod \"e2e-ipp-b-fee9f0-openshift-default-68c7499775-l68kt\" (UID: \"94364f5b-4740-430f-87df-51e9b656d963\") " pod="openshift-ingress/e2e-ipp-b-fee9f0-openshift-default-68c7499775-l68kt"
Oct 30 19:04:15.862084 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:04:15.862071    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/94364f5b-4740-430f-87df-51e9b656d963-credential-socket\") pod \"e2e-ipp-b-fee9f0-openshift-default-68c7499775-l68kt\" (UID: \"94364f5b-4740-430f-87df-51e9b656d963\") " pod="openshift-ingress/e2e-ipp-b-fee9f0-openshift-default-68c7499775-l68kt"
Oct 30 19:04:15.862219 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:04:15.862207    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/94364f5b-4740-430f-87df-51e9b656d963-istiod-ca-cert\") pod \"e2e-ipp-b-fee9f0-openshift-default-68c7499775-l68kt\" (UID: \"94364f5b-4740-430f-87df-51e9b656d963\") " pod="openshift-ingress/e2e-ipp-b-fee9f0-openshift-default-68c7499775-l68kt"
Oct 30 19:04:15.862352 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:04:15.862337    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/94364f5b-4740-430f-87df-51e9b656d963-workload-certs\") pod \"e2e-ipp-b-fee9f0-openshift-default-68c7499775-l68kt\" (UID: \"94364f5b-4740-430f-87df-51e9b656d963\") " pod="openshift-ingress/e2e-ipp-b-fee9f0-openshift-default-68c7499775-l68kt"
Oct 30 19:04:15.866799 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:04:15.866775    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-b-fee9f0-openshift-default-dockercfg-vpc8h\""
Oct 30 19:04:15.921112 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:04:15.921078    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-fee9f0-openshift-default-68c7499775-l68kt"]
Oct 30 19:04:15.963863 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:04:15.963833    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/94364f5b-4740-430f-87df-51e9b656d963-istiod-ca-cert\") pod \"e2e-ipp-b-fee9f0-openshift-default-68c7499775-l68kt\" (UID: \"94364f5b-4740-430f-87df-51e9b656d963\") " pod="openshift-ingress/e2e-ipp-b-fee9f0-openshift-default-68c7499775-l68kt"
Oct 30 19:04:15.964063 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:04:15.963883    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/94364f5b-4740-430f-87df-51e9b656d963-workload-certs\") pod \"e2e-ipp-b-fee9f0-openshift-default-68c7499775-l68kt\" (UID: \"94364f5b-4740-430f-87df-51e9b656d963\") " pod="openshift-ingress/e2e-ipp-b-fee9f0-openshift-default-68c7499775-l68kt"
Oct 30 19:04:15.964264 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:04:15.964237    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/94364f5b-4740-430f-87df-51e9b656d963-istio-podinfo\") pod \"e2e-ipp-b-fee9f0-openshift-default-68c7499775-l68kt\" (UID: \"94364f5b-4740-430f-87df-51e9b656d963\") " pod="openshift-ingress/e2e-ipp-b-fee9f0-openshift-default-68c7499775-l68kt"
Oct 30 19:04:15.964402 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:04:15.964377    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/94364f5b-4740-430f-87df-51e9b656d963-workload-certs\") pod \"e2e-ipp-b-fee9f0-openshift-default-68c7499775-l68kt\" (UID: \"94364f5b-4740-430f-87df-51e9b656d963\") " pod="openshift-ingress/e2e-ipp-b-fee9f0-openshift-default-68c7499775-l68kt"
Oct 30 19:04:15.964684 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:04:15.964604    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/94364f5b-4740-430f-87df-51e9b656d963-workload-socket\") pod \"e2e-ipp-b-fee9f0-openshift-default-68c7499775-l68kt\" (UID: \"94364f5b-4740-430f-87df-51e9b656d963\") " pod="openshift-ingress/e2e-ipp-b-fee9f0-openshift-default-68c7499775-l68kt"
Oct 30 19:04:15.964684 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:04:15.964647    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/94364f5b-4740-430f-87df-51e9b656d963-istiod-ca-cert\") pod \"e2e-ipp-b-fee9f0-openshift-default-68c7499775-l68kt\" (UID: \"94364f5b-4740-430f-87df-51e9b656d963\") " pod="openshift-ingress/e2e-ipp-b-fee9f0-openshift-default-68c7499775-l68kt"
Oct 30 19:04:15.964684 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:04:15.964649    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/94364f5b-4740-430f-87df-51e9b656d963-istio-data\") pod \"e2e-ipp-b-fee9f0-openshift-default-68c7499775-l68kt\" (UID: \"94364f5b-4740-430f-87df-51e9b656d963\") " pod="openshift-ingress/e2e-ipp-b-fee9f0-openshift-default-68c7499775-l68kt"
Oct 30 19:04:15.964891 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:04:15.964713    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/94364f5b-4740-430f-87df-51e9b656d963-istio-envoy\") pod \"e2e-ipp-b-fee9f0-openshift-default-68c7499775-l68kt\" (UID: \"94364f5b-4740-430f-87df-51e9b656d963\") " pod="openshift-ingress/e2e-ipp-b-fee9f0-openshift-default-68c7499775-l68kt"
Oct 30 19:04:15.964891 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:04:15.964745    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/94364f5b-4740-430f-87df-51e9b656d963-istio-token\") pod \"e2e-ipp-b-fee9f0-openshift-default-68c7499775-l68kt\" (UID: \"94364f5b-4740-430f-87df-51e9b656d963\") " pod="openshift-ingress/e2e-ipp-b-fee9f0-openshift-default-68c7499775-l68kt"
Oct 30 19:04:15.965508 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:04:15.965417    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/94364f5b-4740-430f-87df-51e9b656d963-workload-socket\") pod \"e2e-ipp-b-fee9f0-openshift-default-68c7499775-l68kt\" (UID: \"94364f5b-4740-430f-87df-51e9b656d963\") " pod="openshift-ingress/e2e-ipp-b-fee9f0-openshift-default-68c7499775-l68kt"
Oct 30 19:04:15.965508 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:04:15.965451    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/94364f5b-4740-430f-87df-51e9b656d963-istio-data\") pod \"e2e-ipp-b-fee9f0-openshift-default-68c7499775-l68kt\" (UID: \"94364f5b-4740-430f-87df-51e9b656d963\") " pod="openshift-ingress/e2e-ipp-b-fee9f0-openshift-default-68c7499775-l68kt"
Oct 30 19:04:15.965647 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:04:15.965517    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4czzj\" (UniqueName: \"kubernetes.io/projected/94364f5b-4740-430f-87df-51e9b656d963-kube-api-access-4czzj\") pod \"e2e-ipp-b-fee9f0-openshift-default-68c7499775-l68kt\" (UID: \"94364f5b-4740-430f-87df-51e9b656d963\") " pod="openshift-ingress/e2e-ipp-b-fee9f0-openshift-default-68c7499775-l68kt"
Oct 30 19:04:15.965647 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:04:15.965582    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/94364f5b-4740-430f-87df-51e9b656d963-credential-socket\") pod \"e2e-ipp-b-fee9f0-openshift-default-68c7499775-l68kt\" (UID: \"94364f5b-4740-430f-87df-51e9b656d963\") " pod="openshift-ingress/e2e-ipp-b-fee9f0-openshift-default-68c7499775-l68kt"
Oct 30 19:04:15.966315 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:04:15.966053    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/94364f5b-4740-430f-87df-51e9b656d963-credential-socket\") pod \"e2e-ipp-b-fee9f0-openshift-default-68c7499775-l68kt\" (UID: \"94364f5b-4740-430f-87df-51e9b656d963\") " pod="openshift-ingress/e2e-ipp-b-fee9f0-openshift-default-68c7499775-l68kt"
Oct 30 19:04:15.980990 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:04:15.980618    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/94364f5b-4740-430f-87df-51e9b656d963-istio-envoy\") pod \"e2e-ipp-b-fee9f0-openshift-default-68c7499775-l68kt\" (UID: \"94364f5b-4740-430f-87df-51e9b656d963\") " pod="openshift-ingress/e2e-ipp-b-fee9f0-openshift-default-68c7499775-l68kt"
Oct 30 19:04:15.981826 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:04:15.981075    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/94364f5b-4740-430f-87df-51e9b656d963-istio-podinfo\") pod \"e2e-ipp-b-fee9f0-openshift-default-68c7499775-l68kt\" (UID: \"94364f5b-4740-430f-87df-51e9b656d963\") " pod="openshift-ingress/e2e-ipp-b-fee9f0-openshift-default-68c7499775-l68kt"
Oct 30 19:04:16.028620 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:04:16.028588    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4czzj\" (UniqueName: \"kubernetes.io/projected/94364f5b-4740-430f-87df-51e9b656d963-kube-api-access-4czzj\") pod \"e2e-ipp-b-fee9f0-openshift-default-68c7499775-l68kt\" (UID: \"94364f5b-4740-430f-87df-51e9b656d963\") " pod="openshift-ingress/e2e-ipp-b-fee9f0-openshift-default-68c7499775-l68kt"
Oct 30 19:04:16.028841 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:04:16.028739    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/94364f5b-4740-430f-87df-51e9b656d963-istio-token\") pod \"e2e-ipp-b-fee9f0-openshift-default-68c7499775-l68kt\" (UID: \"94364f5b-4740-430f-87df-51e9b656d963\") " pod="openshift-ingress/e2e-ipp-b-fee9f0-openshift-default-68c7499775-l68kt"
Oct 30 19:04:16.132311 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:04:16.131581    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-fee9f0-openshift-default-68c7499775-l68kt"
Oct 30 19:04:16.748939 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:04:16.748766    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-fee9f0-openshift-default-68c7499775-l68kt"]
Oct 30 19:04:16.755000 ip-10-0-134-46 kubenswrapper[4888]: W1030 19:04:16.754972    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod94364f5b_4740_430f_87df_51e9b656d963.slice/crio-1d83ecaa3b716617a420f72bbdf384299354d4240e20cbb82c7ffa1ea92dae35 WatchSource:0}: Error finding container 1d83ecaa3b716617a420f72bbdf384299354d4240e20cbb82c7ffa1ea92dae35: Status 404 returned error can't find the container with id 1d83ecaa3b716617a420f72bbdf384299354d4240e20cbb82c7ffa1ea92dae35
Oct 30 19:04:16.757195 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:04:16.757155    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:04:16.757346 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:04:16.757239    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:04:16.757346 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:04:16.757298    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:04:17.442999 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:04:17.442961    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-fee9f0-openshift-default-68c7499775-l68kt" event={"ID":"94364f5b-4740-430f-87df-51e9b656d963","Type":"ContainerStarted","Data":"fb5a4cc3729eff702853803d5f2bf01fbebe67656cad3e6f808e6fb6472e8448"}
Oct 30 19:04:17.443186 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:04:17.443018    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-fee9f0-openshift-default-68c7499775-l68kt" event={"ID":"94364f5b-4740-430f-87df-51e9b656d963","Type":"ContainerStarted","Data":"1d83ecaa3b716617a420f72bbdf384299354d4240e20cbb82c7ffa1ea92dae35"}
Oct 30 19:04:17.567539 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:04:17.567491    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-b-fee9f0-openshift-default-68c7499775-l68kt" podStartSLOduration=2.56747638 podStartE2EDuration="2.56747638s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:04:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:04:17.565856122 +0000 UTC m=+1821.813262148" watchObservedRunningTime="2026-10-30 19:04:17.56747638 +0000 UTC m=+1821.814882404"
Oct 30 19:04:18.133976 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:04:18.133928    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-fee9f0-openshift-default-68c7499775-l68kt"
Oct 30 19:04:18.154610 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:04:18.154583    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-fee9f0-openshift-default-68c7499775-l68kt"
Oct 30 19:04:18.449740 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:04:18.449666    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-fee9f0-openshift-default-68c7499775-l68kt"
Oct 30 19:04:18.450638 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:04:18.450619    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-fee9f0-openshift-default-68c7499775-l68kt"
Oct 30 19:04:24.142154 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:04:24.142120    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-fee9f0-5f799c94f9-msvnw"]
Oct 30 19:04:24.147631 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:04:24.147604    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-fee9f0-5f799c94f9-msvnw"
Oct 30 19:04:24.204249 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:04:24.204220    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-fee9f0-5f799c94f9-msvnw"]
Oct 30 19:04:24.250836 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:04:24.250805    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-fee9f0\""
Oct 30 19:04:24.251039 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:04:24.250878    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-fee9f0-dockercfg-j29k4\""
Oct 30 19:04:24.256518 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:04:24.256494    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rp7pw\" (UniqueName: \"kubernetes.io/projected/8317ca5e-a974-48a6-ad90-4c08cdd6bf57-kube-api-access-rp7pw\") pod \"payload-pre-processing-e2e-ipp-b-fee9f0-5f799c94f9-msvnw\" (UID: \"8317ca5e-a974-48a6-ad90-4c08cdd6bf57\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-fee9f0-5f799c94f9-msvnw"
Oct 30 19:04:24.256640 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:04:24.256619    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8317ca5e-a974-48a6-ad90-4c08cdd6bf57-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-fee9f0-5f799c94f9-msvnw\" (UID: \"8317ca5e-a974-48a6-ad90-4c08cdd6bf57\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-fee9f0-5f799c94f9-msvnw"
Oct 30 19:04:24.359894 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:04:24.359845    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rp7pw\" (UniqueName: \"kubernetes.io/projected/8317ca5e-a974-48a6-ad90-4c08cdd6bf57-kube-api-access-rp7pw\") pod \"payload-pre-processing-e2e-ipp-b-fee9f0-5f799c94f9-msvnw\" (UID: \"8317ca5e-a974-48a6-ad90-4c08cdd6bf57\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-fee9f0-5f799c94f9-msvnw"
Oct 30 19:04:24.366880 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:04:24.366842    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8317ca5e-a974-48a6-ad90-4c08cdd6bf57-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-fee9f0-5f799c94f9-msvnw\" (UID: \"8317ca5e-a974-48a6-ad90-4c08cdd6bf57\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-fee9f0-5f799c94f9-msvnw"
Oct 30 19:04:24.384315 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:04:24.381600    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8317ca5e-a974-48a6-ad90-4c08cdd6bf57-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-fee9f0-5f799c94f9-msvnw\" (UID: \"8317ca5e-a974-48a6-ad90-4c08cdd6bf57\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-fee9f0-5f799c94f9-msvnw"
Oct 30 19:04:24.464848 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:04:24.464707    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rp7pw\" (UniqueName: \"kubernetes.io/projected/8317ca5e-a974-48a6-ad90-4c08cdd6bf57-kube-api-access-rp7pw\") pod \"payload-pre-processing-e2e-ipp-b-fee9f0-5f799c94f9-msvnw\" (UID: \"8317ca5e-a974-48a6-ad90-4c08cdd6bf57\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-fee9f0-5f799c94f9-msvnw"
Oct 30 19:04:24.469474 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:04:24.469361    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-fee9f0-5f799c94f9-msvnw"
Oct 30 19:04:24.868785 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:04:24.868756    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-fee9f0-5f799c94f9-msvnw"]
Oct 30 19:04:24.868913 ip-10-0-134-46 kubenswrapper[4888]: W1030 19:04:24.868808    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8317ca5e_a974_48a6_ad90_4c08cdd6bf57.slice/crio-62c21d0d47c98243e45554cc6c5a990155fb58bbdc37ab7e94318cba51fea0a8 WatchSource:0}: Error finding container 62c21d0d47c98243e45554cc6c5a990155fb58bbdc37ab7e94318cba51fea0a8: Status 404 returned error can't find the container with id 62c21d0d47c98243e45554cc6c5a990155fb58bbdc37ab7e94318cba51fea0a8
Oct 30 19:04:25.503745 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:04:25.503712    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-fee9f0-5f799c94f9-msvnw" event={"ID":"8317ca5e-a974-48a6-ad90-4c08cdd6bf57","Type":"ContainerStarted","Data":"97766aff4ad125b43f5cb316986769b30060005de5c68a9e6f8f3dd833e4daf7"}
Oct 30 19:04:25.504200 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:04:25.503775    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-fee9f0-5f799c94f9-msvnw"
Oct 30 19:04:25.504200 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:04:25.503792    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-fee9f0-5f799c94f9-msvnw" event={"ID":"8317ca5e-a974-48a6-ad90-4c08cdd6bf57","Type":"ContainerStarted","Data":"62c21d0d47c98243e45554cc6c5a990155fb58bbdc37ab7e94318cba51fea0a8"}
Oct 30 19:04:25.624375 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:04:25.624320    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-fee9f0-5f799c94f9-msvnw" podStartSLOduration=1.624303573 podStartE2EDuration="1.624303573s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:04:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:04:25.62242228 +0000 UTC m=+1829.869828306" watchObservedRunningTime="2026-10-30 19:04:25.624303573 +0000 UTC m=+1829.871709593"
Oct 30 19:04:29.163704 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:04:29.163656    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-1acf9b-657dc7b5f-49dc7"]
Oct 30 19:04:29.170896 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:04:29.170854    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-1acf9b-657dc7b5f-49dc7"
Oct 30 19:04:29.219224 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:04:29.219194    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-1acf9b-657dc7b5f-49dc7"]
Oct 30 19:04:29.224772 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:04:29.224745    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-1acf9b-dockercfg-ltlhm\""
Oct 30 19:04:29.224963 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:04:29.224754    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-1acf9b\""
Oct 30 19:04:29.236181 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:04:29.236151    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8ff6e4b1-7681-4936-b30f-240ece473e12-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-1acf9b-657dc7b5f-49dc7\" (UID: \"8ff6e4b1-7681-4936-b30f-240ece473e12\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-1acf9b-657dc7b5f-49dc7"
Oct 30 19:04:29.236442 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:04:29.236345    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n2t25\" (UniqueName: \"kubernetes.io/projected/8ff6e4b1-7681-4936-b30f-240ece473e12-kube-api-access-n2t25\") pod \"payload-pre-processing-e2e-ipp-a-1acf9b-657dc7b5f-49dc7\" (UID: \"8ff6e4b1-7681-4936-b30f-240ece473e12\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-1acf9b-657dc7b5f-49dc7"
Oct 30 19:04:29.401445 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:04:29.401415    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8ff6e4b1-7681-4936-b30f-240ece473e12-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-1acf9b-657dc7b5f-49dc7\" (UID: \"8ff6e4b1-7681-4936-b30f-240ece473e12\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-1acf9b-657dc7b5f-49dc7"
Oct 30 19:04:29.401762 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:04:29.401743    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n2t25\" (UniqueName: \"kubernetes.io/projected/8ff6e4b1-7681-4936-b30f-240ece473e12-kube-api-access-n2t25\") pod \"payload-pre-processing-e2e-ipp-a-1acf9b-657dc7b5f-49dc7\" (UID: \"8ff6e4b1-7681-4936-b30f-240ece473e12\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-1acf9b-657dc7b5f-49dc7"
Oct 30 19:04:29.402865 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:04:29.402846    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8ff6e4b1-7681-4936-b30f-240ece473e12-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-1acf9b-657dc7b5f-49dc7\" (UID: \"8ff6e4b1-7681-4936-b30f-240ece473e12\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-1acf9b-657dc7b5f-49dc7"
Oct 30 19:04:29.466524 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:04:29.466441    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n2t25\" (UniqueName: \"kubernetes.io/projected/8ff6e4b1-7681-4936-b30f-240ece473e12-kube-api-access-n2t25\") pod \"payload-pre-processing-e2e-ipp-a-1acf9b-657dc7b5f-49dc7\" (UID: \"8ff6e4b1-7681-4936-b30f-240ece473e12\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-1acf9b-657dc7b5f-49dc7"
Oct 30 19:04:29.485150 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:04:29.485121    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-1acf9b-657dc7b5f-49dc7"
Oct 30 19:04:29.871007 ip-10-0-134-46 kubenswrapper[4888]: W1030 19:04:29.870963    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ff6e4b1_7681_4936_b30f_240ece473e12.slice/crio-5d463c695e95d9634c790473ba1b952f0b5c80bd9ac87968011fe534575869cc WatchSource:0}: Error finding container 5d463c695e95d9634c790473ba1b952f0b5c80bd9ac87968011fe534575869cc: Status 404 returned error can't find the container with id 5d463c695e95d9634c790473ba1b952f0b5c80bd9ac87968011fe534575869cc
Oct 30 19:04:29.871122 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:04:29.871017    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-1acf9b-657dc7b5f-49dc7"]
Oct 30 19:04:30.546752 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:04:30.546708    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-1acf9b-657dc7b5f-49dc7" event={"ID":"8ff6e4b1-7681-4936-b30f-240ece473e12","Type":"ContainerStarted","Data":"062c6759fcc1217ee30141be29322ebd3ffc7ec50197d50f7551ded91ca58de7"}
Oct 30 19:04:30.547273 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:04:30.546763    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-1acf9b-657dc7b5f-49dc7" event={"ID":"8ff6e4b1-7681-4936-b30f-240ece473e12","Type":"ContainerStarted","Data":"5d463c695e95d9634c790473ba1b952f0b5c80bd9ac87968011fe534575869cc"}
Oct 30 19:04:30.547273 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:04:30.546809    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-1acf9b-657dc7b5f-49dc7"
Oct 30 19:04:30.667982 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:04:30.667929    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-1acf9b-657dc7b5f-49dc7" podStartSLOduration=1.6679158410000001 podStartE2EDuration="1.667915841s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:04:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:04:30.664217223 +0000 UTC m=+1834.911623250" watchObservedRunningTime="2026-10-30 19:04:30.667915841 +0000 UTC m=+1834.915321872"
Oct 30 19:04:35.621589 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:04:35.621508    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-jbsw6" containerName="extractor"
Oct 30 19:04:36.515011 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:04:36.514978    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-fee9f0-5f799c94f9-msvnw"
Oct 30 19:04:40.897844 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:04:40.897801    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-rjmdz"]
Oct 30 19:04:41.557772 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:04:41.557740    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-1acf9b-657dc7b5f-49dc7"
Oct 30 19:05:01.334757 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:01.334720    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-6trhv"]
Oct 30 19:05:01.335428 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:01.335074    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-6trhv" podUID="509acd5d-92ed-4366-881e-3181be309e13" containerName="istio-proxy" containerID="cri-o://9f865bc4a0bfbe50ae203eda8a1a4f3dd3ea4695ea82c1003076c6676e818c84" gracePeriod=30
Oct 30 19:05:01.352096 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:01.352064    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-2h9dw"]
Oct 30 19:05:01.352382 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:01.352336    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-2h9dw" podUID="cea29cd6-99e6-4c4b-b232-6f2ca2ac5b91" containerName="istio-proxy" containerID="cri-o://c9d73471a5244580b90cb1dcb49bb798fa16f9da9dc49b4b8d969b46a17f00d0" gracePeriod=30
Oct 30 19:05:01.363549 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:01.363527    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-jlp7l"]
Oct 30 19:05:01.363712 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:01.363693    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-jlp7l" podUID="c9b213cb-4804-4bf8-a9d4-657bef3a11ba" containerName="istio-proxy" containerID="cri-o://e7b2ddb4be2750affca10dc6c0b3a60f7866a4330759d9d8c4e9793494e47803" gracePeriod=30
Oct 30 19:05:01.547800 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:01.547746    4888 patch_prober.go:29] interesting pod/e2e-api-b-4ebcd2-openshift-default-59d766c94c-6trhv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 19:05:01.548002 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:01.547814    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-6trhv" podUID="509acd5d-92ed-4366-881e-3181be309e13" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 19:05:01.827101 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:01.827068    4888 patch_prober.go:29] interesting pod/e2e-api-b-4ebcd2-openshift-default-59d766c94c-jlp7l container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 19:05:01.827274 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:01.827126    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-jlp7l" podUID="c9b213cb-4804-4bf8-a9d4-657bef3a11ba" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 19:05:01.876603 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:01.876567    4888 patch_prober.go:29] interesting pod/e2e-api-b-4ebcd2-openshift-default-59d766c94c-2h9dw container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 19:05:01.876759 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:01.876623    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-2h9dw" podUID="cea29cd6-99e6-4c4b-b232-6f2ca2ac5b91" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 19:05:03.170717 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:03.170679    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-a-0c9a2b-openshift-default-85695844fc-wsjcl"]
Oct 30 19:05:03.177228 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:03.177206    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-0c9a2b-openshift-default-85695844fc-wsjcl"
Oct 30 19:05:03.242018 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:03.241975    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-0c9a2b-openshift-default-85695844fc-wsjcl"]
Oct 30 19:05:03.246096 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:03.246071    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-a-0c9a2b-openshift-default-dockercfg-tbm95\""
Oct 30 19:05:03.350365 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:03.345936    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3df2a357-4896-4c99-831b-96fe4cd18fb5-istio-envoy\") pod \"e2e-ipp-a-0c9a2b-openshift-default-85695844fc-wsjcl\" (UID: \"3df2a357-4896-4c99-831b-96fe4cd18fb5\") " pod="openshift-ingress/e2e-ipp-a-0c9a2b-openshift-default-85695844fc-wsjcl"
Oct 30 19:05:03.350365 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:03.346005    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3df2a357-4896-4c99-831b-96fe4cd18fb5-istiod-ca-cert\") pod \"e2e-ipp-a-0c9a2b-openshift-default-85695844fc-wsjcl\" (UID: \"3df2a357-4896-4c99-831b-96fe4cd18fb5\") " pod="openshift-ingress/e2e-ipp-a-0c9a2b-openshift-default-85695844fc-wsjcl"
Oct 30 19:05:03.350365 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:03.346034    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3df2a357-4896-4c99-831b-96fe4cd18fb5-istio-data\") pod \"e2e-ipp-a-0c9a2b-openshift-default-85695844fc-wsjcl\" (UID: \"3df2a357-4896-4c99-831b-96fe4cd18fb5\") " pod="openshift-ingress/e2e-ipp-a-0c9a2b-openshift-default-85695844fc-wsjcl"
Oct 30 19:05:03.350365 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:03.346071    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3df2a357-4896-4c99-831b-96fe4cd18fb5-workload-certs\") pod \"e2e-ipp-a-0c9a2b-openshift-default-85695844fc-wsjcl\" (UID: \"3df2a357-4896-4c99-831b-96fe4cd18fb5\") " pod="openshift-ingress/e2e-ipp-a-0c9a2b-openshift-default-85695844fc-wsjcl"
Oct 30 19:05:03.350365 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:03.346126    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3df2a357-4896-4c99-831b-96fe4cd18fb5-credential-socket\") pod \"e2e-ipp-a-0c9a2b-openshift-default-85695844fc-wsjcl\" (UID: \"3df2a357-4896-4c99-831b-96fe4cd18fb5\") " pod="openshift-ingress/e2e-ipp-a-0c9a2b-openshift-default-85695844fc-wsjcl"
Oct 30 19:05:03.350365 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:03.346209    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r9m7n\" (UniqueName: \"kubernetes.io/projected/3df2a357-4896-4c99-831b-96fe4cd18fb5-kube-api-access-r9m7n\") pod \"e2e-ipp-a-0c9a2b-openshift-default-85695844fc-wsjcl\" (UID: \"3df2a357-4896-4c99-831b-96fe4cd18fb5\") " pod="openshift-ingress/e2e-ipp-a-0c9a2b-openshift-default-85695844fc-wsjcl"
Oct 30 19:05:03.350365 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:03.346260    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3df2a357-4896-4c99-831b-96fe4cd18fb5-workload-socket\") pod \"e2e-ipp-a-0c9a2b-openshift-default-85695844fc-wsjcl\" (UID: \"3df2a357-4896-4c99-831b-96fe4cd18fb5\") " pod="openshift-ingress/e2e-ipp-a-0c9a2b-openshift-default-85695844fc-wsjcl"
Oct 30 19:05:03.350365 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:03.346301    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3df2a357-4896-4c99-831b-96fe4cd18fb5-istio-token\") pod \"e2e-ipp-a-0c9a2b-openshift-default-85695844fc-wsjcl\" (UID: \"3df2a357-4896-4c99-831b-96fe4cd18fb5\") " pod="openshift-ingress/e2e-ipp-a-0c9a2b-openshift-default-85695844fc-wsjcl"
Oct 30 19:05:03.350365 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:03.346388    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3df2a357-4896-4c99-831b-96fe4cd18fb5-istio-podinfo\") pod \"e2e-ipp-a-0c9a2b-openshift-default-85695844fc-wsjcl\" (UID: \"3df2a357-4896-4c99-831b-96fe4cd18fb5\") " pod="openshift-ingress/e2e-ipp-a-0c9a2b-openshift-default-85695844fc-wsjcl"
Oct 30 19:05:03.448000 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:03.447891    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r9m7n\" (UniqueName: \"kubernetes.io/projected/3df2a357-4896-4c99-831b-96fe4cd18fb5-kube-api-access-r9m7n\") pod \"e2e-ipp-a-0c9a2b-openshift-default-85695844fc-wsjcl\" (UID: \"3df2a357-4896-4c99-831b-96fe4cd18fb5\") " pod="openshift-ingress/e2e-ipp-a-0c9a2b-openshift-default-85695844fc-wsjcl"
Oct 30 19:05:03.448000 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:03.447967    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3df2a357-4896-4c99-831b-96fe4cd18fb5-workload-socket\") pod \"e2e-ipp-a-0c9a2b-openshift-default-85695844fc-wsjcl\" (UID: \"3df2a357-4896-4c99-831b-96fe4cd18fb5\") " pod="openshift-ingress/e2e-ipp-a-0c9a2b-openshift-default-85695844fc-wsjcl"
Oct 30 19:05:03.448000 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:03.447994    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3df2a357-4896-4c99-831b-96fe4cd18fb5-istio-token\") pod \"e2e-ipp-a-0c9a2b-openshift-default-85695844fc-wsjcl\" (UID: \"3df2a357-4896-4c99-831b-96fe4cd18fb5\") " pod="openshift-ingress/e2e-ipp-a-0c9a2b-openshift-default-85695844fc-wsjcl"
Oct 30 19:05:03.448373 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:03.448064    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3df2a357-4896-4c99-831b-96fe4cd18fb5-istio-podinfo\") pod \"e2e-ipp-a-0c9a2b-openshift-default-85695844fc-wsjcl\" (UID: \"3df2a357-4896-4c99-831b-96fe4cd18fb5\") " pod="openshift-ingress/e2e-ipp-a-0c9a2b-openshift-default-85695844fc-wsjcl"
Oct 30 19:05:03.448373 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:03.448179    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3df2a357-4896-4c99-831b-96fe4cd18fb5-istio-envoy\") pod \"e2e-ipp-a-0c9a2b-openshift-default-85695844fc-wsjcl\" (UID: \"3df2a357-4896-4c99-831b-96fe4cd18fb5\") " pod="openshift-ingress/e2e-ipp-a-0c9a2b-openshift-default-85695844fc-wsjcl"
Oct 30 19:05:03.448373 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:03.448222    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3df2a357-4896-4c99-831b-96fe4cd18fb5-istiod-ca-cert\") pod \"e2e-ipp-a-0c9a2b-openshift-default-85695844fc-wsjcl\" (UID: \"3df2a357-4896-4c99-831b-96fe4cd18fb5\") " pod="openshift-ingress/e2e-ipp-a-0c9a2b-openshift-default-85695844fc-wsjcl"
Oct 30 19:05:03.448373 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:03.448254    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3df2a357-4896-4c99-831b-96fe4cd18fb5-istio-data\") pod \"e2e-ipp-a-0c9a2b-openshift-default-85695844fc-wsjcl\" (UID: \"3df2a357-4896-4c99-831b-96fe4cd18fb5\") " pod="openshift-ingress/e2e-ipp-a-0c9a2b-openshift-default-85695844fc-wsjcl"
Oct 30 19:05:03.448373 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:03.448315    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3df2a357-4896-4c99-831b-96fe4cd18fb5-workload-certs\") pod \"e2e-ipp-a-0c9a2b-openshift-default-85695844fc-wsjcl\" (UID: \"3df2a357-4896-4c99-831b-96fe4cd18fb5\") " pod="openshift-ingress/e2e-ipp-a-0c9a2b-openshift-default-85695844fc-wsjcl"
Oct 30 19:05:03.448628 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:03.448376    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3df2a357-4896-4c99-831b-96fe4cd18fb5-credential-socket\") pod \"e2e-ipp-a-0c9a2b-openshift-default-85695844fc-wsjcl\" (UID: \"3df2a357-4896-4c99-831b-96fe4cd18fb5\") " pod="openshift-ingress/e2e-ipp-a-0c9a2b-openshift-default-85695844fc-wsjcl"
Oct 30 19:05:03.448628 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:03.448440    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3df2a357-4896-4c99-831b-96fe4cd18fb5-workload-socket\") pod \"e2e-ipp-a-0c9a2b-openshift-default-85695844fc-wsjcl\" (UID: \"3df2a357-4896-4c99-831b-96fe4cd18fb5\") " pod="openshift-ingress/e2e-ipp-a-0c9a2b-openshift-default-85695844fc-wsjcl"
Oct 30 19:05:03.448724 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:03.448699    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3df2a357-4896-4c99-831b-96fe4cd18fb5-istio-data\") pod \"e2e-ipp-a-0c9a2b-openshift-default-85695844fc-wsjcl\" (UID: \"3df2a357-4896-4c99-831b-96fe4cd18fb5\") " pod="openshift-ingress/e2e-ipp-a-0c9a2b-openshift-default-85695844fc-wsjcl"
Oct 30 19:05:03.448780 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:03.448751    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3df2a357-4896-4c99-831b-96fe4cd18fb5-workload-certs\") pod \"e2e-ipp-a-0c9a2b-openshift-default-85695844fc-wsjcl\" (UID: \"3df2a357-4896-4c99-831b-96fe4cd18fb5\") " pod="openshift-ingress/e2e-ipp-a-0c9a2b-openshift-default-85695844fc-wsjcl"
Oct 30 19:05:03.448831 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:03.448785    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3df2a357-4896-4c99-831b-96fe4cd18fb5-credential-socket\") pod \"e2e-ipp-a-0c9a2b-openshift-default-85695844fc-wsjcl\" (UID: \"3df2a357-4896-4c99-831b-96fe4cd18fb5\") " pod="openshift-ingress/e2e-ipp-a-0c9a2b-openshift-default-85695844fc-wsjcl"
Oct 30 19:05:03.449331 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:03.449261    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3df2a357-4896-4c99-831b-96fe4cd18fb5-istiod-ca-cert\") pod \"e2e-ipp-a-0c9a2b-openshift-default-85695844fc-wsjcl\" (UID: \"3df2a357-4896-4c99-831b-96fe4cd18fb5\") " pod="openshift-ingress/e2e-ipp-a-0c9a2b-openshift-default-85695844fc-wsjcl"
Oct 30 19:05:03.450842 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:03.450814    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3df2a357-4896-4c99-831b-96fe4cd18fb5-istio-envoy\") pod \"e2e-ipp-a-0c9a2b-openshift-default-85695844fc-wsjcl\" (UID: \"3df2a357-4896-4c99-831b-96fe4cd18fb5\") " pod="openshift-ingress/e2e-ipp-a-0c9a2b-openshift-default-85695844fc-wsjcl"
Oct 30 19:05:03.451062 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:03.451045    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3df2a357-4896-4c99-831b-96fe4cd18fb5-istio-podinfo\") pod \"e2e-ipp-a-0c9a2b-openshift-default-85695844fc-wsjcl\" (UID: \"3df2a357-4896-4c99-831b-96fe4cd18fb5\") " pod="openshift-ingress/e2e-ipp-a-0c9a2b-openshift-default-85695844fc-wsjcl"
Oct 30 19:05:03.511225 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:03.511193    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3df2a357-4896-4c99-831b-96fe4cd18fb5-istio-token\") pod \"e2e-ipp-a-0c9a2b-openshift-default-85695844fc-wsjcl\" (UID: \"3df2a357-4896-4c99-831b-96fe4cd18fb5\") " pod="openshift-ingress/e2e-ipp-a-0c9a2b-openshift-default-85695844fc-wsjcl"
Oct 30 19:05:03.511561 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:03.511533    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r9m7n\" (UniqueName: \"kubernetes.io/projected/3df2a357-4896-4c99-831b-96fe4cd18fb5-kube-api-access-r9m7n\") pod \"e2e-ipp-a-0c9a2b-openshift-default-85695844fc-wsjcl\" (UID: \"3df2a357-4896-4c99-831b-96fe4cd18fb5\") " pod="openshift-ingress/e2e-ipp-a-0c9a2b-openshift-default-85695844fc-wsjcl"
Oct 30 19:05:03.805442 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:03.805341    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-0c9a2b-openshift-default-85695844fc-wsjcl"
Oct 30 19:05:04.258556 ip-10-0-134-46 kubenswrapper[4888]: W1030 19:05:04.258529    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3df2a357_4896_4c99_831b_96fe4cd18fb5.slice/crio-70aadca04473b75faa967ba029c393237997501d6ebae28d53bb2e3cf44195bc WatchSource:0}: Error finding container 70aadca04473b75faa967ba029c393237997501d6ebae28d53bb2e3cf44195bc: Status 404 returned error can't find the container with id 70aadca04473b75faa967ba029c393237997501d6ebae28d53bb2e3cf44195bc
Oct 30 19:05:04.259081 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:04.259058    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-0c9a2b-openshift-default-85695844fc-wsjcl"]
Oct 30 19:05:04.261013 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:04.260977    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:05:04.261125 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:04.261054    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:05:04.261125 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:04.261100    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:05:04.900412 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:04.900369    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-0c9a2b-openshift-default-85695844fc-wsjcl" event={"ID":"3df2a357-4896-4c99-831b-96fe4cd18fb5","Type":"ContainerStarted","Data":"503b82a28a2b628b8837da4256013f10eb9afed70a785e50e74b1daa6ae0a8fe"}
Oct 30 19:05:04.900625 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:04.900436    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-0c9a2b-openshift-default-85695844fc-wsjcl" event={"ID":"3df2a357-4896-4c99-831b-96fe4cd18fb5","Type":"ContainerStarted","Data":"70aadca04473b75faa967ba029c393237997501d6ebae28d53bb2e3cf44195bc"}
Oct 30 19:05:05.130492 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:05.130437    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-a-0c9a2b-openshift-default-85695844fc-wsjcl" podStartSLOduration=2.130422658 podStartE2EDuration="2.130422658s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:05:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:05:05.077787993 +0000 UTC m=+1869.325194034" watchObservedRunningTime="2026-10-30 19:05:05.130422658 +0000 UTC m=+1869.377828684"
Oct 30 19:05:05.624132 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:05:05.624091    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-jbsw6" containerName="extractor"
Oct 30 19:05:05.805946 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:05.805910    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-0c9a2b-openshift-default-85695844fc-wsjcl"
Oct 30 19:05:05.813274 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:05.813246    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-0c9a2b-openshift-default-85695844fc-wsjcl"
Oct 30 19:05:05.908938 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:05.908852    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-0c9a2b-openshift-default-85695844fc-wsjcl"
Oct 30 19:05:05.909927 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:05.909901    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-0c9a2b-openshift-default-85695844fc-wsjcl"
Oct 30 19:05:06.704531 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:06.704505    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-2h9dw"
Oct 30 19:05:06.723426 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:06.723393    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-6trhv"
Oct 30 19:05:06.740106 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:06.740087    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-jlp7l"
Oct 30 19:05:06.795389 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:06.795357    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/509acd5d-92ed-4366-881e-3181be309e13-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/509acd5d-92ed-4366-881e-3181be309e13-workload-certs\") pod \"509acd5d-92ed-4366-881e-3181be309e13\" (UID: \"509acd5d-92ed-4366-881e-3181be309e13\") "
Oct 30 19:05:06.795545 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:06.795473    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/509acd5d-92ed-4366-881e-3181be309e13-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/509acd5d-92ed-4366-881e-3181be309e13-istiod-ca-cert\") pod \"509acd5d-92ed-4366-881e-3181be309e13\" (UID: \"509acd5d-92ed-4366-881e-3181be309e13\") "
Oct 30 19:05:06.795545 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:06.795505    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/509acd5d-92ed-4366-881e-3181be309e13-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/509acd5d-92ed-4366-881e-3181be309e13-istio-podinfo\") pod \"509acd5d-92ed-4366-881e-3181be309e13\" (UID: \"509acd5d-92ed-4366-881e-3181be309e13\") "
Oct 30 19:05:06.795638 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:06.795548    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cea29cd6-99e6-4c4b-b232-6f2ca2ac5b91-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cea29cd6-99e6-4c4b-b232-6f2ca2ac5b91-workload-certs\") pod \"cea29cd6-99e6-4c4b-b232-6f2ca2ac5b91\" (UID: \"cea29cd6-99e6-4c4b-b232-6f2ca2ac5b91\") "
Oct 30 19:05:06.795638 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:06.795573    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cea29cd6-99e6-4c4b-b232-6f2ca2ac5b91-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cea29cd6-99e6-4c4b-b232-6f2ca2ac5b91-istio-envoy\") pod \"cea29cd6-99e6-4c4b-b232-6f2ca2ac5b91\" (UID: \"cea29cd6-99e6-4c4b-b232-6f2ca2ac5b91\") "
Oct 30 19:05:06.795638 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:06.795610    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/509acd5d-92ed-4366-881e-3181be309e13-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/509acd5d-92ed-4366-881e-3181be309e13-istio-envoy\") pod \"509acd5d-92ed-4366-881e-3181be309e13\" (UID: \"509acd5d-92ed-4366-881e-3181be309e13\") "
Oct 30 19:05:06.795808 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:06.795636    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cea29cd6-99e6-4c4b-b232-6f2ca2ac5b91-istio-token\" (UniqueName: \"kubernetes.io/projected/cea29cd6-99e6-4c4b-b232-6f2ca2ac5b91-istio-token\") pod \"cea29cd6-99e6-4c4b-b232-6f2ca2ac5b91\" (UID: \"cea29cd6-99e6-4c4b-b232-6f2ca2ac5b91\") "
Oct 30 19:05:06.795808 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:06.795686    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cea29cd6-99e6-4c4b-b232-6f2ca2ac5b91-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cea29cd6-99e6-4c4b-b232-6f2ca2ac5b91-credential-socket\") pod \"cea29cd6-99e6-4c4b-b232-6f2ca2ac5b91\" (UID: \"cea29cd6-99e6-4c4b-b232-6f2ca2ac5b91\") "
Oct 30 19:05:06.795808 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:06.795721    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/509acd5d-92ed-4366-881e-3181be309e13-kube-api-access-c9c89\" (UniqueName: \"kubernetes.io/projected/509acd5d-92ed-4366-881e-3181be309e13-kube-api-access-c9c89\") pod \"509acd5d-92ed-4366-881e-3181be309e13\" (UID: \"509acd5d-92ed-4366-881e-3181be309e13\") "
Oct 30 19:05:06.795808 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:06.795755    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cea29cd6-99e6-4c4b-b232-6f2ca2ac5b91-kube-api-access-gnjn2\" (UniqueName: \"kubernetes.io/projected/cea29cd6-99e6-4c4b-b232-6f2ca2ac5b91-kube-api-access-gnjn2\") pod \"cea29cd6-99e6-4c4b-b232-6f2ca2ac5b91\" (UID: \"cea29cd6-99e6-4c4b-b232-6f2ca2ac5b91\") "
Oct 30 19:05:06.795808 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:06.795795    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/509acd5d-92ed-4366-881e-3181be309e13-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/509acd5d-92ed-4366-881e-3181be309e13-istio-data\") pod \"509acd5d-92ed-4366-881e-3181be309e13\" (UID: \"509acd5d-92ed-4366-881e-3181be309e13\") "
Oct 30 19:05:06.796073 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:06.795853    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/509acd5d-92ed-4366-881e-3181be309e13-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/509acd5d-92ed-4366-881e-3181be309e13-credential-socket\") pod \"509acd5d-92ed-4366-881e-3181be309e13\" (UID: \"509acd5d-92ed-4366-881e-3181be309e13\") "
Oct 30 19:05:06.796073 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:06.795902    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/509acd5d-92ed-4366-881e-3181be309e13-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/509acd5d-92ed-4366-881e-3181be309e13-workload-socket\") pod \"509acd5d-92ed-4366-881e-3181be309e13\" (UID: \"509acd5d-92ed-4366-881e-3181be309e13\") "
Oct 30 19:05:06.796073 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:06.795950    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/cea29cd6-99e6-4c4b-b232-6f2ca2ac5b91-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cea29cd6-99e6-4c4b-b232-6f2ca2ac5b91-istio-podinfo\") pod \"cea29cd6-99e6-4c4b-b232-6f2ca2ac5b91\" (UID: \"cea29cd6-99e6-4c4b-b232-6f2ca2ac5b91\") "
Oct 30 19:05:06.796073 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:06.795998    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/cea29cd6-99e6-4c4b-b232-6f2ca2ac5b91-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cea29cd6-99e6-4c4b-b232-6f2ca2ac5b91-istiod-ca-cert\") pod \"cea29cd6-99e6-4c4b-b232-6f2ca2ac5b91\" (UID: \"cea29cd6-99e6-4c4b-b232-6f2ca2ac5b91\") "
Oct 30 19:05:06.796313 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:06.796085    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/509acd5d-92ed-4366-881e-3181be309e13-istio-token\" (UniqueName: \"kubernetes.io/projected/509acd5d-92ed-4366-881e-3181be309e13-istio-token\") pod \"509acd5d-92ed-4366-881e-3181be309e13\" (UID: \"509acd5d-92ed-4366-881e-3181be309e13\") "
Oct 30 19:05:06.796313 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:06.796131    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cea29cd6-99e6-4c4b-b232-6f2ca2ac5b91-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cea29cd6-99e6-4c4b-b232-6f2ca2ac5b91-workload-socket\") pod \"cea29cd6-99e6-4c4b-b232-6f2ca2ac5b91\" (UID: \"cea29cd6-99e6-4c4b-b232-6f2ca2ac5b91\") "
Oct 30 19:05:06.796313 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:06.796163    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cea29cd6-99e6-4c4b-b232-6f2ca2ac5b91-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cea29cd6-99e6-4c4b-b232-6f2ca2ac5b91-istio-data\") pod \"cea29cd6-99e6-4c4b-b232-6f2ca2ac5b91\" (UID: \"cea29cd6-99e6-4c4b-b232-6f2ca2ac5b91\") "
Oct 30 19:05:06.796313 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:06.795681    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/509acd5d-92ed-4366-881e-3181be309e13-workload-certs" pod "509acd5d-92ed-4366-881e-3181be309e13" (UID: "509acd5d-92ed-4366-881e-3181be309e13"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:05:06.796313 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:06.795901    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/509acd5d-92ed-4366-881e-3181be309e13-istiod-ca-cert" pod "509acd5d-92ed-4366-881e-3181be309e13" (UID: "509acd5d-92ed-4366-881e-3181be309e13"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:05:06.796313 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:06.796167    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/509acd5d-92ed-4366-881e-3181be309e13-istio-data" pod "509acd5d-92ed-4366-881e-3181be309e13" (UID: "509acd5d-92ed-4366-881e-3181be309e13"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:05:06.796643 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:06.796563    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cea29cd6-99e6-4c4b-b232-6f2ca2ac5b91-credential-socket" pod "cea29cd6-99e6-4c4b-b232-6f2ca2ac5b91" (UID: "cea29cd6-99e6-4c4b-b232-6f2ca2ac5b91"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:05:06.796853 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:06.796756    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cea29cd6-99e6-4c4b-b232-6f2ca2ac5b91-workload-certs" pod "cea29cd6-99e6-4c4b-b232-6f2ca2ac5b91" (UID: "cea29cd6-99e6-4c4b-b232-6f2ca2ac5b91"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:05:06.797066 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:06.796964    4888 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/509acd5d-92ed-4366-881e-3181be309e13-istio-data\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:05:06.797066 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:06.796974    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/509acd5d-92ed-4366-881e-3181be309e13-credential-socket" pod "509acd5d-92ed-4366-881e-3181be309e13" (UID: "509acd5d-92ed-4366-881e-3181be309e13"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:05:06.797066 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:06.796987    4888 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/509acd5d-92ed-4366-881e-3181be309e13-workload-certs\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:05:06.797066 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:06.797004    4888 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/509acd5d-92ed-4366-881e-3181be309e13-istiod-ca-cert\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:05:06.797066 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:06.797018    4888 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cea29cd6-99e6-4c4b-b232-6f2ca2ac5b91-credential-socket\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:05:06.797388 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:06.797217    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/509acd5d-92ed-4366-881e-3181be309e13-workload-socket" pod "509acd5d-92ed-4366-881e-3181be309e13" (UID: "509acd5d-92ed-4366-881e-3181be309e13"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:05:06.798031 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:06.797606    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cea29cd6-99e6-4c4b-b232-6f2ca2ac5b91-istiod-ca-cert" pod "cea29cd6-99e6-4c4b-b232-6f2ca2ac5b91" (UID: "cea29cd6-99e6-4c4b-b232-6f2ca2ac5b91"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:05:06.798462 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:06.798436    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/509acd5d-92ed-4366-881e-3181be309e13-istio-podinfo" pod "509acd5d-92ed-4366-881e-3181be309e13" (UID: "509acd5d-92ed-4366-881e-3181be309e13"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:05:06.798960 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:06.798884    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cea29cd6-99e6-4c4b-b232-6f2ca2ac5b91-workload-socket" pod "cea29cd6-99e6-4c4b-b232-6f2ca2ac5b91" (UID: "cea29cd6-99e6-4c4b-b232-6f2ca2ac5b91"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:05:06.799043 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:06.798909    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/509acd5d-92ed-4366-881e-3181be309e13-istio-envoy" pod "509acd5d-92ed-4366-881e-3181be309e13" (UID: "509acd5d-92ed-4366-881e-3181be309e13"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:05:06.799250 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:06.799163    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cea29cd6-99e6-4c4b-b232-6f2ca2ac5b91-istio-data" pod "cea29cd6-99e6-4c4b-b232-6f2ca2ac5b91" (UID: "cea29cd6-99e6-4c4b-b232-6f2ca2ac5b91"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:05:06.799702 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:06.799669    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/509acd5d-92ed-4366-881e-3181be309e13-kube-api-access-c9c89" pod "509acd5d-92ed-4366-881e-3181be309e13" (UID: "509acd5d-92ed-4366-881e-3181be309e13"). InnerVolumeSpecName "kube-api-access-c9c89". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:05:06.800186 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:06.800155    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cea29cd6-99e6-4c4b-b232-6f2ca2ac5b91-istio-envoy" pod "cea29cd6-99e6-4c4b-b232-6f2ca2ac5b91" (UID: "cea29cd6-99e6-4c4b-b232-6f2ca2ac5b91"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:05:06.800277 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:06.800238    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/509acd5d-92ed-4366-881e-3181be309e13-istio-token" pod "509acd5d-92ed-4366-881e-3181be309e13" (UID: "509acd5d-92ed-4366-881e-3181be309e13"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:05:06.800580 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:06.800552    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cea29cd6-99e6-4c4b-b232-6f2ca2ac5b91-istio-token" pod "cea29cd6-99e6-4c4b-b232-6f2ca2ac5b91" (UID: "cea29cd6-99e6-4c4b-b232-6f2ca2ac5b91"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:05:06.801243 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:06.801224    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cea29cd6-99e6-4c4b-b232-6f2ca2ac5b91-kube-api-access-gnjn2" pod "cea29cd6-99e6-4c4b-b232-6f2ca2ac5b91" (UID: "cea29cd6-99e6-4c4b-b232-6f2ca2ac5b91"). InnerVolumeSpecName "kube-api-access-gnjn2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:05:06.801366 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:06.801252    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/cea29cd6-99e6-4c4b-b232-6f2ca2ac5b91-istio-podinfo" pod "cea29cd6-99e6-4c4b-b232-6f2ca2ac5b91" (UID: "cea29cd6-99e6-4c4b-b232-6f2ca2ac5b91"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:05:06.898043 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:06.897938    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c9b213cb-4804-4bf8-a9d4-657bef3a11ba-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c9b213cb-4804-4bf8-a9d4-657bef3a11ba-istio-data\") pod \"c9b213cb-4804-4bf8-a9d4-657bef3a11ba\" (UID: \"c9b213cb-4804-4bf8-a9d4-657bef3a11ba\") "
Oct 30 19:05:06.898043 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:06.898001    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c9b213cb-4804-4bf8-a9d4-657bef3a11ba-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c9b213cb-4804-4bf8-a9d4-657bef3a11ba-istiod-ca-cert\") pod \"c9b213cb-4804-4bf8-a9d4-657bef3a11ba\" (UID: \"c9b213cb-4804-4bf8-a9d4-657bef3a11ba\") "
Oct 30 19:05:06.898394 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:06.898050    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c9b213cb-4804-4bf8-a9d4-657bef3a11ba-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c9b213cb-4804-4bf8-a9d4-657bef3a11ba-istio-envoy\") pod \"c9b213cb-4804-4bf8-a9d4-657bef3a11ba\" (UID: \"c9b213cb-4804-4bf8-a9d4-657bef3a11ba\") "
Oct 30 19:05:06.898394 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:06.898095    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c9b213cb-4804-4bf8-a9d4-657bef3a11ba-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c9b213cb-4804-4bf8-a9d4-657bef3a11ba-workload-certs\") pod \"c9b213cb-4804-4bf8-a9d4-657bef3a11ba\" (UID: \"c9b213cb-4804-4bf8-a9d4-657bef3a11ba\") "
Oct 30 19:05:06.898394 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:06.898137    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c9b213cb-4804-4bf8-a9d4-657bef3a11ba-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c9b213cb-4804-4bf8-a9d4-657bef3a11ba-workload-socket\") pod \"c9b213cb-4804-4bf8-a9d4-657bef3a11ba\" (UID: \"c9b213cb-4804-4bf8-a9d4-657bef3a11ba\") "
Oct 30 19:05:06.898394 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:06.898255    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/c9b213cb-4804-4bf8-a9d4-657bef3a11ba-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c9b213cb-4804-4bf8-a9d4-657bef3a11ba-istio-podinfo\") pod \"c9b213cb-4804-4bf8-a9d4-657bef3a11ba\" (UID: \"c9b213cb-4804-4bf8-a9d4-657bef3a11ba\") "
Oct 30 19:05:06.898394 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:06.898340    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c9b213cb-4804-4bf8-a9d4-657bef3a11ba-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c9b213cb-4804-4bf8-a9d4-657bef3a11ba-credential-socket\") pod \"c9b213cb-4804-4bf8-a9d4-657bef3a11ba\" (UID: \"c9b213cb-4804-4bf8-a9d4-657bef3a11ba\") "
Oct 30 19:05:06.898888 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:06.898480    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c9b213cb-4804-4bf8-a9d4-657bef3a11ba-kube-api-access-6wjtz\" (UniqueName: \"kubernetes.io/projected/c9b213cb-4804-4bf8-a9d4-657bef3a11ba-kube-api-access-6wjtz\") pod \"c9b213cb-4804-4bf8-a9d4-657bef3a11ba\" (UID: \"c9b213cb-4804-4bf8-a9d4-657bef3a11ba\") "
Oct 30 19:05:06.898888 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:06.898515    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c9b213cb-4804-4bf8-a9d4-657bef3a11ba-istio-token\" (UniqueName: \"kubernetes.io/projected/c9b213cb-4804-4bf8-a9d4-657bef3a11ba-istio-token\") pod \"c9b213cb-4804-4bf8-a9d4-657bef3a11ba\" (UID: \"c9b213cb-4804-4bf8-a9d4-657bef3a11ba\") "
Oct 30 19:05:06.899072 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:06.899050    4888 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/509acd5d-92ed-4366-881e-3181be309e13-istio-token\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:05:06.899072 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:06.899081    4888 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cea29cd6-99e6-4c4b-b232-6f2ca2ac5b91-workload-socket\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:05:06.899216 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:06.899096    4888 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cea29cd6-99e6-4c4b-b232-6f2ca2ac5b91-istio-data\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:05:06.899216 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:06.899110    4888 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/509acd5d-92ed-4366-881e-3181be309e13-istio-podinfo\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:05:06.899216 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:06.899124    4888 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cea29cd6-99e6-4c4b-b232-6f2ca2ac5b91-workload-certs\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:05:06.899216 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:06.899137    4888 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cea29cd6-99e6-4c4b-b232-6f2ca2ac5b91-istio-envoy\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:05:06.899216 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:06.899151    4888 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/509acd5d-92ed-4366-881e-3181be309e13-istio-envoy\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:05:06.899216 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:06.899164    4888 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cea29cd6-99e6-4c4b-b232-6f2ca2ac5b91-istio-token\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:05:06.899216 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:06.899177    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c9c89\" (UniqueName: \"kubernetes.io/projected/509acd5d-92ed-4366-881e-3181be309e13-kube-api-access-c9c89\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:05:06.899216 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:06.899190    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gnjn2\" (UniqueName: \"kubernetes.io/projected/cea29cd6-99e6-4c4b-b232-6f2ca2ac5b91-kube-api-access-gnjn2\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:05:06.899216 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:06.899203    4888 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/509acd5d-92ed-4366-881e-3181be309e13-credential-socket\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:05:06.899216 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:06.899217    4888 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/509acd5d-92ed-4366-881e-3181be309e13-workload-socket\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:05:06.899847 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:06.899230    4888 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cea29cd6-99e6-4c4b-b232-6f2ca2ac5b91-istio-podinfo\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:05:06.899847 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:06.899242    4888 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cea29cd6-99e6-4c4b-b232-6f2ca2ac5b91-istiod-ca-cert\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:05:06.899847 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:06.899520    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c9b213cb-4804-4bf8-a9d4-657bef3a11ba-workload-certs" pod "c9b213cb-4804-4bf8-a9d4-657bef3a11ba" (UID: "c9b213cb-4804-4bf8-a9d4-657bef3a11ba"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:05:06.900312 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:06.900254    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c9b213cb-4804-4bf8-a9d4-657bef3a11ba-istio-data" pod "c9b213cb-4804-4bf8-a9d4-657bef3a11ba" (UID: "c9b213cb-4804-4bf8-a9d4-657bef3a11ba"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:05:06.901257 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:06.901179    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c9b213cb-4804-4bf8-a9d4-657bef3a11ba-credential-socket" pod "c9b213cb-4804-4bf8-a9d4-657bef3a11ba" (UID: "c9b213cb-4804-4bf8-a9d4-657bef3a11ba"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:05:06.901257 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:06.901235    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c9b213cb-4804-4bf8-a9d4-657bef3a11ba-workload-socket" pod "c9b213cb-4804-4bf8-a9d4-657bef3a11ba" (UID: "c9b213cb-4804-4bf8-a9d4-657bef3a11ba"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:05:06.901839 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:06.901807    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c9b213cb-4804-4bf8-a9d4-657bef3a11ba-istiod-ca-cert" pod "c9b213cb-4804-4bf8-a9d4-657bef3a11ba" (UID: "c9b213cb-4804-4bf8-a9d4-657bef3a11ba"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:05:06.914303 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:06.914248    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c9b213cb-4804-4bf8-a9d4-657bef3a11ba-istio-podinfo" pod "c9b213cb-4804-4bf8-a9d4-657bef3a11ba" (UID: "c9b213cb-4804-4bf8-a9d4-657bef3a11ba"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:05:06.922696 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:06.922651    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c9b213cb-4804-4bf8-a9d4-657bef3a11ba-istio-token" pod "c9b213cb-4804-4bf8-a9d4-657bef3a11ba" (UID: "c9b213cb-4804-4bf8-a9d4-657bef3a11ba"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:05:06.922827 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:06.922781    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c9b213cb-4804-4bf8-a9d4-657bef3a11ba-kube-api-access-6wjtz" pod "c9b213cb-4804-4bf8-a9d4-657bef3a11ba" (UID: "c9b213cb-4804-4bf8-a9d4-657bef3a11ba"). InnerVolumeSpecName "kube-api-access-6wjtz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:05:06.922923 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:06.922897    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c9b213cb-4804-4bf8-a9d4-657bef3a11ba-istio-envoy" pod "c9b213cb-4804-4bf8-a9d4-657bef3a11ba" (UID: "c9b213cb-4804-4bf8-a9d4-657bef3a11ba"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:05:06.925406 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:06.925377    4888 generic.go:356] "Generic (PLEG): container finished" podID="cea29cd6-99e6-4c4b-b232-6f2ca2ac5b91" containerID="c9d73471a5244580b90cb1dcb49bb798fa16f9da9dc49b4b8d969b46a17f00d0" exitCode=0
Oct 30 19:05:06.925547 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:06.925525    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-2h9dw" event={"ID":"cea29cd6-99e6-4c4b-b232-6f2ca2ac5b91","Type":"ContainerDied","Data":"c9d73471a5244580b90cb1dcb49bb798fa16f9da9dc49b4b8d969b46a17f00d0"}
Oct 30 19:05:06.925620 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:06.925587    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-2h9dw" event={"ID":"cea29cd6-99e6-4c4b-b232-6f2ca2ac5b91","Type":"ContainerDied","Data":"95d9fb377e7869ee76d3c7631b6fef4572896b286e33ef97a3050cfc6418fe14"}
Oct 30 19:05:06.925680 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:06.925621    4888 scope.go:122] "RemoveContainer" containerID="c9d73471a5244580b90cb1dcb49bb798fa16f9da9dc49b4b8d969b46a17f00d0"
Oct 30 19:05:06.925773 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:06.925759    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-2h9dw"
Oct 30 19:05:06.930793 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:06.930771    4888 generic.go:356] "Generic (PLEG): container finished" podID="509acd5d-92ed-4366-881e-3181be309e13" containerID="9f865bc4a0bfbe50ae203eda8a1a4f3dd3ea4695ea82c1003076c6676e818c84" exitCode=0
Oct 30 19:05:06.932106 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:06.931083    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-6trhv"
Oct 30 19:05:06.932106 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:06.931476    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-6trhv" event={"ID":"509acd5d-92ed-4366-881e-3181be309e13","Type":"ContainerDied","Data":"9f865bc4a0bfbe50ae203eda8a1a4f3dd3ea4695ea82c1003076c6676e818c84"}
Oct 30 19:05:06.932106 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:06.931540    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-6trhv" event={"ID":"509acd5d-92ed-4366-881e-3181be309e13","Type":"ContainerDied","Data":"73bd163a437a92086ffc58a21804ed6dd6263817a64ab10d2a42473404e3af7f"}
Oct 30 19:05:06.937169 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:06.937048    4888 generic.go:356] "Generic (PLEG): container finished" podID="c9b213cb-4804-4bf8-a9d4-657bef3a11ba" containerID="e7b2ddb4be2750affca10dc6c0b3a60f7866a4330759d9d8c4e9793494e47803" exitCode=0
Oct 30 19:05:06.937416 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:06.937319    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-jlp7l" event={"ID":"c9b213cb-4804-4bf8-a9d4-657bef3a11ba","Type":"ContainerDied","Data":"e7b2ddb4be2750affca10dc6c0b3a60f7866a4330759d9d8c4e9793494e47803"}
Oct 30 19:05:06.937416 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:06.937370    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-jlp7l" event={"ID":"c9b213cb-4804-4bf8-a9d4-657bef3a11ba","Type":"ContainerDied","Data":"d05c6af5baec3ab4713037030e2bb6500fb126d622d8ff4e743bc499d2da2922"}
Oct 30 19:05:06.937603 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:06.937580    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-jlp7l"
Oct 30 19:05:06.950024 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:06.949907    4888 scope.go:122] "RemoveContainer" containerID="c9d73471a5244580b90cb1dcb49bb798fa16f9da9dc49b4b8d969b46a17f00d0"
Oct 30 19:05:06.950909 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:05:06.950882    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c9d73471a5244580b90cb1dcb49bb798fa16f9da9dc49b4b8d969b46a17f00d0\": container with ID starting with c9d73471a5244580b90cb1dcb49bb798fa16f9da9dc49b4b8d969b46a17f00d0 not found: ID does not exist" containerID="c9d73471a5244580b90cb1dcb49bb798fa16f9da9dc49b4b8d969b46a17f00d0"
Oct 30 19:05:06.950996 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:06.950916    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c9d73471a5244580b90cb1dcb49bb798fa16f9da9dc49b4b8d969b46a17f00d0"} err="failed to get container status \"c9d73471a5244580b90cb1dcb49bb798fa16f9da9dc49b4b8d969b46a17f00d0\": rpc error: code = NotFound desc = could not find container \"c9d73471a5244580b90cb1dcb49bb798fa16f9da9dc49b4b8d969b46a17f00d0\": container with ID starting with c9d73471a5244580b90cb1dcb49bb798fa16f9da9dc49b4b8d969b46a17f00d0 not found: ID does not exist"
Oct 30 19:05:06.950996 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:06.950932    4888 scope.go:122] "RemoveContainer" containerID="9f865bc4a0bfbe50ae203eda8a1a4f3dd3ea4695ea82c1003076c6676e818c84"
Oct 30 19:05:06.963620 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:06.963595    4888 scope.go:122] "RemoveContainer" containerID="9f865bc4a0bfbe50ae203eda8a1a4f3dd3ea4695ea82c1003076c6676e818c84"
Oct 30 19:05:06.963962 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:05:06.963940    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9f865bc4a0bfbe50ae203eda8a1a4f3dd3ea4695ea82c1003076c6676e818c84\": container with ID starting with 9f865bc4a0bfbe50ae203eda8a1a4f3dd3ea4695ea82c1003076c6676e818c84 not found: ID does not exist" containerID="9f865bc4a0bfbe50ae203eda8a1a4f3dd3ea4695ea82c1003076c6676e818c84"
Oct 30 19:05:06.964057 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:06.963973    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9f865bc4a0bfbe50ae203eda8a1a4f3dd3ea4695ea82c1003076c6676e818c84"} err="failed to get container status \"9f865bc4a0bfbe50ae203eda8a1a4f3dd3ea4695ea82c1003076c6676e818c84\": rpc error: code = NotFound desc = could not find container \"9f865bc4a0bfbe50ae203eda8a1a4f3dd3ea4695ea82c1003076c6676e818c84\": container with ID starting with 9f865bc4a0bfbe50ae203eda8a1a4f3dd3ea4695ea82c1003076c6676e818c84 not found: ID does not exist"
Oct 30 19:05:06.964057 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:06.963995    4888 scope.go:122] "RemoveContainer" containerID="e7b2ddb4be2750affca10dc6c0b3a60f7866a4330759d9d8c4e9793494e47803"
Oct 30 19:05:06.976900 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:06.976878    4888 scope.go:122] "RemoveContainer" containerID="e7b2ddb4be2750affca10dc6c0b3a60f7866a4330759d9d8c4e9793494e47803"
Oct 30 19:05:06.977187 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:05:06.977160    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e7b2ddb4be2750affca10dc6c0b3a60f7866a4330759d9d8c4e9793494e47803\": container with ID starting with e7b2ddb4be2750affca10dc6c0b3a60f7866a4330759d9d8c4e9793494e47803 not found: ID does not exist" containerID="e7b2ddb4be2750affca10dc6c0b3a60f7866a4330759d9d8c4e9793494e47803"
Oct 30 19:05:06.977270 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:06.977197    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e7b2ddb4be2750affca10dc6c0b3a60f7866a4330759d9d8c4e9793494e47803"} err="failed to get container status \"e7b2ddb4be2750affca10dc6c0b3a60f7866a4330759d9d8c4e9793494e47803\": rpc error: code = NotFound desc = could not find container \"e7b2ddb4be2750affca10dc6c0b3a60f7866a4330759d9d8c4e9793494e47803\": container with ID starting with e7b2ddb4be2750affca10dc6c0b3a60f7866a4330759d9d8c4e9793494e47803 not found: ID does not exist"
Oct 30 19:05:07.000298 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:07.000254    4888 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c9b213cb-4804-4bf8-a9d4-657bef3a11ba-istio-data\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:05:07.000412 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:07.000307    4888 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c9b213cb-4804-4bf8-a9d4-657bef3a11ba-istiod-ca-cert\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:05:07.000412 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:07.000324    4888 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c9b213cb-4804-4bf8-a9d4-657bef3a11ba-istio-envoy\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:05:07.000412 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:07.000336    4888 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c9b213cb-4804-4bf8-a9d4-657bef3a11ba-workload-certs\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:05:07.000412 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:07.000351    4888 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c9b213cb-4804-4bf8-a9d4-657bef3a11ba-workload-socket\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:05:07.000412 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:07.000363    4888 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c9b213cb-4804-4bf8-a9d4-657bef3a11ba-istio-podinfo\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:05:07.000412 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:07.000377    4888 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c9b213cb-4804-4bf8-a9d4-657bef3a11ba-credential-socket\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:05:07.000412 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:07.000393    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6wjtz\" (UniqueName: \"kubernetes.io/projected/c9b213cb-4804-4bf8-a9d4-657bef3a11ba-kube-api-access-6wjtz\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:05:07.000412 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:07.000407    4888 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c9b213cb-4804-4bf8-a9d4-657bef3a11ba-istio-token\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:05:07.336215 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:07.336181    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-6trhv"]
Oct 30 19:05:07.345403 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:07.345373    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-6trhv"]
Oct 30 19:05:07.461698 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:07.461662    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-2h9dw"]
Oct 30 19:05:07.467784 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:07.467753    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-2h9dw"]
Oct 30 19:05:07.625402 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:07.624343    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-jlp7l"]
Oct 30 19:05:07.633060 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:07.632959    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-4ebcd2-openshift-default-59d766c94c-jlp7l"]
Oct 30 19:05:07.892791 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:07.892708    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="509acd5d-92ed-4366-881e-3181be309e13" path="/var/lib/kubelet/pods/509acd5d-92ed-4366-881e-3181be309e13/volumes"
Oct 30 19:05:07.893472 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:07.893443    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c9b213cb-4804-4bf8-a9d4-657bef3a11ba" path="/var/lib/kubelet/pods/c9b213cb-4804-4bf8-a9d4-657bef3a11ba/volumes"
Oct 30 19:05:07.894031 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:07.894012    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="cea29cd6-99e6-4c4b-b232-6f2ca2ac5b91" path="/var/lib/kubelet/pods/cea29cd6-99e6-4c4b-b232-6f2ca2ac5b91/volumes"
Oct 30 19:05:09.586448 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:09.586409    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-rjmdz"]
Oct 30 19:05:11.324000 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:11.323962    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-b-0ee379-openshift-default-647f98d679-s29zb"]
Oct 30 19:05:11.324795 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:11.324774    4888 cpu_manager.go:414] "containerMap: removing container" podUID="509acd5d-92ed-4366-881e-3181be309e13" containerName="istio-proxy"
Oct 30 19:05:11.324904 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:11.324796    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="509acd5d-92ed-4366-881e-3181be309e13" containerName="istio-proxy"
Oct 30 19:05:11.324904 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:11.324820    4888 cpu_manager.go:414] "containerMap: removing container" podUID="c9b213cb-4804-4bf8-a9d4-657bef3a11ba" containerName="istio-proxy"
Oct 30 19:05:11.324904 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:11.324826    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c9b213cb-4804-4bf8-a9d4-657bef3a11ba" containerName="istio-proxy"
Oct 30 19:05:11.324904 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:11.324854    4888 cpu_manager.go:414] "containerMap: removing container" podUID="cea29cd6-99e6-4c4b-b232-6f2ca2ac5b91" containerName="istio-proxy"
Oct 30 19:05:11.324904 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:11.324862    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cea29cd6-99e6-4c4b-b232-6f2ca2ac5b91" containerName="istio-proxy"
Oct 30 19:05:11.325164 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:11.325007    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="cea29cd6-99e6-4c4b-b232-6f2ca2ac5b91" containerName="istio-proxy"
Oct 30 19:05:11.325164 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:11.325030    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="509acd5d-92ed-4366-881e-3181be309e13" containerName="istio-proxy"
Oct 30 19:05:11.325164 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:11.325044    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="c9b213cb-4804-4bf8-a9d4-657bef3a11ba" containerName="istio-proxy"
Oct 30 19:05:11.332512 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:11.332488    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-0ee379-openshift-default-647f98d679-s29zb"
Oct 30 19:05:11.368799 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:11.368766    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/29c22c2d-64fe-476b-8b35-eab525253a5c-istio-token\") pod \"e2e-ipp-b-0ee379-openshift-default-647f98d679-s29zb\" (UID: \"29c22c2d-64fe-476b-8b35-eab525253a5c\") " pod="openshift-ingress/e2e-ipp-b-0ee379-openshift-default-647f98d679-s29zb"
Oct 30 19:05:11.368968 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:11.368821    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/29c22c2d-64fe-476b-8b35-eab525253a5c-istio-podinfo\") pod \"e2e-ipp-b-0ee379-openshift-default-647f98d679-s29zb\" (UID: \"29c22c2d-64fe-476b-8b35-eab525253a5c\") " pod="openshift-ingress/e2e-ipp-b-0ee379-openshift-default-647f98d679-s29zb"
Oct 30 19:05:11.368968 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:11.368950    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/29c22c2d-64fe-476b-8b35-eab525253a5c-workload-socket\") pod \"e2e-ipp-b-0ee379-openshift-default-647f98d679-s29zb\" (UID: \"29c22c2d-64fe-476b-8b35-eab525253a5c\") " pod="openshift-ingress/e2e-ipp-b-0ee379-openshift-default-647f98d679-s29zb"
Oct 30 19:05:11.369095 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:11.368998    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/29c22c2d-64fe-476b-8b35-eab525253a5c-istio-data\") pod \"e2e-ipp-b-0ee379-openshift-default-647f98d679-s29zb\" (UID: \"29c22c2d-64fe-476b-8b35-eab525253a5c\") " pod="openshift-ingress/e2e-ipp-b-0ee379-openshift-default-647f98d679-s29zb"
Oct 30 19:05:11.369095 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:11.369049    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/29c22c2d-64fe-476b-8b35-eab525253a5c-istio-envoy\") pod \"e2e-ipp-b-0ee379-openshift-default-647f98d679-s29zb\" (UID: \"29c22c2d-64fe-476b-8b35-eab525253a5c\") " pod="openshift-ingress/e2e-ipp-b-0ee379-openshift-default-647f98d679-s29zb"
Oct 30 19:05:11.369207 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:11.369101    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/29c22c2d-64fe-476b-8b35-eab525253a5c-workload-certs\") pod \"e2e-ipp-b-0ee379-openshift-default-647f98d679-s29zb\" (UID: \"29c22c2d-64fe-476b-8b35-eab525253a5c\") " pod="openshift-ingress/e2e-ipp-b-0ee379-openshift-default-647f98d679-s29zb"
Oct 30 19:05:11.369207 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:11.369182    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/29c22c2d-64fe-476b-8b35-eab525253a5c-istiod-ca-cert\") pod \"e2e-ipp-b-0ee379-openshift-default-647f98d679-s29zb\" (UID: \"29c22c2d-64fe-476b-8b35-eab525253a5c\") " pod="openshift-ingress/e2e-ipp-b-0ee379-openshift-default-647f98d679-s29zb"
Oct 30 19:05:11.369346 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:11.369207    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/29c22c2d-64fe-476b-8b35-eab525253a5c-credential-socket\") pod \"e2e-ipp-b-0ee379-openshift-default-647f98d679-s29zb\" (UID: \"29c22c2d-64fe-476b-8b35-eab525253a5c\") " pod="openshift-ingress/e2e-ipp-b-0ee379-openshift-default-647f98d679-s29zb"
Oct 30 19:05:11.369346 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:11.369278    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8lqs4\" (UniqueName: \"kubernetes.io/projected/29c22c2d-64fe-476b-8b35-eab525253a5c-kube-api-access-8lqs4\") pod \"e2e-ipp-b-0ee379-openshift-default-647f98d679-s29zb\" (UID: \"29c22c2d-64fe-476b-8b35-eab525253a5c\") " pod="openshift-ingress/e2e-ipp-b-0ee379-openshift-default-647f98d679-s29zb"
Oct 30 19:05:11.424926 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:11.424897    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-0ee379-openshift-default-647f98d679-s29zb"]
Oct 30 19:05:11.450388 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:11.448491    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-b-0ee379-openshift-default-dockercfg-fj9j8\""
Oct 30 19:05:11.474362 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:11.471152    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/29c22c2d-64fe-476b-8b35-eab525253a5c-istiod-ca-cert\") pod \"e2e-ipp-b-0ee379-openshift-default-647f98d679-s29zb\" (UID: \"29c22c2d-64fe-476b-8b35-eab525253a5c\") " pod="openshift-ingress/e2e-ipp-b-0ee379-openshift-default-647f98d679-s29zb"
Oct 30 19:05:11.474569 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:11.474406    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/29c22c2d-64fe-476b-8b35-eab525253a5c-credential-socket\") pod \"e2e-ipp-b-0ee379-openshift-default-647f98d679-s29zb\" (UID: \"29c22c2d-64fe-476b-8b35-eab525253a5c\") " pod="openshift-ingress/e2e-ipp-b-0ee379-openshift-default-647f98d679-s29zb"
Oct 30 19:05:11.474569 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:11.474495    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8lqs4\" (UniqueName: \"kubernetes.io/projected/29c22c2d-64fe-476b-8b35-eab525253a5c-kube-api-access-8lqs4\") pod \"e2e-ipp-b-0ee379-openshift-default-647f98d679-s29zb\" (UID: \"29c22c2d-64fe-476b-8b35-eab525253a5c\") " pod="openshift-ingress/e2e-ipp-b-0ee379-openshift-default-647f98d679-s29zb"
Oct 30 19:05:11.474704 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:11.474613    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/29c22c2d-64fe-476b-8b35-eab525253a5c-istio-token\") pod \"e2e-ipp-b-0ee379-openshift-default-647f98d679-s29zb\" (UID: \"29c22c2d-64fe-476b-8b35-eab525253a5c\") " pod="openshift-ingress/e2e-ipp-b-0ee379-openshift-default-647f98d679-s29zb"
Oct 30 19:05:11.474704 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:11.474652    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/29c22c2d-64fe-476b-8b35-eab525253a5c-istio-podinfo\") pod \"e2e-ipp-b-0ee379-openshift-default-647f98d679-s29zb\" (UID: \"29c22c2d-64fe-476b-8b35-eab525253a5c\") " pod="openshift-ingress/e2e-ipp-b-0ee379-openshift-default-647f98d679-s29zb"
Oct 30 19:05:11.474828 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:11.474720    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/29c22c2d-64fe-476b-8b35-eab525253a5c-workload-socket\") pod \"e2e-ipp-b-0ee379-openshift-default-647f98d679-s29zb\" (UID: \"29c22c2d-64fe-476b-8b35-eab525253a5c\") " pod="openshift-ingress/e2e-ipp-b-0ee379-openshift-default-647f98d679-s29zb"
Oct 30 19:05:11.474828 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:11.474757    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/29c22c2d-64fe-476b-8b35-eab525253a5c-istio-data\") pod \"e2e-ipp-b-0ee379-openshift-default-647f98d679-s29zb\" (UID: \"29c22c2d-64fe-476b-8b35-eab525253a5c\") " pod="openshift-ingress/e2e-ipp-b-0ee379-openshift-default-647f98d679-s29zb"
Oct 30 19:05:11.474828 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:11.474813    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/29c22c2d-64fe-476b-8b35-eab525253a5c-istio-envoy\") pod \"e2e-ipp-b-0ee379-openshift-default-647f98d679-s29zb\" (UID: \"29c22c2d-64fe-476b-8b35-eab525253a5c\") " pod="openshift-ingress/e2e-ipp-b-0ee379-openshift-default-647f98d679-s29zb"
Oct 30 19:05:11.474986 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:11.474870    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/29c22c2d-64fe-476b-8b35-eab525253a5c-workload-certs\") pod \"e2e-ipp-b-0ee379-openshift-default-647f98d679-s29zb\" (UID: \"29c22c2d-64fe-476b-8b35-eab525253a5c\") " pod="openshift-ingress/e2e-ipp-b-0ee379-openshift-default-647f98d679-s29zb"
Oct 30 19:05:11.475382 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:11.475356    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/29c22c2d-64fe-476b-8b35-eab525253a5c-workload-certs\") pod \"e2e-ipp-b-0ee379-openshift-default-647f98d679-s29zb\" (UID: \"29c22c2d-64fe-476b-8b35-eab525253a5c\") " pod="openshift-ingress/e2e-ipp-b-0ee379-openshift-default-647f98d679-s29zb"
Oct 30 19:05:11.476151 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:11.476125    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/29c22c2d-64fe-476b-8b35-eab525253a5c-istiod-ca-cert\") pod \"e2e-ipp-b-0ee379-openshift-default-647f98d679-s29zb\" (UID: \"29c22c2d-64fe-476b-8b35-eab525253a5c\") " pod="openshift-ingress/e2e-ipp-b-0ee379-openshift-default-647f98d679-s29zb"
Oct 30 19:05:11.476425 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:11.476403    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/29c22c2d-64fe-476b-8b35-eab525253a5c-credential-socket\") pod \"e2e-ipp-b-0ee379-openshift-default-647f98d679-s29zb\" (UID: \"29c22c2d-64fe-476b-8b35-eab525253a5c\") " pod="openshift-ingress/e2e-ipp-b-0ee379-openshift-default-647f98d679-s29zb"
Oct 30 19:05:11.477446 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:11.477422    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/29c22c2d-64fe-476b-8b35-eab525253a5c-istio-data\") pod \"e2e-ipp-b-0ee379-openshift-default-647f98d679-s29zb\" (UID: \"29c22c2d-64fe-476b-8b35-eab525253a5c\") " pod="openshift-ingress/e2e-ipp-b-0ee379-openshift-default-647f98d679-s29zb"
Oct 30 19:05:11.477665 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:11.477645    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/29c22c2d-64fe-476b-8b35-eab525253a5c-workload-socket\") pod \"e2e-ipp-b-0ee379-openshift-default-647f98d679-s29zb\" (UID: \"29c22c2d-64fe-476b-8b35-eab525253a5c\") " pod="openshift-ingress/e2e-ipp-b-0ee379-openshift-default-647f98d679-s29zb"
Oct 30 19:05:11.484596 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:11.482899    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/29c22c2d-64fe-476b-8b35-eab525253a5c-istio-envoy\") pod \"e2e-ipp-b-0ee379-openshift-default-647f98d679-s29zb\" (UID: \"29c22c2d-64fe-476b-8b35-eab525253a5c\") " pod="openshift-ingress/e2e-ipp-b-0ee379-openshift-default-647f98d679-s29zb"
Oct 30 19:05:11.512314 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:11.509155    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/29c22c2d-64fe-476b-8b35-eab525253a5c-istio-podinfo\") pod \"e2e-ipp-b-0ee379-openshift-default-647f98d679-s29zb\" (UID: \"29c22c2d-64fe-476b-8b35-eab525253a5c\") " pod="openshift-ingress/e2e-ipp-b-0ee379-openshift-default-647f98d679-s29zb"
Oct 30 19:05:11.578787 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:11.578706    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8lqs4\" (UniqueName: \"kubernetes.io/projected/29c22c2d-64fe-476b-8b35-eab525253a5c-kube-api-access-8lqs4\") pod \"e2e-ipp-b-0ee379-openshift-default-647f98d679-s29zb\" (UID: \"29c22c2d-64fe-476b-8b35-eab525253a5c\") " pod="openshift-ingress/e2e-ipp-b-0ee379-openshift-default-647f98d679-s29zb"
Oct 30 19:05:11.580719 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:11.580694    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/29c22c2d-64fe-476b-8b35-eab525253a5c-istio-token\") pod \"e2e-ipp-b-0ee379-openshift-default-647f98d679-s29zb\" (UID: \"29c22c2d-64fe-476b-8b35-eab525253a5c\") " pod="openshift-ingress/e2e-ipp-b-0ee379-openshift-default-647f98d679-s29zb"
Oct 30 19:05:11.646332 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:11.646296    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-0ee379-openshift-default-647f98d679-s29zb"
Oct 30 19:05:12.107690 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:12.107653    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-0ee379-openshift-default-647f98d679-s29zb"]
Oct 30 19:05:12.135733 ip-10-0-134-46 kubenswrapper[4888]: W1030 19:05:12.135700    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29c22c2d_64fe_476b_8b35_eab525253a5c.slice/crio-ec1c73b44df6389f2813ffd258dcf5641fd752f69573903e03e75a515015887a WatchSource:0}: Error finding container ec1c73b44df6389f2813ffd258dcf5641fd752f69573903e03e75a515015887a: Status 404 returned error can't find the container with id ec1c73b44df6389f2813ffd258dcf5641fd752f69573903e03e75a515015887a
Oct 30 19:05:12.138220 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:12.138190    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:05:12.138307 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:12.138260    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:05:12.138352 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:12.138320    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:05:13.001304 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:12.993805    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-0ee379-openshift-default-647f98d679-s29zb" event={"ID":"29c22c2d-64fe-476b-8b35-eab525253a5c","Type":"ContainerStarted","Data":"78524c76cd12e11a69585cf1dce2977bb240bc37702e952b6653eadd2f6a393d"}
Oct 30 19:05:13.001304 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:12.993870    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-0ee379-openshift-default-647f98d679-s29zb" event={"ID":"29c22c2d-64fe-476b-8b35-eab525253a5c","Type":"ContainerStarted","Data":"ec1c73b44df6389f2813ffd258dcf5641fd752f69573903e03e75a515015887a"}
Oct 30 19:05:13.234131 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:13.234079    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-b-0ee379-openshift-default-647f98d679-s29zb" podStartSLOduration=2.234065272 podStartE2EDuration="2.234065272s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:05:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:05:13.181066055 +0000 UTC m=+1877.428472082" watchObservedRunningTime="2026-10-30 19:05:13.234065272 +0000 UTC m=+1877.481471296"
Oct 30 19:05:13.647431 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:13.647391    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-0ee379-openshift-default-647f98d679-s29zb"
Oct 30 19:05:13.653830 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:13.653804    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-0ee379-openshift-default-647f98d679-s29zb"
Oct 30 19:05:14.001647 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:14.001572    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-0ee379-openshift-default-647f98d679-s29zb"
Oct 30 19:05:14.002838 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:14.002812    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-0ee379-openshift-default-647f98d679-s29zb"
Oct 30 19:05:34.492424 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:34.492389    4888 patch_prober.go:29] interesting pod/e2e-ipp-b-fee9f0-openshift-default-68c7499775-l68kt container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.117:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:05:34.492871 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:34.492448    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-b-fee9f0-openshift-default-68c7499775-l68kt" podUID="94364f5b-4740-430f-87df-51e9b656d963" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.117:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:05:35.619978 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:05:35.619932    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-jbsw6" containerName="extractor"
Oct 30 19:05:36.265351 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:36.265246    4888 generic.go:356] "Generic (PLEG): container finished" podID="94364f5b-4740-430f-87df-51e9b656d963" containerID="fb5a4cc3729eff702853803d5f2bf01fbebe67656cad3e6f808e6fb6472e8448" exitCode=0
Oct 30 19:05:36.265351 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:36.265317    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-fee9f0-openshift-default-68c7499775-l68kt" event={"ID":"94364f5b-4740-430f-87df-51e9b656d963","Type":"ContainerDied","Data":"fb5a4cc3729eff702853803d5f2bf01fbebe67656cad3e6f808e6fb6472e8448"}
Oct 30 19:05:36.265744 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:36.265719    4888 scope.go:122] "RemoveContainer" containerID="fb5a4cc3729eff702853803d5f2bf01fbebe67656cad3e6f808e6fb6472e8448"
Oct 30 19:05:36.267278 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:36.267248    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:05:36.267422 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:36.267352    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:05:36.267422 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:36.267415    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:05:37.276658 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:37.276626    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-fee9f0-openshift-default-68c7499775-l68kt" event={"ID":"94364f5b-4740-430f-87df-51e9b656d963","Type":"ContainerStarted","Data":"43090d61fd202d9007705e78bc00d336c6094a6bcbe3288cef75d606cc8ed3cb"}
Oct 30 19:05:38.132587 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:38.132548    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-fee9f0-openshift-default-68c7499775-l68kt"
Oct 30 19:05:38.132777 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:38.132707    4888 patch_prober.go:29] interesting pod/e2e-ipp-b-fee9f0-openshift-default-68c7499775-l68kt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.117:15021/healthz/ready\": dial tcp 10.132.0.117:15021: connect: connection refused" start-of-body=
Oct 30 19:05:38.132777 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:38.132760    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-fee9f0-openshift-default-68c7499775-l68kt" podUID="94364f5b-4740-430f-87df-51e9b656d963" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.117:15021/healthz/ready\": dial tcp 10.132.0.117:15021: connect: connection refused"
Oct 30 19:05:39.132608 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:39.132553    4888 patch_prober.go:29] interesting pod/e2e-ipp-b-fee9f0-openshift-default-68c7499775-l68kt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.117:15021/healthz/ready\": dial tcp 10.132.0.117:15021: connect: connection refused" start-of-body=
Oct 30 19:05:39.133018 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:39.132624    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-fee9f0-openshift-default-68c7499775-l68kt" podUID="94364f5b-4740-430f-87df-51e9b656d963" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.117:15021/healthz/ready\": dial tcp 10.132.0.117:15021: connect: connection refused"
Oct 30 19:05:40.132893 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:40.132851    4888 patch_prober.go:29] interesting pod/e2e-ipp-b-fee9f0-openshift-default-68c7499775-l68kt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.117:15021/healthz/ready\": dial tcp 10.132.0.117:15021: connect: connection refused" start-of-body=
Oct 30 19:05:40.133437 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:40.132923    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-fee9f0-openshift-default-68c7499775-l68kt" podUID="94364f5b-4740-430f-87df-51e9b656d963" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.117:15021/healthz/ready\": dial tcp 10.132.0.117:15021: connect: connection refused"
Oct 30 19:05:41.133131 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:41.133094    4888 patch_prober.go:29] interesting pod/e2e-ipp-b-fee9f0-openshift-default-68c7499775-l68kt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.117:15021/healthz/ready\": dial tcp 10.132.0.117:15021: connect: connection refused" start-of-body=
Oct 30 19:05:41.133576 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:41.133154    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-fee9f0-openshift-default-68c7499775-l68kt" podUID="94364f5b-4740-430f-87df-51e9b656d963" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.117:15021/healthz/ready\": dial tcp 10.132.0.117:15021: connect: connection refused"
Oct 30 19:05:42.132346 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:42.132314    4888 patch_prober.go:29] interesting pod/e2e-ipp-b-fee9f0-openshift-default-68c7499775-l68kt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.117:15021/healthz/ready\": dial tcp 10.132.0.117:15021: connect: connection refused" start-of-body=
Oct 30 19:05:42.132530 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:42.132381    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-fee9f0-openshift-default-68c7499775-l68kt" podUID="94364f5b-4740-430f-87df-51e9b656d963" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.117:15021/healthz/ready\": dial tcp 10.132.0.117:15021: connect: connection refused"
Oct 30 19:05:43.276626 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:43.276588    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-fee9f0-openshift-default-68c7499775-l68kt"
Oct 30 19:05:43.277955 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:43.277931    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-fee9f0-openshift-default-68c7499775-l68kt"
Oct 30 19:05:43.280817 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:43.279181    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-fee9f0-openshift-default-68c7499775-l68kt"
Oct 30 19:05:43.391577 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:05:43.391546    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-rjmdz"]
Oct 30 19:06:05.618351 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:06:05.618240    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-jbsw6" containerName="extractor"
Oct 30 19:06:11.331966 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:06:11.331935    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-0c9a2b-5f4f444dc9-rvxxc"]
Oct 30 19:06:11.337551 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:06:11.337529    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0c9a2b-5f4f444dc9-rvxxc"
Oct 30 19:06:11.443969 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:06:11.443939    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-0c9a2b-serving-cert\""
Oct 30 19:06:11.447614 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:06:11.447594    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-0c9a2b-5f4f444dc9-rvxxc"]
Oct 30 19:06:11.494445 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:06:11.494410    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/108b1dc2-3b32-4ede-b94c-403cc9ffa73e-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-0c9a2b-5f4f444dc9-rvxxc\" (UID: \"108b1dc2-3b32-4ede-b94c-403cc9ffa73e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0c9a2b-5f4f444dc9-rvxxc"
Oct 30 19:06:11.494592 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:06:11.494553    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/108b1dc2-3b32-4ede-b94c-403cc9ffa73e-maas-api-tls\") pod \"maas-api-e2e-ipp-a-0c9a2b-5f4f444dc9-rvxxc\" (UID: \"108b1dc2-3b32-4ede-b94c-403cc9ffa73e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0c9a2b-5f4f444dc9-rvxxc"
Oct 30 19:06:11.494644 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:06:11.494590    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/108b1dc2-3b32-4ede-b94c-403cc9ffa73e-metrics-certs\") pod \"maas-api-e2e-ipp-a-0c9a2b-5f4f444dc9-rvxxc\" (UID: \"108b1dc2-3b32-4ede-b94c-403cc9ffa73e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0c9a2b-5f4f444dc9-rvxxc"
Oct 30 19:06:11.494644 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:06:11.494637    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b2prb\" (UniqueName: \"kubernetes.io/projected/108b1dc2-3b32-4ede-b94c-403cc9ffa73e-kube-api-access-b2prb\") pod \"maas-api-e2e-ipp-a-0c9a2b-5f4f444dc9-rvxxc\" (UID: \"108b1dc2-3b32-4ede-b94c-403cc9ffa73e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0c9a2b-5f4f444dc9-rvxxc"
Oct 30 19:06:11.595580 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:06:11.595508    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/108b1dc2-3b32-4ede-b94c-403cc9ffa73e-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-0c9a2b-5f4f444dc9-rvxxc\" (UID: \"108b1dc2-3b32-4ede-b94c-403cc9ffa73e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0c9a2b-5f4f444dc9-rvxxc"
Oct 30 19:06:11.595730 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:06:11.595620    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/108b1dc2-3b32-4ede-b94c-403cc9ffa73e-maas-api-tls\") pod \"maas-api-e2e-ipp-a-0c9a2b-5f4f444dc9-rvxxc\" (UID: \"108b1dc2-3b32-4ede-b94c-403cc9ffa73e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0c9a2b-5f4f444dc9-rvxxc"
Oct 30 19:06:11.595730 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:06:11.595650    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/108b1dc2-3b32-4ede-b94c-403cc9ffa73e-metrics-certs\") pod \"maas-api-e2e-ipp-a-0c9a2b-5f4f444dc9-rvxxc\" (UID: \"108b1dc2-3b32-4ede-b94c-403cc9ffa73e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0c9a2b-5f4f444dc9-rvxxc"
Oct 30 19:06:11.595730 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:06:11.595704    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b2prb\" (UniqueName: \"kubernetes.io/projected/108b1dc2-3b32-4ede-b94c-403cc9ffa73e-kube-api-access-b2prb\") pod \"maas-api-e2e-ipp-a-0c9a2b-5f4f444dc9-rvxxc\" (UID: \"108b1dc2-3b32-4ede-b94c-403cc9ffa73e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0c9a2b-5f4f444dc9-rvxxc"
Oct 30 19:06:11.596325 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:06:11.596243    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/108b1dc2-3b32-4ede-b94c-403cc9ffa73e-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-0c9a2b-5f4f444dc9-rvxxc\" (UID: \"108b1dc2-3b32-4ede-b94c-403cc9ffa73e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0c9a2b-5f4f444dc9-rvxxc"
Oct 30 19:06:11.598189 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:06:11.598168    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/108b1dc2-3b32-4ede-b94c-403cc9ffa73e-maas-api-tls\") pod \"maas-api-e2e-ipp-a-0c9a2b-5f4f444dc9-rvxxc\" (UID: \"108b1dc2-3b32-4ede-b94c-403cc9ffa73e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0c9a2b-5f4f444dc9-rvxxc"
Oct 30 19:06:11.598341 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:06:11.598250    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/108b1dc2-3b32-4ede-b94c-403cc9ffa73e-metrics-certs\") pod \"maas-api-e2e-ipp-a-0c9a2b-5f4f444dc9-rvxxc\" (UID: \"108b1dc2-3b32-4ede-b94c-403cc9ffa73e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0c9a2b-5f4f444dc9-rvxxc"
Oct 30 19:06:11.658651 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:06:11.658617    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b2prb\" (UniqueName: \"kubernetes.io/projected/108b1dc2-3b32-4ede-b94c-403cc9ffa73e-kube-api-access-b2prb\") pod \"maas-api-e2e-ipp-a-0c9a2b-5f4f444dc9-rvxxc\" (UID: \"108b1dc2-3b32-4ede-b94c-403cc9ffa73e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0c9a2b-5f4f444dc9-rvxxc"
Oct 30 19:06:11.950974 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:06:11.950900    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0c9a2b-5f4f444dc9-rvxxc"
Oct 30 19:06:12.331497 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:06:12.331466    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-0c9a2b-5f4f444dc9-rvxxc"]
Oct 30 19:06:12.332002 ip-10-0-134-46 kubenswrapper[4888]: W1030 19:06:12.331976    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod108b1dc2_3b32_4ede_b94c_403cc9ffa73e.slice/crio-70954ee5c18d4bb5ac4168611dd702634df23c0abbdaeca1ca4fe694b227cf4e WatchSource:0}: Error finding container 70954ee5c18d4bb5ac4168611dd702634df23c0abbdaeca1ca4fe694b227cf4e: Status 404 returned error can't find the container with id 70954ee5c18d4bb5ac4168611dd702634df23c0abbdaeca1ca4fe694b227cf4e
Oct 30 19:06:12.333454 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:06:12.333433    4888 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 19:06:12.645318 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:06:12.645227    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0c9a2b-5f4f444dc9-rvxxc" event={"ID":"108b1dc2-3b32-4ede-b94c-403cc9ffa73e","Type":"ContainerStarted","Data":"70954ee5c18d4bb5ac4168611dd702634df23c0abbdaeca1ca4fe694b227cf4e"}
Oct 30 19:06:13.653675 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:06:13.653637    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0c9a2b-5f4f444dc9-rvxxc" event={"ID":"108b1dc2-3b32-4ede-b94c-403cc9ffa73e","Type":"ContainerStarted","Data":"73a9718c2012a19fb7f62b4ad93945282dfe35028379a584fcb9f044ce36e578"}
Oct 30 19:06:13.654078 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:06:13.653806    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0c9a2b-5f4f444dc9-rvxxc"
Oct 30 19:06:13.774718 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:06:13.774661    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0c9a2b-5f4f444dc9-rvxxc" podStartSLOduration=1.9250397910000001 podStartE2EDuration="2.774642985s" totalImagesPullingTime="849.603194ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:06:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:06:13.773731824 +0000 UTC m=+1938.021137851" watchObservedRunningTime="2026-10-30 19:06:13.774642985 +0000 UTC m=+1938.022049014"
Oct 30 19:06:19.666225 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:06:19.666193    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0c9a2b-5f4f444dc9-rvxxc"
Oct 30 19:06:25.568080 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:06:25.568043    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-fee9f0-5f799c94f9-msvnw"]
Oct 30 19:06:27.656081 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:06:27.656047    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-rjmdz"]
Oct 30 19:06:30.577324 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:06:30.574459    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-fee9f0-5f799c94f9-msvnw" podUID="8317ca5e-a974-48a6-ad90-4c08cdd6bf57" containerName="payload-pre-processing" containerID="cri-o://97766aff4ad125b43f5cb316986769b30060005de5c68a9e6f8f3dd833e4daf7" gracePeriod=25
Oct 30 19:06:30.826105 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:06:30.825897    4888 generic.go:356] "Generic (PLEG): container finished" podID="8317ca5e-a974-48a6-ad90-4c08cdd6bf57" containerID="97766aff4ad125b43f5cb316986769b30060005de5c68a9e6f8f3dd833e4daf7" exitCode=0
Oct 30 19:06:30.826105 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:06:30.826029    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-fee9f0-5f799c94f9-msvnw" event={"ID":"8317ca5e-a974-48a6-ad90-4c08cdd6bf57","Type":"ContainerDied","Data":"97766aff4ad125b43f5cb316986769b30060005de5c68a9e6f8f3dd833e4daf7"}
Oct 30 19:06:30.955922 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:06:30.955845    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-fee9f0-5f799c94f9-msvnw"
Oct 30 19:06:31.017203 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:06:31.017173    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8317ca5e-a974-48a6-ad90-4c08cdd6bf57-kube-api-access-rp7pw\" (UniqueName: \"kubernetes.io/projected/8317ca5e-a974-48a6-ad90-4c08cdd6bf57-kube-api-access-rp7pw\") pod \"8317ca5e-a974-48a6-ad90-4c08cdd6bf57\" (UID: \"8317ca5e-a974-48a6-ad90-4c08cdd6bf57\") "
Oct 30 19:06:31.017413 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:06:31.017395    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8317ca5e-a974-48a6-ad90-4c08cdd6bf57-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8317ca5e-a974-48a6-ad90-4c08cdd6bf57-plugins-config-volume\") pod \"8317ca5e-a974-48a6-ad90-4c08cdd6bf57\" (UID: \"8317ca5e-a974-48a6-ad90-4c08cdd6bf57\") "
Oct 30 19:06:31.017755 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:06:31.017735    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8317ca5e-a974-48a6-ad90-4c08cdd6bf57-plugins-config-volume" pod "8317ca5e-a974-48a6-ad90-4c08cdd6bf57" (UID: "8317ca5e-a974-48a6-ad90-4c08cdd6bf57"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:06:31.019544 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:06:31.019524    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8317ca5e-a974-48a6-ad90-4c08cdd6bf57-kube-api-access-rp7pw" pod "8317ca5e-a974-48a6-ad90-4c08cdd6bf57" (UID: "8317ca5e-a974-48a6-ad90-4c08cdd6bf57"). InnerVolumeSpecName "kube-api-access-rp7pw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:06:31.119103 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:06:31.119064    4888 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8317ca5e-a974-48a6-ad90-4c08cdd6bf57-plugins-config-volume\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:06:31.119103 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:06:31.119093    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rp7pw\" (UniqueName: \"kubernetes.io/projected/8317ca5e-a974-48a6-ad90-4c08cdd6bf57-kube-api-access-rp7pw\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:06:31.834708 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:06:31.834684    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-fee9f0-5f799c94f9-msvnw"
Oct 30 19:06:31.835198 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:06:31.834713    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-fee9f0-5f799c94f9-msvnw" event={"ID":"8317ca5e-a974-48a6-ad90-4c08cdd6bf57","Type":"ContainerDied","Data":"62c21d0d47c98243e45554cc6c5a990155fb58bbdc37ab7e94318cba51fea0a8"}
Oct 30 19:06:31.835198 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:06:31.834771    4888 scope.go:122] "RemoveContainer" containerID="97766aff4ad125b43f5cb316986769b30060005de5c68a9e6f8f3dd833e4daf7"
Oct 30 19:06:32.043224 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:06:32.043193    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-fee9f0-5f799c94f9-msvnw"]
Oct 30 19:06:32.055574 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:06:32.055547    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-fee9f0-5f799c94f9-msvnw"]
Oct 30 19:06:33.894141 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:06:33.894101    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8317ca5e-a974-48a6-ad90-4c08cdd6bf57" path="/var/lib/kubelet/pods/8317ca5e-a974-48a6-ad90-4c08cdd6bf57/volumes"
Oct 30 19:06:35.628776 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:06:35.628737    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-jbsw6" containerName="extractor"
Oct 30 19:06:36.950422 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:06:36.950386    4888 patch_prober.go:29] interesting pod/e2e-ipp-a-0c9a2b-openshift-default-85695844fc-wsjcl container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.120:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:06:36.950840 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:06:36.950446    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-a-0c9a2b-openshift-default-85695844fc-wsjcl" podUID="3df2a357-4896-4c99-831b-96fe4cd18fb5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.120:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:06:37.881145 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:06:37.881111    4888 generic.go:356] "Generic (PLEG): container finished" podID="3df2a357-4896-4c99-831b-96fe4cd18fb5" containerID="503b82a28a2b628b8837da4256013f10eb9afed70a785e50e74b1daa6ae0a8fe" exitCode=0
Oct 30 19:06:37.881393 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:06:37.881193    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-0c9a2b-openshift-default-85695844fc-wsjcl" event={"ID":"3df2a357-4896-4c99-831b-96fe4cd18fb5","Type":"ContainerDied","Data":"503b82a28a2b628b8837da4256013f10eb9afed70a785e50e74b1daa6ae0a8fe"}
Oct 30 19:06:37.881691 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:06:37.881673    4888 scope.go:122] "RemoveContainer" containerID="503b82a28a2b628b8837da4256013f10eb9afed70a785e50e74b1daa6ae0a8fe"
Oct 30 19:06:37.883397 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:06:37.883370    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:06:37.883485 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:06:37.883434    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:06:37.883485 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:06:37.883468    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:06:38.908243 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:06:38.908206    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-0c9a2b-openshift-default-85695844fc-wsjcl" event={"ID":"3df2a357-4896-4c99-831b-96fe4cd18fb5","Type":"ContainerStarted","Data":"5d1b1eb66907ff2011cf4aac8c7e07bdb8cc86674b20b3a6011c689327fc5729"}
Oct 30 19:06:39.806127 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:06:39.806086    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-0c9a2b-openshift-default-85695844fc-wsjcl"
Oct 30 19:06:39.806475 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:06:39.806227    4888 patch_prober.go:29] interesting pod/e2e-ipp-a-0c9a2b-openshift-default-85695844fc-wsjcl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.120:15021/healthz/ready\": dial tcp 10.132.0.120:15021: connect: connection refused" start-of-body=
Oct 30 19:06:39.806475 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:06:39.806315    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-0c9a2b-openshift-default-85695844fc-wsjcl" podUID="3df2a357-4896-4c99-831b-96fe4cd18fb5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.120:15021/healthz/ready\": dial tcp 10.132.0.120:15021: connect: connection refused"
Oct 30 19:06:40.805898 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:06:40.805864    4888 patch_prober.go:29] interesting pod/e2e-ipp-a-0c9a2b-openshift-default-85695844fc-wsjcl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.120:15021/healthz/ready\": dial tcp 10.132.0.120:15021: connect: connection refused" start-of-body=
Oct 30 19:06:40.806413 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:06:40.805916    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-0c9a2b-openshift-default-85695844fc-wsjcl" podUID="3df2a357-4896-4c99-831b-96fe4cd18fb5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.120:15021/healthz/ready\": dial tcp 10.132.0.120:15021: connect: connection refused"
Oct 30 19:06:41.483309 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:06:41.469599    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-1acf9b-657dc7b5f-49dc7"]
Oct 30 19:06:41.806867 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:06:41.806769    4888 patch_prober.go:29] interesting pod/e2e-ipp-a-0c9a2b-openshift-default-85695844fc-wsjcl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.120:15021/healthz/ready\": dial tcp 10.132.0.120:15021: connect: connection refused" start-of-body=
Oct 30 19:06:41.807771 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:06:41.807727    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-0c9a2b-openshift-default-85695844fc-wsjcl" podUID="3df2a357-4896-4c99-831b-96fe4cd18fb5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.120:15021/healthz/ready\": dial tcp 10.132.0.120:15021: connect: connection refused"
Oct 30 19:06:42.806317 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:06:42.806248    4888 patch_prober.go:29] interesting pod/e2e-ipp-a-0c9a2b-openshift-default-85695844fc-wsjcl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.120:15021/healthz/ready\": dial tcp 10.132.0.120:15021: connect: connection refused" start-of-body=
Oct 30 19:06:42.806550 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:06:42.806348    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-0c9a2b-openshift-default-85695844fc-wsjcl" podUID="3df2a357-4896-4c99-831b-96fe4cd18fb5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.120:15021/healthz/ready\": dial tcp 10.132.0.120:15021: connect: connection refused"
Oct 30 19:06:43.805652 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:06:43.805618    4888 patch_prober.go:29] interesting pod/e2e-ipp-a-0c9a2b-openshift-default-85695844fc-wsjcl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.120:15021/healthz/ready\": dial tcp 10.132.0.120:15021: connect: connection refused" start-of-body=
Oct 30 19:06:43.806066 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:06:43.805672    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-0c9a2b-openshift-default-85695844fc-wsjcl" podUID="3df2a357-4896-4c99-831b-96fe4cd18fb5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.120:15021/healthz/ready\": dial tcp 10.132.0.120:15021: connect: connection refused"
Oct 30 19:06:44.810137 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:06:44.810105    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-0c9a2b-openshift-default-85695844fc-wsjcl"
Oct 30 19:06:44.810674 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:06:44.810572    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-0c9a2b-openshift-default-85695844fc-wsjcl"
Oct 30 19:06:44.811383 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:06:44.811362    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-0c9a2b-openshift-default-85695844fc-wsjcl"
Oct 30 19:06:46.484821 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:06:46.484779    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-1acf9b-657dc7b5f-49dc7" podUID="8ff6e4b1-7681-4936-b30f-240ece473e12" containerName="payload-pre-processing" containerID="cri-o://062c6759fcc1217ee30141be29322ebd3ffc7ec50197d50f7551ded91ca58de7" gracePeriod=25
Oct 30 19:06:46.767851 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:06:46.767821    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-1acf9b-657dc7b5f-49dc7"
Oct 30 19:06:46.851264 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:06:46.851237    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8ff6e4b1-7681-4936-b30f-240ece473e12-kube-api-access-n2t25\" (UniqueName: \"kubernetes.io/projected/8ff6e4b1-7681-4936-b30f-240ece473e12-kube-api-access-n2t25\") pod \"8ff6e4b1-7681-4936-b30f-240ece473e12\" (UID: \"8ff6e4b1-7681-4936-b30f-240ece473e12\") "
Oct 30 19:06:46.851535 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:06:46.851517    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8ff6e4b1-7681-4936-b30f-240ece473e12-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8ff6e4b1-7681-4936-b30f-240ece473e12-plugins-config-volume\") pod \"8ff6e4b1-7681-4936-b30f-240ece473e12\" (UID: \"8ff6e4b1-7681-4936-b30f-240ece473e12\") "
Oct 30 19:06:46.851863 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:06:46.851841    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8ff6e4b1-7681-4936-b30f-240ece473e12-plugins-config-volume" pod "8ff6e4b1-7681-4936-b30f-240ece473e12" (UID: "8ff6e4b1-7681-4936-b30f-240ece473e12"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:06:46.851984 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:06:46.851971    4888 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8ff6e4b1-7681-4936-b30f-240ece473e12-plugins-config-volume\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:06:46.855048 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:06:46.855019    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8ff6e4b1-7681-4936-b30f-240ece473e12-kube-api-access-n2t25" pod "8ff6e4b1-7681-4936-b30f-240ece473e12" (UID: "8ff6e4b1-7681-4936-b30f-240ece473e12"). InnerVolumeSpecName "kube-api-access-n2t25". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:06:46.953003 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:06:46.952970    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n2t25\" (UniqueName: \"kubernetes.io/projected/8ff6e4b1-7681-4936-b30f-240ece473e12-kube-api-access-n2t25\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:06:46.978803 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:06:46.978775    4888 generic.go:356] "Generic (PLEG): container finished" podID="8ff6e4b1-7681-4936-b30f-240ece473e12" containerID="062c6759fcc1217ee30141be29322ebd3ffc7ec50197d50f7551ded91ca58de7" exitCode=0
Oct 30 19:06:46.978975 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:06:46.978824    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-1acf9b-657dc7b5f-49dc7"
Oct 30 19:06:46.978975 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:06:46.978859    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-1acf9b-657dc7b5f-49dc7" event={"ID":"8ff6e4b1-7681-4936-b30f-240ece473e12","Type":"ContainerDied","Data":"062c6759fcc1217ee30141be29322ebd3ffc7ec50197d50f7551ded91ca58de7"}
Oct 30 19:06:46.978975 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:06:46.978933    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-1acf9b-657dc7b5f-49dc7" event={"ID":"8ff6e4b1-7681-4936-b30f-240ece473e12","Type":"ContainerDied","Data":"5d463c695e95d9634c790473ba1b952f0b5c80bd9ac87968011fe534575869cc"}
Oct 30 19:06:46.978975 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:06:46.978958    4888 scope.go:122] "RemoveContainer" containerID="062c6759fcc1217ee30141be29322ebd3ffc7ec50197d50f7551ded91ca58de7"
Oct 30 19:06:46.991314 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:06:46.991259    4888 scope.go:122] "RemoveContainer" containerID="062c6759fcc1217ee30141be29322ebd3ffc7ec50197d50f7551ded91ca58de7"
Oct 30 19:06:46.991553 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:06:46.991535    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"062c6759fcc1217ee30141be29322ebd3ffc7ec50197d50f7551ded91ca58de7\": container with ID starting with 062c6759fcc1217ee30141be29322ebd3ffc7ec50197d50f7551ded91ca58de7 not found: ID does not exist" containerID="062c6759fcc1217ee30141be29322ebd3ffc7ec50197d50f7551ded91ca58de7"
Oct 30 19:06:46.991624 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:06:46.991566    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"062c6759fcc1217ee30141be29322ebd3ffc7ec50197d50f7551ded91ca58de7"} err="failed to get container status \"062c6759fcc1217ee30141be29322ebd3ffc7ec50197d50f7551ded91ca58de7\": rpc error: code = NotFound desc = could not find container \"062c6759fcc1217ee30141be29322ebd3ffc7ec50197d50f7551ded91ca58de7\": container with ID starting with 062c6759fcc1217ee30141be29322ebd3ffc7ec50197d50f7551ded91ca58de7 not found: ID does not exist"
Oct 30 19:06:47.153636 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:06:47.153598    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-1acf9b-657dc7b5f-49dc7"]
Oct 30 19:06:47.158935 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:06:47.158908    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-1acf9b-657dc7b5f-49dc7"]
Oct 30 19:06:47.893078 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:06:47.893043    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8ff6e4b1-7681-4936-b30f-240ece473e12" path="/var/lib/kubelet/pods/8ff6e4b1-7681-4936-b30f-240ece473e12/volumes"
Oct 30 19:06:55.907768 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:06:55.907733    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-rjmdz"]
Oct 30 19:07:02.002950 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:07:02.002908    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-82305a-openshift-default-7cdd8f9877-sf7w8"]
Oct 30 19:07:02.003590 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:07:02.003564    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-a-82305a-openshift-default-7cdd8f9877-sf7w8" podUID="f9c90752-5508-445c-823e-3d9ea37f54b2" containerName="istio-proxy" containerID="cri-o://ded22ec4462b0c07cf005e7b484f8f79e2938b72652d3f2c232932dd98c96b7b" gracePeriod=30
Oct 30 19:07:02.941179 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:07:02.941143    4888 patch_prober.go:29] interesting pod/e2e-ipp-a-82305a-openshift-default-7cdd8f9877-sf7w8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 19:07:02.941383 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:07:02.941205    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-a-82305a-openshift-default-7cdd8f9877-sf7w8" podUID="f9c90752-5508-445c-823e-3d9ea37f54b2" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 19:07:05.620192 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:07:05.620151    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-jbsw6" containerName="extractor"
Oct 30 19:07:07.263745 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:07:07.263712    4888 generic.go:356] "Generic (PLEG): container finished" podID="29c22c2d-64fe-476b-8b35-eab525253a5c" containerID="78524c76cd12e11a69585cf1dce2977bb240bc37702e952b6653eadd2f6a393d" exitCode=0
Oct 30 19:07:07.264236 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:07:07.263793    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-0ee379-openshift-default-647f98d679-s29zb" event={"ID":"29c22c2d-64fe-476b-8b35-eab525253a5c","Type":"ContainerDied","Data":"78524c76cd12e11a69585cf1dce2977bb240bc37702e952b6653eadd2f6a393d"}
Oct 30 19:07:07.264398 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:07:07.264344    4888 scope.go:122] "RemoveContainer" containerID="78524c76cd12e11a69585cf1dce2977bb240bc37702e952b6653eadd2f6a393d"
Oct 30 19:07:07.266389 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:07:07.266361    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:07:07.266528 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:07:07.266456    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:07:07.266528 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:07:07.266500    4888 generic.go:356] "Generic (PLEG): container finished" podID="f9c90752-5508-445c-823e-3d9ea37f54b2" containerID="ded22ec4462b0c07cf005e7b484f8f79e2938b72652d3f2c232932dd98c96b7b" exitCode=0
Oct 30 19:07:07.266649 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:07:07.266525    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-82305a-openshift-default-7cdd8f9877-sf7w8" event={"ID":"f9c90752-5508-445c-823e-3d9ea37f54b2","Type":"ContainerDied","Data":"ded22ec4462b0c07cf005e7b484f8f79e2938b72652d3f2c232932dd98c96b7b"}
Oct 30 19:07:07.266649 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:07:07.266580    4888 scope.go:122] "RemoveContainer" containerID="657ccac8b2eaa14fb21da70f9b5bf1b4f53aced0226bff2c13e5c64eb9e8ca8b"
Oct 30 19:07:07.266649 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:07:07.266507    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:07:07.337497 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:07:07.337477    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-82305a-openshift-default-7cdd8f9877-sf7w8"
Oct 30 19:07:07.526056 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:07:07.496220    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f9c90752-5508-445c-823e-3d9ea37f54b2-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f9c90752-5508-445c-823e-3d9ea37f54b2-istiod-ca-cert\") pod \"f9c90752-5508-445c-823e-3d9ea37f54b2\" (UID: \"f9c90752-5508-445c-823e-3d9ea37f54b2\") "
Oct 30 19:07:07.526056 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:07:07.496262    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/f9c90752-5508-445c-823e-3d9ea37f54b2-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f9c90752-5508-445c-823e-3d9ea37f54b2-istio-podinfo\") pod \"f9c90752-5508-445c-823e-3d9ea37f54b2\" (UID: \"f9c90752-5508-445c-823e-3d9ea37f54b2\") "
Oct 30 19:07:07.526056 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:07:07.496366    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f9c90752-5508-445c-823e-3d9ea37f54b2-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f9c90752-5508-445c-823e-3d9ea37f54b2-workload-certs\") pod \"f9c90752-5508-445c-823e-3d9ea37f54b2\" (UID: \"f9c90752-5508-445c-823e-3d9ea37f54b2\") "
Oct 30 19:07:07.526056 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:07:07.496450    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f9c90752-5508-445c-823e-3d9ea37f54b2-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f9c90752-5508-445c-823e-3d9ea37f54b2-istio-envoy\") pod \"f9c90752-5508-445c-823e-3d9ea37f54b2\" (UID: \"f9c90752-5508-445c-823e-3d9ea37f54b2\") "
Oct 30 19:07:07.526056 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:07:07.496482    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f9c90752-5508-445c-823e-3d9ea37f54b2-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f9c90752-5508-445c-823e-3d9ea37f54b2-istio-data\") pod \"f9c90752-5508-445c-823e-3d9ea37f54b2\" (UID: \"f9c90752-5508-445c-823e-3d9ea37f54b2\") "
Oct 30 19:07:07.526056 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:07:07.496537    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f9c90752-5508-445c-823e-3d9ea37f54b2-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9c90752-5508-445c-823e-3d9ea37f54b2-credential-socket\") pod \"f9c90752-5508-445c-823e-3d9ea37f54b2\" (UID: \"f9c90752-5508-445c-823e-3d9ea37f54b2\") "
Oct 30 19:07:07.526056 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:07:07.496568    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f9c90752-5508-445c-823e-3d9ea37f54b2-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9c90752-5508-445c-823e-3d9ea37f54b2-workload-socket\") pod \"f9c90752-5508-445c-823e-3d9ea37f54b2\" (UID: \"f9c90752-5508-445c-823e-3d9ea37f54b2\") "
Oct 30 19:07:07.526056 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:07:07.496640    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f9c90752-5508-445c-823e-3d9ea37f54b2-kube-api-access-xtjhn\" (UniqueName: \"kubernetes.io/projected/f9c90752-5508-445c-823e-3d9ea37f54b2-kube-api-access-xtjhn\") pod \"f9c90752-5508-445c-823e-3d9ea37f54b2\" (UID: \"f9c90752-5508-445c-823e-3d9ea37f54b2\") "
Oct 30 19:07:07.526056 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:07:07.496750    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f9c90752-5508-445c-823e-3d9ea37f54b2-istio-token\" (UniqueName: \"kubernetes.io/projected/f9c90752-5508-445c-823e-3d9ea37f54b2-istio-token\") pod \"f9c90752-5508-445c-823e-3d9ea37f54b2\" (UID: \"f9c90752-5508-445c-823e-3d9ea37f54b2\") "
Oct 30 19:07:07.526056 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:07:07.500208    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f9c90752-5508-445c-823e-3d9ea37f54b2-istio-token" pod "f9c90752-5508-445c-823e-3d9ea37f54b2" (UID: "f9c90752-5508-445c-823e-3d9ea37f54b2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:07:07.526056 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:07:07.500779    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f9c90752-5508-445c-823e-3d9ea37f54b2-istiod-ca-cert" pod "f9c90752-5508-445c-823e-3d9ea37f54b2" (UID: "f9c90752-5508-445c-823e-3d9ea37f54b2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:07:07.526056 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:07:07.508384    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f9c90752-5508-445c-823e-3d9ea37f54b2-workload-certs" pod "f9c90752-5508-445c-823e-3d9ea37f54b2" (UID: "f9c90752-5508-445c-823e-3d9ea37f54b2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:07:07.526056 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:07:07.511506    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f9c90752-5508-445c-823e-3d9ea37f54b2-istio-envoy" pod "f9c90752-5508-445c-823e-3d9ea37f54b2" (UID: "f9c90752-5508-445c-823e-3d9ea37f54b2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:07:07.526056 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:07:07.511820    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f9c90752-5508-445c-823e-3d9ea37f54b2-istio-data" pod "f9c90752-5508-445c-823e-3d9ea37f54b2" (UID: "f9c90752-5508-445c-823e-3d9ea37f54b2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:07:07.526056 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:07:07.512064    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f9c90752-5508-445c-823e-3d9ea37f54b2-credential-socket" pod "f9c90752-5508-445c-823e-3d9ea37f54b2" (UID: "f9c90752-5508-445c-823e-3d9ea37f54b2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:07:07.526056 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:07:07.512324    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f9c90752-5508-445c-823e-3d9ea37f54b2-workload-socket" pod "f9c90752-5508-445c-823e-3d9ea37f54b2" (UID: "f9c90752-5508-445c-823e-3d9ea37f54b2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:07:07.526056 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:07:07.515201    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f9c90752-5508-445c-823e-3d9ea37f54b2-kube-api-access-xtjhn" pod "f9c90752-5508-445c-823e-3d9ea37f54b2" (UID: "f9c90752-5508-445c-823e-3d9ea37f54b2"). InnerVolumeSpecName "kube-api-access-xtjhn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:07:07.526056 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:07:07.515268    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f9c90752-5508-445c-823e-3d9ea37f54b2-istio-podinfo" pod "f9c90752-5508-445c-823e-3d9ea37f54b2" (UID: "f9c90752-5508-445c-823e-3d9ea37f54b2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:07:07.599864 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:07:07.599680    4888 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f9c90752-5508-445c-823e-3d9ea37f54b2-istio-envoy\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:07:07.599864 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:07:07.599718    4888 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f9c90752-5508-445c-823e-3d9ea37f54b2-istio-data\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:07:07.599864 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:07:07.599733    4888 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9c90752-5508-445c-823e-3d9ea37f54b2-credential-socket\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:07:07.599864 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:07:07.599748    4888 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9c90752-5508-445c-823e-3d9ea37f54b2-workload-socket\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:07:07.599864 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:07:07.599762    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xtjhn\" (UniqueName: \"kubernetes.io/projected/f9c90752-5508-445c-823e-3d9ea37f54b2-kube-api-access-xtjhn\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:07:07.599864 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:07:07.599777    4888 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f9c90752-5508-445c-823e-3d9ea37f54b2-istio-token\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:07:07.599864 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:07:07.599789    4888 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f9c90752-5508-445c-823e-3d9ea37f54b2-istiod-ca-cert\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:07:07.599864 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:07:07.599803    4888 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f9c90752-5508-445c-823e-3d9ea37f54b2-istio-podinfo\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:07:07.599864 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:07:07.599816    4888 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f9c90752-5508-445c-823e-3d9ea37f54b2-workload-certs\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:07:08.278743 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:07:08.278661    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-0ee379-openshift-default-647f98d679-s29zb" event={"ID":"29c22c2d-64fe-476b-8b35-eab525253a5c","Type":"ContainerStarted","Data":"83c57b222565e88d1637c9290156e822aca275eb7d71a162012e2c9bdd86d1d4"}
Oct 30 19:07:08.280488 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:07:08.280464    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-82305a-openshift-default-7cdd8f9877-sf7w8"
Oct 30 19:07:08.280623 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:07:08.280490    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-82305a-openshift-default-7cdd8f9877-sf7w8" event={"ID":"f9c90752-5508-445c-823e-3d9ea37f54b2","Type":"ContainerDied","Data":"1055798aa2d74b89965a965515468e5d5af8c9f481bcb6c4bd8acc1a13166aa7"}
Oct 30 19:07:08.280623 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:07:08.280540    4888 scope.go:122] "RemoveContainer" containerID="ded22ec4462b0c07cf005e7b484f8f79e2938b72652d3f2c232932dd98c96b7b"
Oct 30 19:07:08.562956 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:07:08.562873    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-82305a-openshift-default-7cdd8f9877-sf7w8"]
Oct 30 19:07:08.580309 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:07:08.579006    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-a-82305a-openshift-default-7cdd8f9877-sf7w8"]
Oct 30 19:07:08.646880 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:07:08.646844    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-0ee379-openshift-default-647f98d679-s29zb"
Oct 30 19:07:08.647198 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:07:08.647175    4888 patch_prober.go:29] interesting pod/e2e-ipp-b-0ee379-openshift-default-647f98d679-s29zb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.121:15021/healthz/ready\": dial tcp 10.132.0.121:15021: connect: connection refused" start-of-body=
Oct 30 19:07:08.647264 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:07:08.647226    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-0ee379-openshift-default-647f98d679-s29zb" podUID="29c22c2d-64fe-476b-8b35-eab525253a5c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.121:15021/healthz/ready\": dial tcp 10.132.0.121:15021: connect: connection refused"
Oct 30 19:07:09.646833 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:07:09.646796    4888 patch_prober.go:29] interesting pod/e2e-ipp-b-0ee379-openshift-default-647f98d679-s29zb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.121:15021/healthz/ready\": dial tcp 10.132.0.121:15021: connect: connection refused" start-of-body=
Oct 30 19:07:09.647254 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:07:09.646857    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-0ee379-openshift-default-647f98d679-s29zb" podUID="29c22c2d-64fe-476b-8b35-eab525253a5c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.121:15021/healthz/ready\": dial tcp 10.132.0.121:15021: connect: connection refused"
Oct 30 19:07:09.909783 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:07:09.909705    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f9c90752-5508-445c-823e-3d9ea37f54b2" path="/var/lib/kubelet/pods/f9c90752-5508-445c-823e-3d9ea37f54b2/volumes"
Oct 30 19:07:10.647560 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:07:10.647525    4888 patch_prober.go:29] interesting pod/e2e-ipp-b-0ee379-openshift-default-647f98d679-s29zb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.121:15021/healthz/ready\": dial tcp 10.132.0.121:15021: connect: connection refused" start-of-body=
Oct 30 19:07:10.647955 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:07:10.647576    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-0ee379-openshift-default-647f98d679-s29zb" podUID="29c22c2d-64fe-476b-8b35-eab525253a5c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.121:15021/healthz/ready\": dial tcp 10.132.0.121:15021: connect: connection refused"
Oct 30 19:07:11.647486 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:07:11.647449    4888 patch_prober.go:29] interesting pod/e2e-ipp-b-0ee379-openshift-default-647f98d679-s29zb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.121:15021/healthz/ready\": dial tcp 10.132.0.121:15021: connect: connection refused" start-of-body=
Oct 30 19:07:11.647669 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:07:11.647504    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-0ee379-openshift-default-647f98d679-s29zb" podUID="29c22c2d-64fe-476b-8b35-eab525253a5c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.121:15021/healthz/ready\": dial tcp 10.132.0.121:15021: connect: connection refused"
Oct 30 19:07:12.647205 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:07:12.647164    4888 patch_prober.go:29] interesting pod/e2e-ipp-b-0ee379-openshift-default-647f98d679-s29zb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.121:15021/healthz/ready\": dial tcp 10.132.0.121:15021: connect: connection refused" start-of-body=
Oct 30 19:07:12.647431 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:07:12.647220    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-0ee379-openshift-default-647f98d679-s29zb" podUID="29c22c2d-64fe-476b-8b35-eab525253a5c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.121:15021/healthz/ready\": dial tcp 10.132.0.121:15021: connect: connection refused"
Oct 30 19:07:13.647638 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:07:13.647602    4888 patch_prober.go:29] interesting pod/e2e-ipp-b-0ee379-openshift-default-647f98d679-s29zb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.121:15021/healthz/ready\": dial tcp 10.132.0.121:15021: connect: connection refused" start-of-body=
Oct 30 19:07:13.648044 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:07:13.647654    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-0ee379-openshift-default-647f98d679-s29zb" podUID="29c22c2d-64fe-476b-8b35-eab525253a5c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.121:15021/healthz/ready\": dial tcp 10.132.0.121:15021: connect: connection refused"
Oct 30 19:07:14.001878 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:07:14.001794    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-0ee379-openshift-default-647f98d679-s29zb"
Oct 30 19:07:14.652712 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:07:14.652684    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-0ee379-openshift-default-647f98d679-s29zb"
Oct 30 19:07:14.653758 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:07:14.653740    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-0ee379-openshift-default-647f98d679-s29zb"
Oct 30 19:07:35.089230 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:07:35.089189    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-fee9f0-openshift-default-68c7499775-l68kt"]
Oct 30 19:07:35.089715 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:07:35.089464    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-b-fee9f0-openshift-default-68c7499775-l68kt" podUID="94364f5b-4740-430f-87df-51e9b656d963" containerName="istio-proxy" containerID="cri-o://43090d61fd202d9007705e78bc00d336c6094a6bcbe3288cef75d606cc8ed3cb" gracePeriod=30
Oct 30 19:07:35.624475 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:07:35.621117    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-jbsw6" containerName="extractor"
Oct 30 19:07:40.373886 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:07:40.373858    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-fee9f0-openshift-default-68c7499775-l68kt"
Oct 30 19:07:40.527028 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:07:40.526989    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/94364f5b-4740-430f-87df-51e9b656d963-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/94364f5b-4740-430f-87df-51e9b656d963-istiod-ca-cert\") pod \"94364f5b-4740-430f-87df-51e9b656d963\" (UID: \"94364f5b-4740-430f-87df-51e9b656d963\") "
Oct 30 19:07:40.527235 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:07:40.527050    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/94364f5b-4740-430f-87df-51e9b656d963-istio-token\" (UniqueName: \"kubernetes.io/projected/94364f5b-4740-430f-87df-51e9b656d963-istio-token\") pod \"94364f5b-4740-430f-87df-51e9b656d963\" (UID: \"94364f5b-4740-430f-87df-51e9b656d963\") "
Oct 30 19:07:40.527235 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:07:40.527135    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/94364f5b-4740-430f-87df-51e9b656d963-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/94364f5b-4740-430f-87df-51e9b656d963-istio-data\") pod \"94364f5b-4740-430f-87df-51e9b656d963\" (UID: \"94364f5b-4740-430f-87df-51e9b656d963\") "
Oct 30 19:07:40.527235 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:07:40.527205    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/94364f5b-4740-430f-87df-51e9b656d963-kube-api-access-4czzj\" (UniqueName: \"kubernetes.io/projected/94364f5b-4740-430f-87df-51e9b656d963-kube-api-access-4czzj\") pod \"94364f5b-4740-430f-87df-51e9b656d963\" (UID: \"94364f5b-4740-430f-87df-51e9b656d963\") "
Oct 30 19:07:40.527495 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:07:40.527279    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/94364f5b-4740-430f-87df-51e9b656d963-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/94364f5b-4740-430f-87df-51e9b656d963-workload-socket\") pod \"94364f5b-4740-430f-87df-51e9b656d963\" (UID: \"94364f5b-4740-430f-87df-51e9b656d963\") "
Oct 30 19:07:40.527495 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:07:40.527366    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/94364f5b-4740-430f-87df-51e9b656d963-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/94364f5b-4740-430f-87df-51e9b656d963-workload-certs\") pod \"94364f5b-4740-430f-87df-51e9b656d963\" (UID: \"94364f5b-4740-430f-87df-51e9b656d963\") "
Oct 30 19:07:40.527495 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:07:40.527395    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/94364f5b-4740-430f-87df-51e9b656d963-istio-data" pod "94364f5b-4740-430f-87df-51e9b656d963" (UID: "94364f5b-4740-430f-87df-51e9b656d963"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:07:40.527495 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:07:40.527419    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/94364f5b-4740-430f-87df-51e9b656d963-istiod-ca-cert" pod "94364f5b-4740-430f-87df-51e9b656d963" (UID: "94364f5b-4740-430f-87df-51e9b656d963"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:07:40.527676 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:07:40.527508    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/94364f5b-4740-430f-87df-51e9b656d963-workload-socket" pod "94364f5b-4740-430f-87df-51e9b656d963" (UID: "94364f5b-4740-430f-87df-51e9b656d963"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:07:40.527676 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:07:40.527537    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/94364f5b-4740-430f-87df-51e9b656d963-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/94364f5b-4740-430f-87df-51e9b656d963-credential-socket\") pod \"94364f5b-4740-430f-87df-51e9b656d963\" (UID: \"94364f5b-4740-430f-87df-51e9b656d963\") "
Oct 30 19:07:40.527676 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:07:40.527567    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/94364f5b-4740-430f-87df-51e9b656d963-workload-certs" pod "94364f5b-4740-430f-87df-51e9b656d963" (UID: "94364f5b-4740-430f-87df-51e9b656d963"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:07:40.527676 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:07:40.527579    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/94364f5b-4740-430f-87df-51e9b656d963-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/94364f5b-4740-430f-87df-51e9b656d963-istio-envoy\") pod \"94364f5b-4740-430f-87df-51e9b656d963\" (UID: \"94364f5b-4740-430f-87df-51e9b656d963\") "
Oct 30 19:07:40.527676 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:07:40.527626    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/94364f5b-4740-430f-87df-51e9b656d963-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/94364f5b-4740-430f-87df-51e9b656d963-istio-podinfo\") pod \"94364f5b-4740-430f-87df-51e9b656d963\" (UID: \"94364f5b-4740-430f-87df-51e9b656d963\") "
Oct 30 19:07:40.527942 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:07:40.527698    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/94364f5b-4740-430f-87df-51e9b656d963-credential-socket" pod "94364f5b-4740-430f-87df-51e9b656d963" (UID: "94364f5b-4740-430f-87df-51e9b656d963"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:07:40.528377 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:07:40.528351    4888 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/94364f5b-4740-430f-87df-51e9b656d963-istiod-ca-cert\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:07:40.528653 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:07:40.528381    4888 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/94364f5b-4740-430f-87df-51e9b656d963-istio-data\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:07:40.528653 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:07:40.528396    4888 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/94364f5b-4740-430f-87df-51e9b656d963-workload-socket\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:07:40.528653 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:07:40.528411    4888 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/94364f5b-4740-430f-87df-51e9b656d963-workload-certs\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:07:40.528653 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:07:40.528425    4888 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/94364f5b-4740-430f-87df-51e9b656d963-credential-socket\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:07:40.529569 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:07:40.529545    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/94364f5b-4740-430f-87df-51e9b656d963-kube-api-access-4czzj" pod "94364f5b-4740-430f-87df-51e9b656d963" (UID: "94364f5b-4740-430f-87df-51e9b656d963"). InnerVolumeSpecName "kube-api-access-4czzj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:07:40.529730 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:07:40.529703    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/94364f5b-4740-430f-87df-51e9b656d963-istio-token" pod "94364f5b-4740-430f-87df-51e9b656d963" (UID: "94364f5b-4740-430f-87df-51e9b656d963"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:07:40.529846 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:07:40.529773    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/94364f5b-4740-430f-87df-51e9b656d963-istio-envoy" pod "94364f5b-4740-430f-87df-51e9b656d963" (UID: "94364f5b-4740-430f-87df-51e9b656d963"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:07:40.530068 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:07:40.530054    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/94364f5b-4740-430f-87df-51e9b656d963-istio-podinfo" pod "94364f5b-4740-430f-87df-51e9b656d963" (UID: "94364f5b-4740-430f-87df-51e9b656d963"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:07:40.630076 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:07:40.630004    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4czzj\" (UniqueName: \"kubernetes.io/projected/94364f5b-4740-430f-87df-51e9b656d963-kube-api-access-4czzj\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:07:40.630076 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:07:40.630032    4888 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/94364f5b-4740-430f-87df-51e9b656d963-istio-envoy\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:07:40.630076 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:07:40.630042    4888 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/94364f5b-4740-430f-87df-51e9b656d963-istio-podinfo\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:07:40.630076 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:07:40.630052    4888 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/94364f5b-4740-430f-87df-51e9b656d963-istio-token\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:07:40.647829 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:07:40.647802    4888 generic.go:356] "Generic (PLEG): container finished" podID="94364f5b-4740-430f-87df-51e9b656d963" containerID="43090d61fd202d9007705e78bc00d336c6094a6bcbe3288cef75d606cc8ed3cb" exitCode=0
Oct 30 19:07:40.647976 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:07:40.647852    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-fee9f0-openshift-default-68c7499775-l68kt"
Oct 30 19:07:40.647976 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:07:40.647888    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-fee9f0-openshift-default-68c7499775-l68kt" event={"ID":"94364f5b-4740-430f-87df-51e9b656d963","Type":"ContainerDied","Data":"43090d61fd202d9007705e78bc00d336c6094a6bcbe3288cef75d606cc8ed3cb"}
Oct 30 19:07:40.647976 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:07:40.647950    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-fee9f0-openshift-default-68c7499775-l68kt" event={"ID":"94364f5b-4740-430f-87df-51e9b656d963","Type":"ContainerDied","Data":"1d83ecaa3b716617a420f72bbdf384299354d4240e20cbb82c7ffa1ea92dae35"}
Oct 30 19:07:40.647976 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:07:40.647974    4888 scope.go:122] "RemoveContainer" containerID="43090d61fd202d9007705e78bc00d336c6094a6bcbe3288cef75d606cc8ed3cb"
Oct 30 19:07:40.659765 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:07:40.659741    4888 scope.go:122] "RemoveContainer" containerID="fb5a4cc3729eff702853803d5f2bf01fbebe67656cad3e6f808e6fb6472e8448"
Oct 30 19:07:40.669368 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:07:40.669347    4888 scope.go:122] "RemoveContainer" containerID="43090d61fd202d9007705e78bc00d336c6094a6bcbe3288cef75d606cc8ed3cb"
Oct 30 19:07:40.669732 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:07:40.669700    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"43090d61fd202d9007705e78bc00d336c6094a6bcbe3288cef75d606cc8ed3cb\": container with ID starting with 43090d61fd202d9007705e78bc00d336c6094a6bcbe3288cef75d606cc8ed3cb not found: ID does not exist" containerID="43090d61fd202d9007705e78bc00d336c6094a6bcbe3288cef75d606cc8ed3cb"
Oct 30 19:07:40.669836 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:07:40.669744    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"43090d61fd202d9007705e78bc00d336c6094a6bcbe3288cef75d606cc8ed3cb"} err="failed to get container status \"43090d61fd202d9007705e78bc00d336c6094a6bcbe3288cef75d606cc8ed3cb\": rpc error: code = NotFound desc = could not find container \"43090d61fd202d9007705e78bc00d336c6094a6bcbe3288cef75d606cc8ed3cb\": container with ID starting with 43090d61fd202d9007705e78bc00d336c6094a6bcbe3288cef75d606cc8ed3cb not found: ID does not exist"
Oct 30 19:07:40.669836 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:07:40.669770    4888 scope.go:122] "RemoveContainer" containerID="fb5a4cc3729eff702853803d5f2bf01fbebe67656cad3e6f808e6fb6472e8448"
Oct 30 19:07:40.670095 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:07:40.670071    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fb5a4cc3729eff702853803d5f2bf01fbebe67656cad3e6f808e6fb6472e8448\": container with ID starting with fb5a4cc3729eff702853803d5f2bf01fbebe67656cad3e6f808e6fb6472e8448 not found: ID does not exist" containerID="fb5a4cc3729eff702853803d5f2bf01fbebe67656cad3e6f808e6fb6472e8448"
Oct 30 19:07:40.670167 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:07:40.670106    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fb5a4cc3729eff702853803d5f2bf01fbebe67656cad3e6f808e6fb6472e8448"} err="failed to get container status \"fb5a4cc3729eff702853803d5f2bf01fbebe67656cad3e6f808e6fb6472e8448\": rpc error: code = NotFound desc = could not find container \"fb5a4cc3729eff702853803d5f2bf01fbebe67656cad3e6f808e6fb6472e8448\": container with ID starting with fb5a4cc3729eff702853803d5f2bf01fbebe67656cad3e6f808e6fb6472e8448 not found: ID does not exist"
Oct 30 19:07:40.824607 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:07:40.824549    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-fee9f0-openshift-default-68c7499775-l68kt"]
Oct 30 19:07:40.849585 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:07:40.849554    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-b-fee9f0-openshift-default-68c7499775-l68kt"]
Oct 30 19:07:41.895441 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:07:41.894163    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="94364f5b-4740-430f-87df-51e9b656d963" path="/var/lib/kubelet/pods/94364f5b-4740-430f-87df-51e9b656d963/volumes"
Oct 30 19:07:42.057345 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:07:42.055303    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-rjmdz"]
Oct 30 19:08:05.624759 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:08:05.620821    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-jbsw6" containerName="extractor"
Oct 30 19:08:35.620656 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:08:35.620620    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-jbsw6" containerName="extractor"
Oct 30 19:08:39.654300 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:08:39.654244    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-rjmdz"]
Oct 30 19:08:47.874829 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:08:47.874795    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-66db998887-jsv6r"]
Oct 30 19:08:47.875265 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:08:47.875017    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-jsv6r" podUID="74852e36-70a6-48f9-8f69-61f180f3c772" containerName="istio-proxy" containerID="cri-o://80f094a9a59c2bca8846d63aeb73518ec2716f8f23a7cb84ebe1f4cff0f56518" gracePeriod=30
Oct 30 19:08:49.503434 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:08:49.503376    4888 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-66db998887-jsv6r container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 19:08:49.503835 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:08:49.503462    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-jsv6r" podUID="74852e36-70a6-48f9-8f69-61f180f3c772" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 19:08:49.963901 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:08:49.963870    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-5h9tz"]
Oct 30 19:08:49.964097 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:08:49.964060    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-5h9tz" podUID="f5ffda80-881d-406d-8ef1-3664bf010a67" containerName="istio-proxy" containerID="cri-o://57b68653ad3ea8bc9ed04b8eb1be80126b104daa84ab371dfa97f97a8b90d314" gracePeriod=30
Oct 30 19:08:52.227263 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:08:52.227223    4888 patch_prober.go:29] interesting pod/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-5h9tz container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 19:08:52.227671 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:08:52.227327    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-5h9tz" podUID="f5ffda80-881d-406d-8ef1-3664bf010a67" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 19:08:53.160747 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:08:53.160687    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-jsv6r"
Oct 30 19:08:53.263334 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:08:53.263259    4888 generic.go:356] "Generic (PLEG): container finished" podID="74852e36-70a6-48f9-8f69-61f180f3c772" containerID="80f094a9a59c2bca8846d63aeb73518ec2716f8f23a7cb84ebe1f4cff0f56518" exitCode=0
Oct 30 19:08:53.263334 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:08:53.263319    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-jsv6r" event={"ID":"74852e36-70a6-48f9-8f69-61f180f3c772","Type":"ContainerDied","Data":"80f094a9a59c2bca8846d63aeb73518ec2716f8f23a7cb84ebe1f4cff0f56518"}
Oct 30 19:08:53.263768 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:08:53.263398    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-jsv6r" event={"ID":"74852e36-70a6-48f9-8f69-61f180f3c772","Type":"ContainerDied","Data":"94099d30ccf31481f785032f18f3152efed97399a26ac81276ccf3c339461322"}
Oct 30 19:08:53.263768 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:08:53.263327    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-jsv6r"
Oct 30 19:08:53.263768 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:08:53.263430    4888 scope.go:122] "RemoveContainer" containerID="80f094a9a59c2bca8846d63aeb73518ec2716f8f23a7cb84ebe1f4cff0f56518"
Oct 30 19:08:53.271630 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:08:53.271604    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/74852e36-70a6-48f9-8f69-61f180f3c772-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/74852e36-70a6-48f9-8f69-61f180f3c772-istio-data\") pod \"74852e36-70a6-48f9-8f69-61f180f3c772\" (UID: \"74852e36-70a6-48f9-8f69-61f180f3c772\") "
Oct 30 19:08:53.271748 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:08:53.271646    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/74852e36-70a6-48f9-8f69-61f180f3c772-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/74852e36-70a6-48f9-8f69-61f180f3c772-workload-certs\") pod \"74852e36-70a6-48f9-8f69-61f180f3c772\" (UID: \"74852e36-70a6-48f9-8f69-61f180f3c772\") "
Oct 30 19:08:53.271748 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:08:53.271676    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/74852e36-70a6-48f9-8f69-61f180f3c772-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/74852e36-70a6-48f9-8f69-61f180f3c772-istio-podinfo\") pod \"74852e36-70a6-48f9-8f69-61f180f3c772\" (UID: \"74852e36-70a6-48f9-8f69-61f180f3c772\") "
Oct 30 19:08:53.271748 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:08:53.271729    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/74852e36-70a6-48f9-8f69-61f180f3c772-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/74852e36-70a6-48f9-8f69-61f180f3c772-credential-socket\") pod \"74852e36-70a6-48f9-8f69-61f180f3c772\" (UID: \"74852e36-70a6-48f9-8f69-61f180f3c772\") "
Oct 30 19:08:53.271856 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:08:53.271788    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/74852e36-70a6-48f9-8f69-61f180f3c772-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/74852e36-70a6-48f9-8f69-61f180f3c772-istio-envoy\") pod \"74852e36-70a6-48f9-8f69-61f180f3c772\" (UID: \"74852e36-70a6-48f9-8f69-61f180f3c772\") "
Oct 30 19:08:53.271930 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:08:53.271905    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/74852e36-70a6-48f9-8f69-61f180f3c772-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/74852e36-70a6-48f9-8f69-61f180f3c772-workload-socket\") pod \"74852e36-70a6-48f9-8f69-61f180f3c772\" (UID: \"74852e36-70a6-48f9-8f69-61f180f3c772\") "
Oct 30 19:08:53.272051 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:08:53.271978    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/74852e36-70a6-48f9-8f69-61f180f3c772-istio-token\" (UniqueName: \"kubernetes.io/projected/74852e36-70a6-48f9-8f69-61f180f3c772-istio-token\") pod \"74852e36-70a6-48f9-8f69-61f180f3c772\" (UID: \"74852e36-70a6-48f9-8f69-61f180f3c772\") "
Oct 30 19:08:53.272051 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:08:53.272046    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/74852e36-70a6-48f9-8f69-61f180f3c772-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/74852e36-70a6-48f9-8f69-61f180f3c772-istiod-ca-cert\") pod \"74852e36-70a6-48f9-8f69-61f180f3c772\" (UID: \"74852e36-70a6-48f9-8f69-61f180f3c772\") "
Oct 30 19:08:53.272188 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:08:53.272073    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/74852e36-70a6-48f9-8f69-61f180f3c772-kube-api-access-pzrqj\" (UniqueName: \"kubernetes.io/projected/74852e36-70a6-48f9-8f69-61f180f3c772-kube-api-access-pzrqj\") pod \"74852e36-70a6-48f9-8f69-61f180f3c772\" (UID: \"74852e36-70a6-48f9-8f69-61f180f3c772\") "
Oct 30 19:08:53.273882 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:08:53.273587    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/74852e36-70a6-48f9-8f69-61f180f3c772-credential-socket" pod "74852e36-70a6-48f9-8f69-61f180f3c772" (UID: "74852e36-70a6-48f9-8f69-61f180f3c772"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:08:53.273882 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:08:53.273630    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/74852e36-70a6-48f9-8f69-61f180f3c772-workload-socket" pod "74852e36-70a6-48f9-8f69-61f180f3c772" (UID: "74852e36-70a6-48f9-8f69-61f180f3c772"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:08:53.273882 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:08:53.273778    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/74852e36-70a6-48f9-8f69-61f180f3c772-istio-data" pod "74852e36-70a6-48f9-8f69-61f180f3c772" (UID: "74852e36-70a6-48f9-8f69-61f180f3c772"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:08:53.274417 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:08:53.274333    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/74852e36-70a6-48f9-8f69-61f180f3c772-workload-certs" pod "74852e36-70a6-48f9-8f69-61f180f3c772" (UID: "74852e36-70a6-48f9-8f69-61f180f3c772"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:08:53.274795 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:08:53.274773    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/74852e36-70a6-48f9-8f69-61f180f3c772-istiod-ca-cert" pod "74852e36-70a6-48f9-8f69-61f180f3c772" (UID: "74852e36-70a6-48f9-8f69-61f180f3c772"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:08:53.275513 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:08:53.275452    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/74852e36-70a6-48f9-8f69-61f180f3c772-kube-api-access-pzrqj" pod "74852e36-70a6-48f9-8f69-61f180f3c772" (UID: "74852e36-70a6-48f9-8f69-61f180f3c772"). InnerVolumeSpecName "kube-api-access-pzrqj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:08:53.275593 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:08:53.275552    4888 scope.go:122] "RemoveContainer" containerID="80f094a9a59c2bca8846d63aeb73518ec2716f8f23a7cb84ebe1f4cff0f56518"
Oct 30 19:08:53.278323 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:08:53.276631    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/74852e36-70a6-48f9-8f69-61f180f3c772-istio-envoy" pod "74852e36-70a6-48f9-8f69-61f180f3c772" (UID: "74852e36-70a6-48f9-8f69-61f180f3c772"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:08:53.278323 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:08:53.276683    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"80f094a9a59c2bca8846d63aeb73518ec2716f8f23a7cb84ebe1f4cff0f56518\": container with ID starting with 80f094a9a59c2bca8846d63aeb73518ec2716f8f23a7cb84ebe1f4cff0f56518 not found: ID does not exist" containerID="80f094a9a59c2bca8846d63aeb73518ec2716f8f23a7cb84ebe1f4cff0f56518"
Oct 30 19:08:53.278323 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:08:53.276720    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"80f094a9a59c2bca8846d63aeb73518ec2716f8f23a7cb84ebe1f4cff0f56518"} err="failed to get container status \"80f094a9a59c2bca8846d63aeb73518ec2716f8f23a7cb84ebe1f4cff0f56518\": rpc error: code = NotFound desc = could not find container \"80f094a9a59c2bca8846d63aeb73518ec2716f8f23a7cb84ebe1f4cff0f56518\": container with ID starting with 80f094a9a59c2bca8846d63aeb73518ec2716f8f23a7cb84ebe1f4cff0f56518 not found: ID does not exist"
Oct 30 19:08:53.278323 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:08:53.276916    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/74852e36-70a6-48f9-8f69-61f180f3c772-istio-podinfo" pod "74852e36-70a6-48f9-8f69-61f180f3c772" (UID: "74852e36-70a6-48f9-8f69-61f180f3c772"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:08:53.278687 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:08:53.278665    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/74852e36-70a6-48f9-8f69-61f180f3c772-istio-token" pod "74852e36-70a6-48f9-8f69-61f180f3c772" (UID: "74852e36-70a6-48f9-8f69-61f180f3c772"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:08:53.373699 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:08:53.373661    4888 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/74852e36-70a6-48f9-8f69-61f180f3c772-istio-data\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:08:53.373699 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:08:53.373693    4888 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/74852e36-70a6-48f9-8f69-61f180f3c772-workload-certs\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:08:53.373906 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:08:53.373707    4888 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/74852e36-70a6-48f9-8f69-61f180f3c772-istio-podinfo\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:08:53.373906 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:08:53.373720    4888 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/74852e36-70a6-48f9-8f69-61f180f3c772-credential-socket\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:08:53.373906 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:08:53.373733    4888 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/74852e36-70a6-48f9-8f69-61f180f3c772-istio-envoy\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:08:53.373906 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:08:53.373744    4888 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/74852e36-70a6-48f9-8f69-61f180f3c772-workload-socket\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:08:53.373906 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:08:53.373756    4888 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/74852e36-70a6-48f9-8f69-61f180f3c772-istio-token\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:08:53.373906 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:08:53.373768    4888 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/74852e36-70a6-48f9-8f69-61f180f3c772-istiod-ca-cert\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:08:53.373906 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:08:53.373783    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pzrqj\" (UniqueName: \"kubernetes.io/projected/74852e36-70a6-48f9-8f69-61f180f3c772-kube-api-access-pzrqj\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:08:53.701783 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:08:53.701750    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-66db998887-jsv6r"]
Oct 30 19:08:53.707047 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:08:53.707021    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-66db998887-jsv6r"]
Oct 30 19:08:53.893495 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:08:53.893415    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="74852e36-70a6-48f9-8f69-61f180f3c772" path="/var/lib/kubelet/pods/74852e36-70a6-48f9-8f69-61f180f3c772/volumes"
Oct 30 19:08:55.254743 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:08:55.254721    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-5h9tz"
Oct 30 19:08:55.281150 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:08:55.281123    4888 generic.go:356] "Generic (PLEG): container finished" podID="f5ffda80-881d-406d-8ef1-3664bf010a67" containerID="57b68653ad3ea8bc9ed04b8eb1be80126b104daa84ab371dfa97f97a8b90d314" exitCode=0
Oct 30 19:08:55.281339 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:08:55.281179    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-5h9tz"
Oct 30 19:08:55.281339 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:08:55.281212    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-5h9tz" event={"ID":"f5ffda80-881d-406d-8ef1-3664bf010a67","Type":"ContainerDied","Data":"57b68653ad3ea8bc9ed04b8eb1be80126b104daa84ab371dfa97f97a8b90d314"}
Oct 30 19:08:55.281339 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:08:55.281267    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-5h9tz" event={"ID":"f5ffda80-881d-406d-8ef1-3664bf010a67","Type":"ContainerDied","Data":"d2abb2363e118338bb9cf40bd776a36fd868a4119963f72fc1ff687cbc41142a"}
Oct 30 19:08:55.281339 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:08:55.281331    4888 scope.go:122] "RemoveContainer" containerID="57b68653ad3ea8bc9ed04b8eb1be80126b104daa84ab371dfa97f97a8b90d314"
Oct 30 19:08:55.291634 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:08:55.291603    4888 scope.go:122] "RemoveContainer" containerID="57b68653ad3ea8bc9ed04b8eb1be80126b104daa84ab371dfa97f97a8b90d314"
Oct 30 19:08:55.291942 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:08:55.291922    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"57b68653ad3ea8bc9ed04b8eb1be80126b104daa84ab371dfa97f97a8b90d314\": container with ID starting with 57b68653ad3ea8bc9ed04b8eb1be80126b104daa84ab371dfa97f97a8b90d314 not found: ID does not exist" containerID="57b68653ad3ea8bc9ed04b8eb1be80126b104daa84ab371dfa97f97a8b90d314"
Oct 30 19:08:55.291995 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:08:55.291953    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"57b68653ad3ea8bc9ed04b8eb1be80126b104daa84ab371dfa97f97a8b90d314"} err="failed to get container status \"57b68653ad3ea8bc9ed04b8eb1be80126b104daa84ab371dfa97f97a8b90d314\": rpc error: code = NotFound desc = could not find container \"57b68653ad3ea8bc9ed04b8eb1be80126b104daa84ab371dfa97f97a8b90d314\": container with ID starting with 57b68653ad3ea8bc9ed04b8eb1be80126b104daa84ab371dfa97f97a8b90d314 not found: ID does not exist"
Oct 30 19:08:55.403039 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:08:55.403004    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/f5ffda80-881d-406d-8ef1-3664bf010a67-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f5ffda80-881d-406d-8ef1-3664bf010a67-istio-podinfo\") pod \"f5ffda80-881d-406d-8ef1-3664bf010a67\" (UID: \"f5ffda80-881d-406d-8ef1-3664bf010a67\") "
Oct 30 19:08:55.403039 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:08:55.403042    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f5ffda80-881d-406d-8ef1-3664bf010a67-kube-api-access-dtjzr\" (UniqueName: \"kubernetes.io/projected/f5ffda80-881d-406d-8ef1-3664bf010a67-kube-api-access-dtjzr\") pod \"f5ffda80-881d-406d-8ef1-3664bf010a67\" (UID: \"f5ffda80-881d-406d-8ef1-3664bf010a67\") "
Oct 30 19:08:55.403312 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:08:55.403092    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f5ffda80-881d-406d-8ef1-3664bf010a67-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f5ffda80-881d-406d-8ef1-3664bf010a67-istio-envoy\") pod \"f5ffda80-881d-406d-8ef1-3664bf010a67\" (UID: \"f5ffda80-881d-406d-8ef1-3664bf010a67\") "
Oct 30 19:08:55.403312 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:08:55.403121    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f5ffda80-881d-406d-8ef1-3664bf010a67-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f5ffda80-881d-406d-8ef1-3664bf010a67-istio-data\") pod \"f5ffda80-881d-406d-8ef1-3664bf010a67\" (UID: \"f5ffda80-881d-406d-8ef1-3664bf010a67\") "
Oct 30 19:08:55.403312 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:08:55.403179    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f5ffda80-881d-406d-8ef1-3664bf010a67-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5ffda80-881d-406d-8ef1-3664bf010a67-credential-socket\") pod \"f5ffda80-881d-406d-8ef1-3664bf010a67\" (UID: \"f5ffda80-881d-406d-8ef1-3664bf010a67\") "
Oct 30 19:08:55.403312 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:08:55.403250    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f5ffda80-881d-406d-8ef1-3664bf010a67-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f5ffda80-881d-406d-8ef1-3664bf010a67-istiod-ca-cert\") pod \"f5ffda80-881d-406d-8ef1-3664bf010a67\" (UID: \"f5ffda80-881d-406d-8ef1-3664bf010a67\") "
Oct 30 19:08:55.403516 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:08:55.403428    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f5ffda80-881d-406d-8ef1-3664bf010a67-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5ffda80-881d-406d-8ef1-3664bf010a67-workload-socket\") pod \"f5ffda80-881d-406d-8ef1-3664bf010a67\" (UID: \"f5ffda80-881d-406d-8ef1-3664bf010a67\") "
Oct 30 19:08:55.403516 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:08:55.403469    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f5ffda80-881d-406d-8ef1-3664bf010a67-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f5ffda80-881d-406d-8ef1-3664bf010a67-workload-certs\") pod \"f5ffda80-881d-406d-8ef1-3664bf010a67\" (UID: \"f5ffda80-881d-406d-8ef1-3664bf010a67\") "
Oct 30 19:08:55.403621 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:08:55.403543    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f5ffda80-881d-406d-8ef1-3664bf010a67-istio-token\" (UniqueName: \"kubernetes.io/projected/f5ffda80-881d-406d-8ef1-3664bf010a67-istio-token\") pod \"f5ffda80-881d-406d-8ef1-3664bf010a67\" (UID: \"f5ffda80-881d-406d-8ef1-3664bf010a67\") "
Oct 30 19:08:55.403835 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:08:55.403767    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f5ffda80-881d-406d-8ef1-3664bf010a67-istio-data" pod "f5ffda80-881d-406d-8ef1-3664bf010a67" (UID: "f5ffda80-881d-406d-8ef1-3664bf010a67"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:08:55.404059 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:08:55.403893    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f5ffda80-881d-406d-8ef1-3664bf010a67-workload-socket" pod "f5ffda80-881d-406d-8ef1-3664bf010a67" (UID: "f5ffda80-881d-406d-8ef1-3664bf010a67"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:08:55.404059 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:08:55.404049    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f5ffda80-881d-406d-8ef1-3664bf010a67-credential-socket" pod "f5ffda80-881d-406d-8ef1-3664bf010a67" (UID: "f5ffda80-881d-406d-8ef1-3664bf010a67"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:08:55.404181 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:08:55.404143    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f5ffda80-881d-406d-8ef1-3664bf010a67-istiod-ca-cert" pod "f5ffda80-881d-406d-8ef1-3664bf010a67" (UID: "f5ffda80-881d-406d-8ef1-3664bf010a67"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:08:55.404246 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:08:55.404233    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f5ffda80-881d-406d-8ef1-3664bf010a67-workload-certs" pod "f5ffda80-881d-406d-8ef1-3664bf010a67" (UID: "f5ffda80-881d-406d-8ef1-3664bf010a67"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:08:55.404514 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:08:55.404495    4888 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f5ffda80-881d-406d-8ef1-3664bf010a67-istio-data\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:08:55.404624 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:08:55.404519    4888 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5ffda80-881d-406d-8ef1-3664bf010a67-credential-socket\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:08:55.404624 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:08:55.404533    4888 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f5ffda80-881d-406d-8ef1-3664bf010a67-istiod-ca-cert\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:08:55.404624 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:08:55.404547    4888 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5ffda80-881d-406d-8ef1-3664bf010a67-workload-socket\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:08:55.404624 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:08:55.404560    4888 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f5ffda80-881d-406d-8ef1-3664bf010a67-workload-certs\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:08:55.405783 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:08:55.405734    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f5ffda80-881d-406d-8ef1-3664bf010a67-istio-podinfo" pod "f5ffda80-881d-406d-8ef1-3664bf010a67" (UID: "f5ffda80-881d-406d-8ef1-3664bf010a67"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:08:55.406148 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:08:55.406120    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f5ffda80-881d-406d-8ef1-3664bf010a67-kube-api-access-dtjzr" pod "f5ffda80-881d-406d-8ef1-3664bf010a67" (UID: "f5ffda80-881d-406d-8ef1-3664bf010a67"). InnerVolumeSpecName "kube-api-access-dtjzr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:08:55.406309 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:08:55.406261    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f5ffda80-881d-406d-8ef1-3664bf010a67-istio-token" pod "f5ffda80-881d-406d-8ef1-3664bf010a67" (UID: "f5ffda80-881d-406d-8ef1-3664bf010a67"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:08:55.406637 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:08:55.406606    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f5ffda80-881d-406d-8ef1-3664bf010a67-istio-envoy" pod "f5ffda80-881d-406d-8ef1-3664bf010a67" (UID: "f5ffda80-881d-406d-8ef1-3664bf010a67"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:08:55.506300 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:08:55.506210    4888 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f5ffda80-881d-406d-8ef1-3664bf010a67-istio-token\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:08:55.506300 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:08:55.506238    4888 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f5ffda80-881d-406d-8ef1-3664bf010a67-istio-podinfo\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:08:55.506300 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:08:55.506250    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dtjzr\" (UniqueName: \"kubernetes.io/projected/f5ffda80-881d-406d-8ef1-3664bf010a67-kube-api-access-dtjzr\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:08:55.506300 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:08:55.506259    4888 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f5ffda80-881d-406d-8ef1-3664bf010a67-istio-envoy\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:08:55.710031 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:08:55.709991    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-5h9tz"]
Oct 30 19:08:55.714995 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:08:55.714964    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-5h9tz"]
Oct 30 19:08:55.892667 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:08:55.892632    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f5ffda80-881d-406d-8ef1-3664bf010a67" path="/var/lib/kubelet/pods/f5ffda80-881d-406d-8ef1-3664bf010a67/volumes"
Oct 30 19:08:56.543400 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:08:56.543373    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-7x4rb_382e8b31-a7fa-43f3-9f15-4c61627b0c04/console-operator/2.log"
Oct 30 19:08:56.545876 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:08:56.545853    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-7x4rb_382e8b31-a7fa-43f3-9f15-4c61627b0c04/console-operator/2.log"
Oct 30 19:08:56.551080 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:08:56.551061    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xfcdl_3f3fe764-2652-46c9-8e54-7882c0e6bcf1/ovn-acl-logging/0.log"
Oct 30 19:08:56.553187 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:08:56.553163    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xfcdl_3f3fe764-2652-46c9-8e54-7882c0e6bcf1/ovn-acl-logging/0.log"
Oct 30 19:09:05.621923 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:09:05.621835    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-jbsw6" containerName="extractor"
Oct 30 19:09:08.793328 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:09:08.793277    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-4ebf78-67779b8999-78snt"]
Oct 30 19:09:08.793904 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:09:08.793886    4888 cpu_manager.go:414] "containerMap: removing container" podUID="8ff6e4b1-7681-4936-b30f-240ece473e12" containerName="payload-pre-processing"
Oct 30 19:09:08.793952 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:09:08.793906    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8ff6e4b1-7681-4936-b30f-240ece473e12" containerName="payload-pre-processing"
Oct 30 19:09:08.793952 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:09:08.793932    4888 cpu_manager.go:414] "containerMap: removing container" podUID="f5ffda80-881d-406d-8ef1-3664bf010a67" containerName="istio-proxy"
Oct 30 19:09:08.793952 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:09:08.793937    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f5ffda80-881d-406d-8ef1-3664bf010a67" containerName="istio-proxy"
Oct 30 19:09:08.794049 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:09:08.793954    4888 cpu_manager.go:414] "containerMap: removing container" podUID="f9c90752-5508-445c-823e-3d9ea37f54b2" containerName="istio-proxy"
Oct 30 19:09:08.794049 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:09:08.793959    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f9c90752-5508-445c-823e-3d9ea37f54b2" containerName="istio-proxy"
Oct 30 19:09:08.794049 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:09:08.793971    4888 cpu_manager.go:414] "containerMap: removing container" podUID="94364f5b-4740-430f-87df-51e9b656d963" containerName="istio-proxy"
Oct 30 19:09:08.794049 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:09:08.793975    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="94364f5b-4740-430f-87df-51e9b656d963" containerName="istio-proxy"
Oct 30 19:09:08.794049 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:09:08.793986    4888 cpu_manager.go:414] "containerMap: removing container" podUID="74852e36-70a6-48f9-8f69-61f180f3c772" containerName="istio-proxy"
Oct 30 19:09:08.794049 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:09:08.793991    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="74852e36-70a6-48f9-8f69-61f180f3c772" containerName="istio-proxy"
Oct 30 19:09:08.794049 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:09:08.794004    4888 cpu_manager.go:414] "containerMap: removing container" podUID="8317ca5e-a974-48a6-ad90-4c08cdd6bf57" containerName="payload-pre-processing"
Oct 30 19:09:08.794049 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:09:08.794009    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8317ca5e-a974-48a6-ad90-4c08cdd6bf57" containerName="payload-pre-processing"
Oct 30 19:09:08.794049 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:09:08.794019    4888 cpu_manager.go:414] "containerMap: removing container" podUID="94364f5b-4740-430f-87df-51e9b656d963" containerName="istio-proxy"
Oct 30 19:09:08.794049 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:09:08.794023    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="94364f5b-4740-430f-87df-51e9b656d963" containerName="istio-proxy"
Oct 30 19:09:08.794362 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:09:08.794100    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="74852e36-70a6-48f9-8f69-61f180f3c772" containerName="istio-proxy"
Oct 30 19:09:08.794362 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:09:08.794111    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="8317ca5e-a974-48a6-ad90-4c08cdd6bf57" containerName="payload-pre-processing"
Oct 30 19:09:08.794362 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:09:08.794117    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="f9c90752-5508-445c-823e-3d9ea37f54b2" containerName="istio-proxy"
Oct 30 19:09:08.794362 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:09:08.794123    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="94364f5b-4740-430f-87df-51e9b656d963" containerName="istio-proxy"
Oct 30 19:09:08.794362 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:09:08.794129    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="8ff6e4b1-7681-4936-b30f-240ece473e12" containerName="payload-pre-processing"
Oct 30 19:09:08.794362 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:09:08.794138    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="f5ffda80-881d-406d-8ef1-3664bf010a67" containerName="istio-proxy"
Oct 30 19:09:08.794362 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:09:08.794145    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="f9c90752-5508-445c-823e-3d9ea37f54b2" containerName="istio-proxy"
Oct 30 19:09:08.794362 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:09:08.794152    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="94364f5b-4740-430f-87df-51e9b656d963" containerName="istio-proxy"
Oct 30 19:09:08.798767 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:09:08.798748    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-4ebf78-67779b8999-78snt"
Oct 30 19:09:08.856464 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:09:08.856436    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-4ebf78-67779b8999-78snt"]
Oct 30 19:09:08.880842 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:09:08.880812    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/79f94bb4-e014-4bee-b2e2-0e823f166b70-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-4ebf78-67779b8999-78snt\" (UID: \"79f94bb4-e014-4bee-b2e2-0e823f166b70\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-4ebf78-67779b8999-78snt"
Oct 30 19:09:08.881022 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:09:08.880857    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zqtnr\" (UniqueName: \"kubernetes.io/projected/79f94bb4-e014-4bee-b2e2-0e823f166b70-kube-api-access-zqtnr\") pod \"maas-api-e2e-ipp-b-4ebf78-67779b8999-78snt\" (UID: \"79f94bb4-e014-4bee-b2e2-0e823f166b70\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-4ebf78-67779b8999-78snt"
Oct 30 19:09:08.881123 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:09:08.881096    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/79f94bb4-e014-4bee-b2e2-0e823f166b70-metrics-certs\") pod \"maas-api-e2e-ipp-b-4ebf78-67779b8999-78snt\" (UID: \"79f94bb4-e014-4bee-b2e2-0e823f166b70\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-4ebf78-67779b8999-78snt"
Oct 30 19:09:08.881230 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:09:08.881216    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/79f94bb4-e014-4bee-b2e2-0e823f166b70-maas-api-tls\") pod \"maas-api-e2e-ipp-b-4ebf78-67779b8999-78snt\" (UID: \"79f94bb4-e014-4bee-b2e2-0e823f166b70\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-4ebf78-67779b8999-78snt"
Oct 30 19:09:08.902935 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:09:08.902892    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-4ebf78-serving-cert\""
Oct 30 19:09:08.982813 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:09:08.982781    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/79f94bb4-e014-4bee-b2e2-0e823f166b70-maas-api-tls\") pod \"maas-api-e2e-ipp-b-4ebf78-67779b8999-78snt\" (UID: \"79f94bb4-e014-4bee-b2e2-0e823f166b70\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-4ebf78-67779b8999-78snt"
Oct 30 19:09:08.982998 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:09:08.982869    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/79f94bb4-e014-4bee-b2e2-0e823f166b70-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-4ebf78-67779b8999-78snt\" (UID: \"79f94bb4-e014-4bee-b2e2-0e823f166b70\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-4ebf78-67779b8999-78snt"
Oct 30 19:09:08.982998 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:09:08.982926    4888 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-b-4ebf78-serving-cert: secret "maas-api-e2e-ipp-b-4ebf78-serving-cert" not found
Oct 30 19:09:08.982998 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:09:08.982977    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zqtnr\" (UniqueName: \"kubernetes.io/projected/79f94bb4-e014-4bee-b2e2-0e823f166b70-kube-api-access-zqtnr\") pod \"maas-api-e2e-ipp-b-4ebf78-67779b8999-78snt\" (UID: \"79f94bb4-e014-4bee-b2e2-0e823f166b70\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-4ebf78-67779b8999-78snt"
Oct 30 19:09:08.982998 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:09:08.982995    4888 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79f94bb4-e014-4bee-b2e2-0e823f166b70-maas-api-tls podName:79f94bb4-e014-4bee-b2e2-0e823f166b70 nodeName:}" failed. No retries permitted until 2026-10-30 19:09:09.482974016 +0000 UTC m=+2113.730380040 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/79f94bb4-e014-4bee-b2e2-0e823f166b70-maas-api-tls") pod "maas-api-e2e-ipp-b-4ebf78-67779b8999-78snt" (UID: "79f94bb4-e014-4bee-b2e2-0e823f166b70") : secret "maas-api-e2e-ipp-b-4ebf78-serving-cert" not found
Oct 30 19:09:08.983245 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:09:08.983146    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/79f94bb4-e014-4bee-b2e2-0e823f166b70-metrics-certs\") pod \"maas-api-e2e-ipp-b-4ebf78-67779b8999-78snt\" (UID: \"79f94bb4-e014-4bee-b2e2-0e823f166b70\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-4ebf78-67779b8999-78snt"
Oct 30 19:09:08.983605 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:09:08.983583    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/79f94bb4-e014-4bee-b2e2-0e823f166b70-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-4ebf78-67779b8999-78snt\" (UID: \"79f94bb4-e014-4bee-b2e2-0e823f166b70\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-4ebf78-67779b8999-78snt"
Oct 30 19:09:08.985717 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:09:08.985695    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/79f94bb4-e014-4bee-b2e2-0e823f166b70-metrics-certs\") pod \"maas-api-e2e-ipp-b-4ebf78-67779b8999-78snt\" (UID: \"79f94bb4-e014-4bee-b2e2-0e823f166b70\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-4ebf78-67779b8999-78snt"
Oct 30 19:09:09.054894 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:09:09.054780    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zqtnr\" (UniqueName: \"kubernetes.io/projected/79f94bb4-e014-4bee-b2e2-0e823f166b70-kube-api-access-zqtnr\") pod \"maas-api-e2e-ipp-b-4ebf78-67779b8999-78snt\" (UID: \"79f94bb4-e014-4bee-b2e2-0e823f166b70\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-4ebf78-67779b8999-78snt"
Oct 30 19:09:09.487983 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:09:09.487949    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/79f94bb4-e014-4bee-b2e2-0e823f166b70-maas-api-tls\") pod \"maas-api-e2e-ipp-b-4ebf78-67779b8999-78snt\" (UID: \"79f94bb4-e014-4bee-b2e2-0e823f166b70\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-4ebf78-67779b8999-78snt"
Oct 30 19:09:09.490621 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:09:09.490595    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/79f94bb4-e014-4bee-b2e2-0e823f166b70-maas-api-tls\") pod \"maas-api-e2e-ipp-b-4ebf78-67779b8999-78snt\" (UID: \"79f94bb4-e014-4bee-b2e2-0e823f166b70\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-4ebf78-67779b8999-78snt"
Oct 30 19:09:09.713738 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:09:09.713709    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-4ebf78-67779b8999-78snt"
Oct 30 19:09:10.096564 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:09:10.096535    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-4ebf78-67779b8999-78snt"]
Oct 30 19:09:10.096925 ip-10-0-134-46 kubenswrapper[4888]: W1030 19:09:10.096666    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79f94bb4_e014_4bee_b2e2_0e823f166b70.slice/crio-2c049a9001b7c777ac645539bced752a21ef993c21680b0f38bcbc8e11b44d7e WatchSource:0}: Error finding container 2c049a9001b7c777ac645539bced752a21ef993c21680b0f38bcbc8e11b44d7e: Status 404 returned error can't find the container with id 2c049a9001b7c777ac645539bced752a21ef993c21680b0f38bcbc8e11b44d7e
Oct 30 19:09:10.396528 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:09:10.396446    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-4ebf78-67779b8999-78snt" event={"ID":"79f94bb4-e014-4bee-b2e2-0e823f166b70","Type":"ContainerStarted","Data":"2c049a9001b7c777ac645539bced752a21ef993c21680b0f38bcbc8e11b44d7e"}
Oct 30 19:09:11.406200 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:09:11.406168    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-4ebf78-67779b8999-78snt" event={"ID":"79f94bb4-e014-4bee-b2e2-0e823f166b70","Type":"ContainerStarted","Data":"c574077631f8dcd3cc21edfc97720813cbdd87e321c533356d7c117a3dfa4a4e"}
Oct 30 19:09:11.406627 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:09:11.406251    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-4ebf78-67779b8999-78snt"
Oct 30 19:09:11.528050 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:09:11.527991    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-4ebf78-67779b8999-78snt" podStartSLOduration=2.731002237 podStartE2EDuration="3.527973393s" totalImagesPullingTime="796.971156ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:09:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:09:11.526221922 +0000 UTC m=+2115.773627947" watchObservedRunningTime="2026-10-30 19:09:11.527973393 +0000 UTC m=+2115.775379418"
Oct 30 19:09:17.420165 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:09:17.420134    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-4ebf78-67779b8999-78snt"
Oct 30 19:09:28.889650 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:09:28.889610    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-rjmdz"]
Oct 30 19:09:35.620137 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:09:35.620101    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-jbsw6" containerName="extractor"
Oct 30 19:09:46.776671 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:09:46.776633    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-0ee379-openshift-default-647f98d679-s29zb"]
Oct 30 19:09:46.777355 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:09:46.776845    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-b-0ee379-openshift-default-647f98d679-s29zb" podUID="29c22c2d-64fe-476b-8b35-eab525253a5c" containerName="istio-proxy" containerID="cri-o://83c57b222565e88d1637c9290156e822aca275eb7d71a162012e2c9bdd86d1d4" gracePeriod=30
Oct 30 19:09:52.042810 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:09:52.042755    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-0ee379-openshift-default-647f98d679-s29zb"
Oct 30 19:09:52.147708 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:09:52.147677    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/29c22c2d-64fe-476b-8b35-eab525253a5c-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/29c22c2d-64fe-476b-8b35-eab525253a5c-istiod-ca-cert\") pod \"29c22c2d-64fe-476b-8b35-eab525253a5c\" (UID: \"29c22c2d-64fe-476b-8b35-eab525253a5c\") "
Oct 30 19:09:52.147885 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:09:52.147732    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/29c22c2d-64fe-476b-8b35-eab525253a5c-istio-token\" (UniqueName: \"kubernetes.io/projected/29c22c2d-64fe-476b-8b35-eab525253a5c-istio-token\") pod \"29c22c2d-64fe-476b-8b35-eab525253a5c\" (UID: \"29c22c2d-64fe-476b-8b35-eab525253a5c\") "
Oct 30 19:09:52.147885 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:09:52.147781    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/29c22c2d-64fe-476b-8b35-eab525253a5c-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/29c22c2d-64fe-476b-8b35-eab525253a5c-istio-envoy\") pod \"29c22c2d-64fe-476b-8b35-eab525253a5c\" (UID: \"29c22c2d-64fe-476b-8b35-eab525253a5c\") "
Oct 30 19:09:52.147885 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:09:52.147861    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/29c22c2d-64fe-476b-8b35-eab525253a5c-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/29c22c2d-64fe-476b-8b35-eab525253a5c-credential-socket\") pod \"29c22c2d-64fe-476b-8b35-eab525253a5c\" (UID: \"29c22c2d-64fe-476b-8b35-eab525253a5c\") "
Oct 30 19:09:52.148036 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:09:52.147889    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/29c22c2d-64fe-476b-8b35-eab525253a5c-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/29c22c2d-64fe-476b-8b35-eab525253a5c-workload-certs\") pod \"29c22c2d-64fe-476b-8b35-eab525253a5c\" (UID: \"29c22c2d-64fe-476b-8b35-eab525253a5c\") "
Oct 30 19:09:52.148036 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:09:52.147925    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/29c22c2d-64fe-476b-8b35-eab525253a5c-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/29c22c2d-64fe-476b-8b35-eab525253a5c-istio-podinfo\") pod \"29c22c2d-64fe-476b-8b35-eab525253a5c\" (UID: \"29c22c2d-64fe-476b-8b35-eab525253a5c\") "
Oct 30 19:09:52.148036 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:09:52.148021    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/29c22c2d-64fe-476b-8b35-eab525253a5c-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/29c22c2d-64fe-476b-8b35-eab525253a5c-workload-socket\") pod \"29c22c2d-64fe-476b-8b35-eab525253a5c\" (UID: \"29c22c2d-64fe-476b-8b35-eab525253a5c\") "
Oct 30 19:09:52.148171 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:09:52.148070    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/29c22c2d-64fe-476b-8b35-eab525253a5c-kube-api-access-8lqs4\" (UniqueName: \"kubernetes.io/projected/29c22c2d-64fe-476b-8b35-eab525253a5c-kube-api-access-8lqs4\") pod \"29c22c2d-64fe-476b-8b35-eab525253a5c\" (UID: \"29c22c2d-64fe-476b-8b35-eab525253a5c\") "
Oct 30 19:09:52.148171 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:09:52.148115    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/29c22c2d-64fe-476b-8b35-eab525253a5c-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/29c22c2d-64fe-476b-8b35-eab525253a5c-istio-data\") pod \"29c22c2d-64fe-476b-8b35-eab525253a5c\" (UID: \"29c22c2d-64fe-476b-8b35-eab525253a5c\") "
Oct 30 19:09:52.148171 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:09:52.148121    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29c22c2d-64fe-476b-8b35-eab525253a5c-credential-socket" pod "29c22c2d-64fe-476b-8b35-eab525253a5c" (UID: "29c22c2d-64fe-476b-8b35-eab525253a5c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:09:52.148384 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:09:52.148179    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/29c22c2d-64fe-476b-8b35-eab525253a5c-istiod-ca-cert" pod "29c22c2d-64fe-476b-8b35-eab525253a5c" (UID: "29c22c2d-64fe-476b-8b35-eab525253a5c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:09:52.148384 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:09:52.148306    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29c22c2d-64fe-476b-8b35-eab525253a5c-workload-certs" pod "29c22c2d-64fe-476b-8b35-eab525253a5c" (UID: "29c22c2d-64fe-476b-8b35-eab525253a5c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:09:52.148497 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:09:52.148422    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29c22c2d-64fe-476b-8b35-eab525253a5c-workload-socket" pod "29c22c2d-64fe-476b-8b35-eab525253a5c" (UID: "29c22c2d-64fe-476b-8b35-eab525253a5c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:09:52.148558 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:09:52.148523    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29c22c2d-64fe-476b-8b35-eab525253a5c-istio-data" pod "29c22c2d-64fe-476b-8b35-eab525253a5c" (UID: "29c22c2d-64fe-476b-8b35-eab525253a5c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:09:52.148902 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:09:52.148881    4888 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/29c22c2d-64fe-476b-8b35-eab525253a5c-workload-socket\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:09:52.149003 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:09:52.148909    4888 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/29c22c2d-64fe-476b-8b35-eab525253a5c-istio-data\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:09:52.149003 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:09:52.148924    4888 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/29c22c2d-64fe-476b-8b35-eab525253a5c-istiod-ca-cert\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:09:52.149003 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:09:52.148938    4888 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/29c22c2d-64fe-476b-8b35-eab525253a5c-credential-socket\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:09:52.149003 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:09:52.148955    4888 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/29c22c2d-64fe-476b-8b35-eab525253a5c-workload-certs\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:09:52.150314 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:09:52.150271    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/29c22c2d-64fe-476b-8b35-eab525253a5c-istio-podinfo" pod "29c22c2d-64fe-476b-8b35-eab525253a5c" (UID: "29c22c2d-64fe-476b-8b35-eab525253a5c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:09:52.150314 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:09:52.150301    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/29c22c2d-64fe-476b-8b35-eab525253a5c-istio-token" pod "29c22c2d-64fe-476b-8b35-eab525253a5c" (UID: "29c22c2d-64fe-476b-8b35-eab525253a5c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:09:52.150800 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:09:52.150779    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29c22c2d-64fe-476b-8b35-eab525253a5c-istio-envoy" pod "29c22c2d-64fe-476b-8b35-eab525253a5c" (UID: "29c22c2d-64fe-476b-8b35-eab525253a5c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:09:52.150877 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:09:52.150813    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/29c22c2d-64fe-476b-8b35-eab525253a5c-kube-api-access-8lqs4" pod "29c22c2d-64fe-476b-8b35-eab525253a5c" (UID: "29c22c2d-64fe-476b-8b35-eab525253a5c"). InnerVolumeSpecName "kube-api-access-8lqs4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:09:52.249870 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:09:52.249842    4888 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/29c22c2d-64fe-476b-8b35-eab525253a5c-istio-token\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:09:52.249870 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:09:52.249867    4888 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/29c22c2d-64fe-476b-8b35-eab525253a5c-istio-envoy\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:09:52.249870 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:09:52.249876    4888 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/29c22c2d-64fe-476b-8b35-eab525253a5c-istio-podinfo\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:09:52.250171 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:09:52.249887    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8lqs4\" (UniqueName: \"kubernetes.io/projected/29c22c2d-64fe-476b-8b35-eab525253a5c-kube-api-access-8lqs4\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:09:52.736018 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:09:52.735977    4888 generic.go:356] "Generic (PLEG): container finished" podID="29c22c2d-64fe-476b-8b35-eab525253a5c" containerID="83c57b222565e88d1637c9290156e822aca275eb7d71a162012e2c9bdd86d1d4" exitCode=0
Oct 30 19:09:52.736232 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:09:52.736047    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-0ee379-openshift-default-647f98d679-s29zb"
Oct 30 19:09:52.736232 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:09:52.736047    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-0ee379-openshift-default-647f98d679-s29zb" event={"ID":"29c22c2d-64fe-476b-8b35-eab525253a5c","Type":"ContainerDied","Data":"83c57b222565e88d1637c9290156e822aca275eb7d71a162012e2c9bdd86d1d4"}
Oct 30 19:09:52.736232 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:09:52.736180    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-0ee379-openshift-default-647f98d679-s29zb" event={"ID":"29c22c2d-64fe-476b-8b35-eab525253a5c","Type":"ContainerDied","Data":"ec1c73b44df6389f2813ffd258dcf5641fd752f69573903e03e75a515015887a"}
Oct 30 19:09:52.736232 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:09:52.736215    4888 scope.go:122] "RemoveContainer" containerID="83c57b222565e88d1637c9290156e822aca275eb7d71a162012e2c9bdd86d1d4"
Oct 30 19:09:52.748091 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:09:52.748074    4888 scope.go:122] "RemoveContainer" containerID="78524c76cd12e11a69585cf1dce2977bb240bc37702e952b6653eadd2f6a393d"
Oct 30 19:09:52.757606 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:09:52.757589    4888 scope.go:122] "RemoveContainer" containerID="83c57b222565e88d1637c9290156e822aca275eb7d71a162012e2c9bdd86d1d4"
Oct 30 19:09:52.757851 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:09:52.757826    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"83c57b222565e88d1637c9290156e822aca275eb7d71a162012e2c9bdd86d1d4\": container with ID starting with 83c57b222565e88d1637c9290156e822aca275eb7d71a162012e2c9bdd86d1d4 not found: ID does not exist" containerID="83c57b222565e88d1637c9290156e822aca275eb7d71a162012e2c9bdd86d1d4"
Oct 30 19:09:52.757964 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:09:52.757858    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"83c57b222565e88d1637c9290156e822aca275eb7d71a162012e2c9bdd86d1d4"} err="failed to get container status \"83c57b222565e88d1637c9290156e822aca275eb7d71a162012e2c9bdd86d1d4\": rpc error: code = NotFound desc = could not find container \"83c57b222565e88d1637c9290156e822aca275eb7d71a162012e2c9bdd86d1d4\": container with ID starting with 83c57b222565e88d1637c9290156e822aca275eb7d71a162012e2c9bdd86d1d4 not found: ID does not exist"
Oct 30 19:09:52.757964 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:09:52.757875    4888 scope.go:122] "RemoveContainer" containerID="78524c76cd12e11a69585cf1dce2977bb240bc37702e952b6653eadd2f6a393d"
Oct 30 19:09:52.758164 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:09:52.758146    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"78524c76cd12e11a69585cf1dce2977bb240bc37702e952b6653eadd2f6a393d\": container with ID starting with 78524c76cd12e11a69585cf1dce2977bb240bc37702e952b6653eadd2f6a393d not found: ID does not exist" containerID="78524c76cd12e11a69585cf1dce2977bb240bc37702e952b6653eadd2f6a393d"
Oct 30 19:09:52.758210 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:09:52.758169    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"78524c76cd12e11a69585cf1dce2977bb240bc37702e952b6653eadd2f6a393d"} err="failed to get container status \"78524c76cd12e11a69585cf1dce2977bb240bc37702e952b6653eadd2f6a393d\": rpc error: code = NotFound desc = could not find container \"78524c76cd12e11a69585cf1dce2977bb240bc37702e952b6653eadd2f6a393d\": container with ID starting with 78524c76cd12e11a69585cf1dce2977bb240bc37702e952b6653eadd2f6a393d not found: ID does not exist"
Oct 30 19:09:52.959302 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:09:52.959260    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-0ee379-openshift-default-647f98d679-s29zb"]
Oct 30 19:09:53.008515 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:09:53.008435    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-b-0ee379-openshift-default-647f98d679-s29zb"]
Oct 30 19:09:53.892697 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:09:53.892667    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="29c22c2d-64fe-476b-8b35-eab525253a5c" path="/var/lib/kubelet/pods/29c22c2d-64fe-476b-8b35-eab525253a5c/volumes"
Oct 30 19:09:59.545520 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:09:59.545485    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-rjmdz"]
Oct 30 19:10:05.619686 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:10:05.619650    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-jbsw6" containerName="extractor"
Oct 30 19:10:35.619382 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:10:35.619295    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-jbsw6" containerName="extractor"
Oct 30 19:10:41.271914 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:10:41.271877    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-rjmdz"]
Oct 30 19:10:46.707847 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:10:46.707813    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-c682f"]
Oct 30 19:10:46.708310 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:10:46.708035    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-c682f" podUID="6462e226-4e52-40da-8fbd-656be33ed133" containerName="istio-proxy" containerID="cri-o://c8ce1ce99d508fe76b4ba8ba847e8033ecbbb0ee6be74e03754ba147696d2d84" gracePeriod=30
Oct 30 19:10:48.094630 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:10:48.094592    4888 patch_prober.go:29] interesting pod/e2e-worker-w0-2ade09-openshift-default-785b7cf675-c682f container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 19:10:48.095040 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:10:48.094645    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-c682f" podUID="6462e226-4e52-40da-8fbd-656be33ed133" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 19:10:51.999743 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:10:51.999717    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-c682f"
Oct 30 19:10:52.181642 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:10:52.181612    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6462e226-4e52-40da-8fbd-656be33ed133-istio-token\" (UniqueName: \"kubernetes.io/projected/6462e226-4e52-40da-8fbd-656be33ed133-istio-token\") pod \"6462e226-4e52-40da-8fbd-656be33ed133\" (UID: \"6462e226-4e52-40da-8fbd-656be33ed133\") "
Oct 30 19:10:52.181807 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:10:52.181672    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6462e226-4e52-40da-8fbd-656be33ed133-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6462e226-4e52-40da-8fbd-656be33ed133-workload-certs\") pod \"6462e226-4e52-40da-8fbd-656be33ed133\" (UID: \"6462e226-4e52-40da-8fbd-656be33ed133\") "
Oct 30 19:10:52.181807 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:10:52.181778    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6462e226-4e52-40da-8fbd-656be33ed133-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6462e226-4e52-40da-8fbd-656be33ed133-credential-socket\") pod \"6462e226-4e52-40da-8fbd-656be33ed133\" (UID: \"6462e226-4e52-40da-8fbd-656be33ed133\") "
Oct 30 19:10:52.181916 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:10:52.181842    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6462e226-4e52-40da-8fbd-656be33ed133-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6462e226-4e52-40da-8fbd-656be33ed133-workload-socket\") pod \"6462e226-4e52-40da-8fbd-656be33ed133\" (UID: \"6462e226-4e52-40da-8fbd-656be33ed133\") "
Oct 30 19:10:52.181916 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:10:52.181904    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6462e226-4e52-40da-8fbd-656be33ed133-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6462e226-4e52-40da-8fbd-656be33ed133-istio-envoy\") pod \"6462e226-4e52-40da-8fbd-656be33ed133\" (UID: \"6462e226-4e52-40da-8fbd-656be33ed133\") "
Oct 30 19:10:52.182021 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:10:52.181935    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6462e226-4e52-40da-8fbd-656be33ed133-kube-api-access-wxnvm\" (UniqueName: \"kubernetes.io/projected/6462e226-4e52-40da-8fbd-656be33ed133-kube-api-access-wxnvm\") pod \"6462e226-4e52-40da-8fbd-656be33ed133\" (UID: \"6462e226-4e52-40da-8fbd-656be33ed133\") "
Oct 30 19:10:52.182021 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:10:52.181982    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/6462e226-4e52-40da-8fbd-656be33ed133-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6462e226-4e52-40da-8fbd-656be33ed133-istio-podinfo\") pod \"6462e226-4e52-40da-8fbd-656be33ed133\" (UID: \"6462e226-4e52-40da-8fbd-656be33ed133\") "
Oct 30 19:10:52.182137 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:10:52.182020    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6462e226-4e52-40da-8fbd-656be33ed133-workload-certs" pod "6462e226-4e52-40da-8fbd-656be33ed133" (UID: "6462e226-4e52-40da-8fbd-656be33ed133"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:10:52.182137 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:10:52.182076    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6462e226-4e52-40da-8fbd-656be33ed133-credential-socket" pod "6462e226-4e52-40da-8fbd-656be33ed133" (UID: "6462e226-4e52-40da-8fbd-656be33ed133"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:10:52.182137 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:10:52.182117    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6462e226-4e52-40da-8fbd-656be33ed133-workload-socket" pod "6462e226-4e52-40da-8fbd-656be33ed133" (UID: "6462e226-4e52-40da-8fbd-656be33ed133"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:10:52.182308 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:10:52.182125    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6462e226-4e52-40da-8fbd-656be33ed133-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6462e226-4e52-40da-8fbd-656be33ed133-istiod-ca-cert\") pod \"6462e226-4e52-40da-8fbd-656be33ed133\" (UID: \"6462e226-4e52-40da-8fbd-656be33ed133\") "
Oct 30 19:10:52.182308 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:10:52.182206    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6462e226-4e52-40da-8fbd-656be33ed133-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6462e226-4e52-40da-8fbd-656be33ed133-istio-data\") pod \"6462e226-4e52-40da-8fbd-656be33ed133\" (UID: \"6462e226-4e52-40da-8fbd-656be33ed133\") "
Oct 30 19:10:52.182452 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:10:52.182424    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6462e226-4e52-40da-8fbd-656be33ed133-istiod-ca-cert" pod "6462e226-4e52-40da-8fbd-656be33ed133" (UID: "6462e226-4e52-40da-8fbd-656be33ed133"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:10:52.182782 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:10:52.182756    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6462e226-4e52-40da-8fbd-656be33ed133-istio-data" pod "6462e226-4e52-40da-8fbd-656be33ed133" (UID: "6462e226-4e52-40da-8fbd-656be33ed133"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:10:52.182956 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:10:52.182935    4888 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6462e226-4e52-40da-8fbd-656be33ed133-istiod-ca-cert\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:10:52.183017 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:10:52.182962    4888 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6462e226-4e52-40da-8fbd-656be33ed133-istio-data\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:10:52.183017 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:10:52.182993    4888 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6462e226-4e52-40da-8fbd-656be33ed133-workload-certs\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:10:52.183017 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:10:52.183008    4888 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6462e226-4e52-40da-8fbd-656be33ed133-credential-socket\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:10:52.183160 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:10:52.183024    4888 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6462e226-4e52-40da-8fbd-656be33ed133-workload-socket\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:10:52.184657 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:10:52.184631    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6462e226-4e52-40da-8fbd-656be33ed133-istio-token" pod "6462e226-4e52-40da-8fbd-656be33ed133" (UID: "6462e226-4e52-40da-8fbd-656be33ed133"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:10:52.185697 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:10:52.184971    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6462e226-4e52-40da-8fbd-656be33ed133-istio-podinfo" pod "6462e226-4e52-40da-8fbd-656be33ed133" (UID: "6462e226-4e52-40da-8fbd-656be33ed133"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:10:52.186499 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:10:52.186473    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6462e226-4e52-40da-8fbd-656be33ed133-istio-envoy" pod "6462e226-4e52-40da-8fbd-656be33ed133" (UID: "6462e226-4e52-40da-8fbd-656be33ed133"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:10:52.186654 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:10:52.186635    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6462e226-4e52-40da-8fbd-656be33ed133-kube-api-access-wxnvm" pod "6462e226-4e52-40da-8fbd-656be33ed133" (UID: "6462e226-4e52-40da-8fbd-656be33ed133"). InnerVolumeSpecName "kube-api-access-wxnvm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:10:52.236347 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:10:52.236252    4888 generic.go:356] "Generic (PLEG): container finished" podID="6462e226-4e52-40da-8fbd-656be33ed133" containerID="c8ce1ce99d508fe76b4ba8ba847e8033ecbbb0ee6be74e03754ba147696d2d84" exitCode=0
Oct 30 19:10:52.236492 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:10:52.236378    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-c682f" event={"ID":"6462e226-4e52-40da-8fbd-656be33ed133","Type":"ContainerDied","Data":"c8ce1ce99d508fe76b4ba8ba847e8033ecbbb0ee6be74e03754ba147696d2d84"}
Oct 30 19:10:52.236492 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:10:52.236456    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-c682f" event={"ID":"6462e226-4e52-40da-8fbd-656be33ed133","Type":"ContainerDied","Data":"6ebe969f8442d517dafc6a675fc322e4767a9a2cbb28cf023ebc06a24fc81960"}
Oct 30 19:10:52.236625 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:10:52.236497    4888 scope.go:122] "RemoveContainer" containerID="c8ce1ce99d508fe76b4ba8ba847e8033ecbbb0ee6be74e03754ba147696d2d84"
Oct 30 19:10:52.244324 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:10:52.236693    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-c682f"
Oct 30 19:10:52.254925 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:10:52.254902    4888 scope.go:122] "RemoveContainer" containerID="c8ce1ce99d508fe76b4ba8ba847e8033ecbbb0ee6be74e03754ba147696d2d84"
Oct 30 19:10:52.255185 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:10:52.255166    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c8ce1ce99d508fe76b4ba8ba847e8033ecbbb0ee6be74e03754ba147696d2d84\": container with ID starting with c8ce1ce99d508fe76b4ba8ba847e8033ecbbb0ee6be74e03754ba147696d2d84 not found: ID does not exist" containerID="c8ce1ce99d508fe76b4ba8ba847e8033ecbbb0ee6be74e03754ba147696d2d84"
Oct 30 19:10:52.255238 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:10:52.255193    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c8ce1ce99d508fe76b4ba8ba847e8033ecbbb0ee6be74e03754ba147696d2d84"} err="failed to get container status \"c8ce1ce99d508fe76b4ba8ba847e8033ecbbb0ee6be74e03754ba147696d2d84\": rpc error: code = NotFound desc = could not find container \"c8ce1ce99d508fe76b4ba8ba847e8033ecbbb0ee6be74e03754ba147696d2d84\": container with ID starting with c8ce1ce99d508fe76b4ba8ba847e8033ecbbb0ee6be74e03754ba147696d2d84 not found: ID does not exist"
Oct 30 19:10:52.284386 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:10:52.284354    4888 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6462e226-4e52-40da-8fbd-656be33ed133-istio-envoy\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:10:52.284543 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:10:52.284409    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wxnvm\" (UniqueName: \"kubernetes.io/projected/6462e226-4e52-40da-8fbd-656be33ed133-kube-api-access-wxnvm\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:10:52.284543 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:10:52.284420    4888 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6462e226-4e52-40da-8fbd-656be33ed133-istio-podinfo\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:10:52.284543 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:10:52.284429    4888 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6462e226-4e52-40da-8fbd-656be33ed133-istio-token\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:10:52.411272 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:10:52.411228    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-c682f"]
Oct 30 19:10:52.418569 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:10:52.418541    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-c682f"]
Oct 30 19:10:53.890948 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:10:53.890909    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6462e226-4e52-40da-8fbd-656be33ed133" path="/var/lib/kubelet/pods/6462e226-4e52-40da-8fbd-656be33ed133/volumes"
Oct 30 19:11:00.388360 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:11:00.388325    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-5a035d-865f889459-xx7rn"]
Oct 30 19:11:00.388965 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:11:00.388947    4888 cpu_manager.go:414] "containerMap: removing container" podUID="29c22c2d-64fe-476b-8b35-eab525253a5c" containerName="istio-proxy"
Oct 30 19:11:00.389024 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:11:00.388967    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="29c22c2d-64fe-476b-8b35-eab525253a5c" containerName="istio-proxy"
Oct 30 19:11:00.389024 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:11:00.388990    4888 cpu_manager.go:414] "containerMap: removing container" podUID="f9c90752-5508-445c-823e-3d9ea37f54b2" containerName="istio-proxy"
Oct 30 19:11:00.389024 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:11:00.388995    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f9c90752-5508-445c-823e-3d9ea37f54b2" containerName="istio-proxy"
Oct 30 19:11:00.389024 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:11:00.389011    4888 cpu_manager.go:414] "containerMap: removing container" podUID="6462e226-4e52-40da-8fbd-656be33ed133" containerName="istio-proxy"
Oct 30 19:11:00.389024 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:11:00.389016    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6462e226-4e52-40da-8fbd-656be33ed133" containerName="istio-proxy"
Oct 30 19:11:00.389176 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:11:00.389028    4888 cpu_manager.go:414] "containerMap: removing container" podUID="29c22c2d-64fe-476b-8b35-eab525253a5c" containerName="istio-proxy"
Oct 30 19:11:00.389176 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:11:00.389033    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="29c22c2d-64fe-476b-8b35-eab525253a5c" containerName="istio-proxy"
Oct 30 19:11:00.389176 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:11:00.389135    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="29c22c2d-64fe-476b-8b35-eab525253a5c" containerName="istio-proxy"
Oct 30 19:11:00.389176 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:11:00.389148    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="6462e226-4e52-40da-8fbd-656be33ed133" containerName="istio-proxy"
Oct 30 19:11:00.394078 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:11:00.394057    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-5a035d-865f889459-xx7rn"
Oct 30 19:11:00.501926 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:11:00.501889    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-praxis-b-5a035d-serving-cert\""
Oct 30 19:11:00.510860 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:11:00.509386    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-5a035d-865f889459-xx7rn"]
Oct 30 19:11:00.519335 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:11:00.516859    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vrkfz\" (UniqueName: \"kubernetes.io/projected/b141f1fa-213b-4816-b5c4-420693daba2f-kube-api-access-vrkfz\") pod \"maas-api-e2e-praxis-b-5a035d-865f889459-xx7rn\" (UID: \"b141f1fa-213b-4816-b5c4-420693daba2f\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-5a035d-865f889459-xx7rn"
Oct 30 19:11:00.519335 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:11:00.516920    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b141f1fa-213b-4816-b5c4-420693daba2f-openshift-service-ca\") pod \"maas-api-e2e-praxis-b-5a035d-865f889459-xx7rn\" (UID: \"b141f1fa-213b-4816-b5c4-420693daba2f\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-5a035d-865f889459-xx7rn"
Oct 30 19:11:00.519335 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:11:00.516975    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b141f1fa-213b-4816-b5c4-420693daba2f-maas-api-tls\") pod \"maas-api-e2e-praxis-b-5a035d-865f889459-xx7rn\" (UID: \"b141f1fa-213b-4816-b5c4-420693daba2f\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-5a035d-865f889459-xx7rn"
Oct 30 19:11:00.519335 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:11:00.517030    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b141f1fa-213b-4816-b5c4-420693daba2f-metrics-certs\") pod \"maas-api-e2e-praxis-b-5a035d-865f889459-xx7rn\" (UID: \"b141f1fa-213b-4816-b5c4-420693daba2f\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-5a035d-865f889459-xx7rn"
Oct 30 19:11:00.617904 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:11:00.617863    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vrkfz\" (UniqueName: \"kubernetes.io/projected/b141f1fa-213b-4816-b5c4-420693daba2f-kube-api-access-vrkfz\") pod \"maas-api-e2e-praxis-b-5a035d-865f889459-xx7rn\" (UID: \"b141f1fa-213b-4816-b5c4-420693daba2f\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-5a035d-865f889459-xx7rn"
Oct 30 19:11:00.618142 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:11:00.617935    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b141f1fa-213b-4816-b5c4-420693daba2f-openshift-service-ca\") pod \"maas-api-e2e-praxis-b-5a035d-865f889459-xx7rn\" (UID: \"b141f1fa-213b-4816-b5c4-420693daba2f\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-5a035d-865f889459-xx7rn"
Oct 30 19:11:00.618142 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:11:00.617994    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b141f1fa-213b-4816-b5c4-420693daba2f-maas-api-tls\") pod \"maas-api-e2e-praxis-b-5a035d-865f889459-xx7rn\" (UID: \"b141f1fa-213b-4816-b5c4-420693daba2f\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-5a035d-865f889459-xx7rn"
Oct 30 19:11:00.618142 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:11:00.618050    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b141f1fa-213b-4816-b5c4-420693daba2f-metrics-certs\") pod \"maas-api-e2e-praxis-b-5a035d-865f889459-xx7rn\" (UID: \"b141f1fa-213b-4816-b5c4-420693daba2f\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-5a035d-865f889459-xx7rn"
Oct 30 19:11:00.618850 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:11:00.618822    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b141f1fa-213b-4816-b5c4-420693daba2f-openshift-service-ca\") pod \"maas-api-e2e-praxis-b-5a035d-865f889459-xx7rn\" (UID: \"b141f1fa-213b-4816-b5c4-420693daba2f\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-5a035d-865f889459-xx7rn"
Oct 30 19:11:00.621264 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:11:00.621241    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b141f1fa-213b-4816-b5c4-420693daba2f-maas-api-tls\") pod \"maas-api-e2e-praxis-b-5a035d-865f889459-xx7rn\" (UID: \"b141f1fa-213b-4816-b5c4-420693daba2f\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-5a035d-865f889459-xx7rn"
Oct 30 19:11:00.621429 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:11:00.621406    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b141f1fa-213b-4816-b5c4-420693daba2f-metrics-certs\") pod \"maas-api-e2e-praxis-b-5a035d-865f889459-xx7rn\" (UID: \"b141f1fa-213b-4816-b5c4-420693daba2f\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-5a035d-865f889459-xx7rn"
Oct 30 19:11:00.683485 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:11:00.683412    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vrkfz\" (UniqueName: \"kubernetes.io/projected/b141f1fa-213b-4816-b5c4-420693daba2f-kube-api-access-vrkfz\") pod \"maas-api-e2e-praxis-b-5a035d-865f889459-xx7rn\" (UID: \"b141f1fa-213b-4816-b5c4-420693daba2f\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-5a035d-865f889459-xx7rn"
Oct 30 19:11:00.708590 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:11:00.708560    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-5a035d-865f889459-xx7rn"
Oct 30 19:11:01.105140 ip-10-0-134-46 kubenswrapper[4888]: W1030 19:11:01.105102    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb141f1fa_213b_4816_b5c4_420693daba2f.slice/crio-73790fe8809d496735c1919190e4dec2c7c669b8911e15a143083df4099e6894 WatchSource:0}: Error finding container 73790fe8809d496735c1919190e4dec2c7c669b8911e15a143083df4099e6894: Status 404 returned error can't find the container with id 73790fe8809d496735c1919190e4dec2c7c669b8911e15a143083df4099e6894
Oct 30 19:11:01.109773 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:11:01.108776    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-5a035d-865f889459-xx7rn"]
Oct 30 19:11:01.311880 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:11:01.311839    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-5a035d-865f889459-xx7rn" event={"ID":"b141f1fa-213b-4816-b5c4-420693daba2f","Type":"ContainerStarted","Data":"73790fe8809d496735c1919190e4dec2c7c669b8911e15a143083df4099e6894"}
Oct 30 19:11:02.320655 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:11:02.320620    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-5a035d-865f889459-xx7rn" event={"ID":"b141f1fa-213b-4816-b5c4-420693daba2f","Type":"ContainerStarted","Data":"e54895f2dbbba86baecdd6192bd9f0d429fa7cb6acf79e77fb95b940f31cfe06"}
Oct 30 19:11:02.321120 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:11:02.320684    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-5a035d-865f889459-xx7rn"
Oct 30 19:11:02.447335 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:11:02.447220    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-5a035d-865f889459-xx7rn" podStartSLOduration=1.444396493 podStartE2EDuration="2.447208863s" totalImagesPullingTime="1.00281237s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:11:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:11:02.442336425 +0000 UTC m=+2226.689742450" watchObservedRunningTime="2026-10-30 19:11:02.447208863 +0000 UTC m=+2226.694614887"
Oct 30 19:11:05.623715 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:11:05.623681    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-jbsw6" containerName="extractor"
Oct 30 19:11:08.335005 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:11:08.334980    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-5a035d-865f889459-xx7rn"
Oct 30 19:11:19.372253 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:11:19.372219    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-4ebf78-67779b8999-78snt"]
Oct 30 19:11:19.372675 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:11:19.372461    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-4ebf78-67779b8999-78snt" podUID="79f94bb4-e014-4bee-b2e2-0e823f166b70" containerName="maas-api" containerID="cri-o://c574077631f8dcd3cc21edfc97720813cbdd87e321c533356d7c117a3dfa4a4e" gracePeriod=30
Oct 30 19:11:19.682955 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:11:19.682928    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-4ebf78-67779b8999-78snt"
Oct 30 19:11:19.747782 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:11:19.747751    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/79f94bb4-e014-4bee-b2e2-0e823f166b70-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/79f94bb4-e014-4bee-b2e2-0e823f166b70-maas-api-tls\") pod \"79f94bb4-e014-4bee-b2e2-0e823f166b70\" (UID: \"79f94bb4-e014-4bee-b2e2-0e823f166b70\") "
Oct 30 19:11:19.747949 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:11:19.747796    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/79f94bb4-e014-4bee-b2e2-0e823f166b70-kube-api-access-zqtnr\" (UniqueName: \"kubernetes.io/projected/79f94bb4-e014-4bee-b2e2-0e823f166b70-kube-api-access-zqtnr\") pod \"79f94bb4-e014-4bee-b2e2-0e823f166b70\" (UID: \"79f94bb4-e014-4bee-b2e2-0e823f166b70\") "
Oct 30 19:11:19.747949 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:11:19.747907    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/79f94bb4-e014-4bee-b2e2-0e823f166b70-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/79f94bb4-e014-4bee-b2e2-0e823f166b70-openshift-service-ca\") pod \"79f94bb4-e014-4bee-b2e2-0e823f166b70\" (UID: \"79f94bb4-e014-4bee-b2e2-0e823f166b70\") "
Oct 30 19:11:19.748365 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:11:19.748336    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/79f94bb4-e014-4bee-b2e2-0e823f166b70-openshift-service-ca" pod "79f94bb4-e014-4bee-b2e2-0e823f166b70" (UID: "79f94bb4-e014-4bee-b2e2-0e823f166b70"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:11:19.749973 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:11:19.749949    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/79f94bb4-e014-4bee-b2e2-0e823f166b70-maas-api-tls" pod "79f94bb4-e014-4bee-b2e2-0e823f166b70" (UID: "79f94bb4-e014-4bee-b2e2-0e823f166b70"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:11:19.750148 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:11:19.750132    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/79f94bb4-e014-4bee-b2e2-0e823f166b70-kube-api-access-zqtnr" pod "79f94bb4-e014-4bee-b2e2-0e823f166b70" (UID: "79f94bb4-e014-4bee-b2e2-0e823f166b70"). InnerVolumeSpecName "kube-api-access-zqtnr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:11:19.851366 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:11:19.851331    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/79f94bb4-e014-4bee-b2e2-0e823f166b70-metrics-certs\" (UniqueName: \"kubernetes.io/secret/79f94bb4-e014-4bee-b2e2-0e823f166b70-metrics-certs\") pod \"79f94bb4-e014-4bee-b2e2-0e823f166b70\" (UID: \"79f94bb4-e014-4bee-b2e2-0e823f166b70\") "
Oct 30 19:11:19.851894 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:11:19.851869    4888 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/79f94bb4-e014-4bee-b2e2-0e823f166b70-maas-api-tls\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:11:19.852005 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:11:19.851897    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zqtnr\" (UniqueName: \"kubernetes.io/projected/79f94bb4-e014-4bee-b2e2-0e823f166b70-kube-api-access-zqtnr\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:11:19.852005 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:11:19.851912    4888 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/79f94bb4-e014-4bee-b2e2-0e823f166b70-openshift-service-ca\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:11:19.860590 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:11:19.860560    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/79f94bb4-e014-4bee-b2e2-0e823f166b70-metrics-certs" pod "79f94bb4-e014-4bee-b2e2-0e823f166b70" (UID: "79f94bb4-e014-4bee-b2e2-0e823f166b70"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:11:19.954000 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:11:19.953262    4888 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/79f94bb4-e014-4bee-b2e2-0e823f166b70-metrics-certs\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:11:20.454751 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:11:20.454721    4888 generic.go:356] "Generic (PLEG): container finished" podID="79f94bb4-e014-4bee-b2e2-0e823f166b70" containerID="c574077631f8dcd3cc21edfc97720813cbdd87e321c533356d7c117a3dfa4a4e" exitCode=0
Oct 30 19:11:20.455237 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:11:20.454785    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-4ebf78-67779b8999-78snt"
Oct 30 19:11:20.455237 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:11:20.454808    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-4ebf78-67779b8999-78snt" event={"ID":"79f94bb4-e014-4bee-b2e2-0e823f166b70","Type":"ContainerDied","Data":"c574077631f8dcd3cc21edfc97720813cbdd87e321c533356d7c117a3dfa4a4e"}
Oct 30 19:11:20.455237 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:11:20.454858    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-4ebf78-67779b8999-78snt" event={"ID":"79f94bb4-e014-4bee-b2e2-0e823f166b70","Type":"ContainerDied","Data":"2c049a9001b7c777ac645539bced752a21ef993c21680b0f38bcbc8e11b44d7e"}
Oct 30 19:11:20.455237 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:11:20.454886    4888 scope.go:122] "RemoveContainer" containerID="c574077631f8dcd3cc21edfc97720813cbdd87e321c533356d7c117a3dfa4a4e"
Oct 30 19:11:20.465031 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:11:20.465015    4888 scope.go:122] "RemoveContainer" containerID="c574077631f8dcd3cc21edfc97720813cbdd87e321c533356d7c117a3dfa4a4e"
Oct 30 19:11:20.465335 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:11:20.465314    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c574077631f8dcd3cc21edfc97720813cbdd87e321c533356d7c117a3dfa4a4e\": container with ID starting with c574077631f8dcd3cc21edfc97720813cbdd87e321c533356d7c117a3dfa4a4e not found: ID does not exist" containerID="c574077631f8dcd3cc21edfc97720813cbdd87e321c533356d7c117a3dfa4a4e"
Oct 30 19:11:20.465433 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:11:20.465340    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c574077631f8dcd3cc21edfc97720813cbdd87e321c533356d7c117a3dfa4a4e"} err="failed to get container status \"c574077631f8dcd3cc21edfc97720813cbdd87e321c533356d7c117a3dfa4a4e\": rpc error: code = NotFound desc = could not find container \"c574077631f8dcd3cc21edfc97720813cbdd87e321c533356d7c117a3dfa4a4e\": container with ID starting with c574077631f8dcd3cc21edfc97720813cbdd87e321c533356d7c117a3dfa4a4e not found: ID does not exist"
Oct 30 19:11:20.634161 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:11:20.634127    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-4ebf78-67779b8999-78snt"]
Oct 30 19:11:20.687165 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:11:20.687083    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-4ebf78-67779b8999-78snt"]
Oct 30 19:11:21.893326 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:11:21.893267    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="79f94bb4-e014-4bee-b2e2-0e823f166b70" path="/var/lib/kubelet/pods/79f94bb4-e014-4bee-b2e2-0e823f166b70/volumes"
Oct 30 19:11:26.636115 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:11:26.636077    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-rjmdz"]
Oct 30 19:11:31.960223 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:11:31.960185    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-praxis-a-f554bb-openshift-default-5f8f5599d5-x2l7h"]
Oct 30 19:11:31.961071 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:11:31.961052    4888 cpu_manager.go:414] "containerMap: removing container" podUID="79f94bb4-e014-4bee-b2e2-0e823f166b70" containerName="maas-api"
Oct 30 19:11:31.961127 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:11:31.961075    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="79f94bb4-e014-4bee-b2e2-0e823f166b70" containerName="maas-api"
Oct 30 19:11:31.961249 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:11:31.961237    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="79f94bb4-e014-4bee-b2e2-0e823f166b70" containerName="maas-api"
Oct 30 19:11:31.961302 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:11:31.961259    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="29c22c2d-64fe-476b-8b35-eab525253a5c" containerName="istio-proxy"
Oct 30 19:11:31.966766 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:11:31.966745    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-a-f554bb-openshift-default-5f8f5599d5-x2l7h"
Oct 30 19:11:32.020706 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:11:32.020678    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-praxis-a-f554bb-openshift-default-dockercfg-c5hsj\""
Oct 30 19:11:32.075681 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:11:32.075650    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-praxis-a-f554bb-openshift-default-5f8f5599d5-x2l7h"]
Oct 30 19:11:32.121139 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:11:32.116094    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0c0bee10-4345-43a8-9008-b60054dc6850-istio-data\") pod \"e2e-praxis-a-f554bb-openshift-default-5f8f5599d5-x2l7h\" (UID: \"0c0bee10-4345-43a8-9008-b60054dc6850\") " pod="openshift-ingress/e2e-praxis-a-f554bb-openshift-default-5f8f5599d5-x2l7h"
Oct 30 19:11:32.121139 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:11:32.116157    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0c0bee10-4345-43a8-9008-b60054dc6850-istio-token\") pod \"e2e-praxis-a-f554bb-openshift-default-5f8f5599d5-x2l7h\" (UID: \"0c0bee10-4345-43a8-9008-b60054dc6850\") " pod="openshift-ingress/e2e-praxis-a-f554bb-openshift-default-5f8f5599d5-x2l7h"
Oct 30 19:11:32.121139 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:11:32.116191    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0c0bee10-4345-43a8-9008-b60054dc6850-istio-podinfo\") pod \"e2e-praxis-a-f554bb-openshift-default-5f8f5599d5-x2l7h\" (UID: \"0c0bee10-4345-43a8-9008-b60054dc6850\") " pod="openshift-ingress/e2e-praxis-a-f554bb-openshift-default-5f8f5599d5-x2l7h"
Oct 30 19:11:32.121139 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:11:32.116232    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-286jf\" (UniqueName: \"kubernetes.io/projected/0c0bee10-4345-43a8-9008-b60054dc6850-kube-api-access-286jf\") pod \"e2e-praxis-a-f554bb-openshift-default-5f8f5599d5-x2l7h\" (UID: \"0c0bee10-4345-43a8-9008-b60054dc6850\") " pod="openshift-ingress/e2e-praxis-a-f554bb-openshift-default-5f8f5599d5-x2l7h"
Oct 30 19:11:32.121139 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:11:32.116320    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0c0bee10-4345-43a8-9008-b60054dc6850-istio-envoy\") pod \"e2e-praxis-a-f554bb-openshift-default-5f8f5599d5-x2l7h\" (UID: \"0c0bee10-4345-43a8-9008-b60054dc6850\") " pod="openshift-ingress/e2e-praxis-a-f554bb-openshift-default-5f8f5599d5-x2l7h"
Oct 30 19:11:32.121139 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:11:32.116435    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0c0bee10-4345-43a8-9008-b60054dc6850-istiod-ca-cert\") pod \"e2e-praxis-a-f554bb-openshift-default-5f8f5599d5-x2l7h\" (UID: \"0c0bee10-4345-43a8-9008-b60054dc6850\") " pod="openshift-ingress/e2e-praxis-a-f554bb-openshift-default-5f8f5599d5-x2l7h"
Oct 30 19:11:32.121139 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:11:32.116533    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c0bee10-4345-43a8-9008-b60054dc6850-workload-socket\") pod \"e2e-praxis-a-f554bb-openshift-default-5f8f5599d5-x2l7h\" (UID: \"0c0bee10-4345-43a8-9008-b60054dc6850\") " pod="openshift-ingress/e2e-praxis-a-f554bb-openshift-default-5f8f5599d5-x2l7h"
Oct 30 19:11:32.121139 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:11:32.116561    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c0bee10-4345-43a8-9008-b60054dc6850-credential-socket\") pod \"e2e-praxis-a-f554bb-openshift-default-5f8f5599d5-x2l7h\" (UID: \"0c0bee10-4345-43a8-9008-b60054dc6850\") " pod="openshift-ingress/e2e-praxis-a-f554bb-openshift-default-5f8f5599d5-x2l7h"
Oct 30 19:11:32.121139 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:11:32.116611    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0c0bee10-4345-43a8-9008-b60054dc6850-workload-certs\") pod \"e2e-praxis-a-f554bb-openshift-default-5f8f5599d5-x2l7h\" (UID: \"0c0bee10-4345-43a8-9008-b60054dc6850\") " pod="openshift-ingress/e2e-praxis-a-f554bb-openshift-default-5f8f5599d5-x2l7h"
Oct 30 19:11:32.217593 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:11:32.217509    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-286jf\" (UniqueName: \"kubernetes.io/projected/0c0bee10-4345-43a8-9008-b60054dc6850-kube-api-access-286jf\") pod \"e2e-praxis-a-f554bb-openshift-default-5f8f5599d5-x2l7h\" (UID: \"0c0bee10-4345-43a8-9008-b60054dc6850\") " pod="openshift-ingress/e2e-praxis-a-f554bb-openshift-default-5f8f5599d5-x2l7h"
Oct 30 19:11:32.217593 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:11:32.217567    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0c0bee10-4345-43a8-9008-b60054dc6850-istio-envoy\") pod \"e2e-praxis-a-f554bb-openshift-default-5f8f5599d5-x2l7h\" (UID: \"0c0bee10-4345-43a8-9008-b60054dc6850\") " pod="openshift-ingress/e2e-praxis-a-f554bb-openshift-default-5f8f5599d5-x2l7h"
Oct 30 19:11:32.217840 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:11:32.217638    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0c0bee10-4345-43a8-9008-b60054dc6850-istiod-ca-cert\") pod \"e2e-praxis-a-f554bb-openshift-default-5f8f5599d5-x2l7h\" (UID: \"0c0bee10-4345-43a8-9008-b60054dc6850\") " pod="openshift-ingress/e2e-praxis-a-f554bb-openshift-default-5f8f5599d5-x2l7h"
Oct 30 19:11:32.217840 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:11:32.217702    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c0bee10-4345-43a8-9008-b60054dc6850-workload-socket\") pod \"e2e-praxis-a-f554bb-openshift-default-5f8f5599d5-x2l7h\" (UID: \"0c0bee10-4345-43a8-9008-b60054dc6850\") " pod="openshift-ingress/e2e-praxis-a-f554bb-openshift-default-5f8f5599d5-x2l7h"
Oct 30 19:11:32.217840 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:11:32.217719    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c0bee10-4345-43a8-9008-b60054dc6850-credential-socket\") pod \"e2e-praxis-a-f554bb-openshift-default-5f8f5599d5-x2l7h\" (UID: \"0c0bee10-4345-43a8-9008-b60054dc6850\") " pod="openshift-ingress/e2e-praxis-a-f554bb-openshift-default-5f8f5599d5-x2l7h"
Oct 30 19:11:32.217840 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:11:32.217760    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0c0bee10-4345-43a8-9008-b60054dc6850-workload-certs\") pod \"e2e-praxis-a-f554bb-openshift-default-5f8f5599d5-x2l7h\" (UID: \"0c0bee10-4345-43a8-9008-b60054dc6850\") " pod="openshift-ingress/e2e-praxis-a-f554bb-openshift-default-5f8f5599d5-x2l7h"
Oct 30 19:11:32.217840 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:11:32.217806    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0c0bee10-4345-43a8-9008-b60054dc6850-istio-data\") pod \"e2e-praxis-a-f554bb-openshift-default-5f8f5599d5-x2l7h\" (UID: \"0c0bee10-4345-43a8-9008-b60054dc6850\") " pod="openshift-ingress/e2e-praxis-a-f554bb-openshift-default-5f8f5599d5-x2l7h"
Oct 30 19:11:32.217840 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:11:32.217840    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0c0bee10-4345-43a8-9008-b60054dc6850-istio-token\") pod \"e2e-praxis-a-f554bb-openshift-default-5f8f5599d5-x2l7h\" (UID: \"0c0bee10-4345-43a8-9008-b60054dc6850\") " pod="openshift-ingress/e2e-praxis-a-f554bb-openshift-default-5f8f5599d5-x2l7h"
Oct 30 19:11:32.218147 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:11:32.217925    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0c0bee10-4345-43a8-9008-b60054dc6850-istio-podinfo\") pod \"e2e-praxis-a-f554bb-openshift-default-5f8f5599d5-x2l7h\" (UID: \"0c0bee10-4345-43a8-9008-b60054dc6850\") " pod="openshift-ingress/e2e-praxis-a-f554bb-openshift-default-5f8f5599d5-x2l7h"
Oct 30 19:11:32.218147 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:11:32.218111    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c0bee10-4345-43a8-9008-b60054dc6850-workload-socket\") pod \"e2e-praxis-a-f554bb-openshift-default-5f8f5599d5-x2l7h\" (UID: \"0c0bee10-4345-43a8-9008-b60054dc6850\") " pod="openshift-ingress/e2e-praxis-a-f554bb-openshift-default-5f8f5599d5-x2l7h"
Oct 30 19:11:32.218242 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:11:32.218171    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c0bee10-4345-43a8-9008-b60054dc6850-credential-socket\") pod \"e2e-praxis-a-f554bb-openshift-default-5f8f5599d5-x2l7h\" (UID: \"0c0bee10-4345-43a8-9008-b60054dc6850\") " pod="openshift-ingress/e2e-praxis-a-f554bb-openshift-default-5f8f5599d5-x2l7h"
Oct 30 19:11:32.218424 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:11:32.218398    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0c0bee10-4345-43a8-9008-b60054dc6850-workload-certs\") pod \"e2e-praxis-a-f554bb-openshift-default-5f8f5599d5-x2l7h\" (UID: \"0c0bee10-4345-43a8-9008-b60054dc6850\") " pod="openshift-ingress/e2e-praxis-a-f554bb-openshift-default-5f8f5599d5-x2l7h"
Oct 30 19:11:32.218558 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:11:32.218544    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0c0bee10-4345-43a8-9008-b60054dc6850-istiod-ca-cert\") pod \"e2e-praxis-a-f554bb-openshift-default-5f8f5599d5-x2l7h\" (UID: \"0c0bee10-4345-43a8-9008-b60054dc6850\") " pod="openshift-ingress/e2e-praxis-a-f554bb-openshift-default-5f8f5599d5-x2l7h"
Oct 30 19:11:32.218597 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:11:32.218550    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0c0bee10-4345-43a8-9008-b60054dc6850-istio-data\") pod \"e2e-praxis-a-f554bb-openshift-default-5f8f5599d5-x2l7h\" (UID: \"0c0bee10-4345-43a8-9008-b60054dc6850\") " pod="openshift-ingress/e2e-praxis-a-f554bb-openshift-default-5f8f5599d5-x2l7h"
Oct 30 19:11:32.220158 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:11:32.220138    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0c0bee10-4345-43a8-9008-b60054dc6850-istio-envoy\") pod \"e2e-praxis-a-f554bb-openshift-default-5f8f5599d5-x2l7h\" (UID: \"0c0bee10-4345-43a8-9008-b60054dc6850\") " pod="openshift-ingress/e2e-praxis-a-f554bb-openshift-default-5f8f5599d5-x2l7h"
Oct 30 19:11:32.220558 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:11:32.220541    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0c0bee10-4345-43a8-9008-b60054dc6850-istio-podinfo\") pod \"e2e-praxis-a-f554bb-openshift-default-5f8f5599d5-x2l7h\" (UID: \"0c0bee10-4345-43a8-9008-b60054dc6850\") " pod="openshift-ingress/e2e-praxis-a-f554bb-openshift-default-5f8f5599d5-x2l7h"
Oct 30 19:11:32.281253 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:11:32.281233    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0c0bee10-4345-43a8-9008-b60054dc6850-istio-token\") pod \"e2e-praxis-a-f554bb-openshift-default-5f8f5599d5-x2l7h\" (UID: \"0c0bee10-4345-43a8-9008-b60054dc6850\") " pod="openshift-ingress/e2e-praxis-a-f554bb-openshift-default-5f8f5599d5-x2l7h"
Oct 30 19:11:32.281586 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:11:32.281563    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-286jf\" (UniqueName: \"kubernetes.io/projected/0c0bee10-4345-43a8-9008-b60054dc6850-kube-api-access-286jf\") pod \"e2e-praxis-a-f554bb-openshift-default-5f8f5599d5-x2l7h\" (UID: \"0c0bee10-4345-43a8-9008-b60054dc6850\") " pod="openshift-ingress/e2e-praxis-a-f554bb-openshift-default-5f8f5599d5-x2l7h"
Oct 30 19:11:32.581115 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:11:32.581092    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-a-f554bb-openshift-default-5f8f5599d5-x2l7h"
Oct 30 19:11:33.018789 ip-10-0-134-46 kubenswrapper[4888]: W1030 19:11:33.018764    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c0bee10_4345_43a8_9008_b60054dc6850.slice/crio-19afd348df2f13fa7eac18f7dfae5f8dace8bb663d5ad625584285a564fd32fa WatchSource:0}: Error finding container 19afd348df2f13fa7eac18f7dfae5f8dace8bb663d5ad625584285a564fd32fa: Status 404 returned error can't find the container with id 19afd348df2f13fa7eac18f7dfae5f8dace8bb663d5ad625584285a564fd32fa
Oct 30 19:11:33.021187 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:11:33.021154    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:11:33.021276 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:11:33.021219    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:11:33.021276 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:11:33.021250    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:11:33.053044 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:11:33.053010    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-praxis-a-f554bb-openshift-default-5f8f5599d5-x2l7h"]
Oct 30 19:11:33.555117 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:11:33.555081    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-f554bb-openshift-default-5f8f5599d5-x2l7h" event={"ID":"0c0bee10-4345-43a8-9008-b60054dc6850","Type":"ContainerStarted","Data":"7c68227f98c1f0fb0f8bdd8e71b946c750cb37fc4c933852dba832d92a90c7e1"}
Oct 30 19:11:33.555351 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:11:33.555131    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-f554bb-openshift-default-5f8f5599d5-x2l7h" event={"ID":"0c0bee10-4345-43a8-9008-b60054dc6850","Type":"ContainerStarted","Data":"19afd348df2f13fa7eac18f7dfae5f8dace8bb663d5ad625584285a564fd32fa"}
Oct 30 19:11:33.581704 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:11:33.581672    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-praxis-a-f554bb-openshift-default-5f8f5599d5-x2l7h"
Oct 30 19:11:33.731937 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:11:33.731886    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-praxis-a-f554bb-openshift-default-5f8f5599d5-x2l7h" podStartSLOduration=2.731872691 podStartE2EDuration="2.731872691s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:11:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:11:33.729770875 +0000 UTC m=+2257.977176900" watchObservedRunningTime="2026-10-30 19:11:33.731872691 +0000 UTC m=+2257.979278763"
Oct 30 19:11:34.587439 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:11:34.587409    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-praxis-a-f554bb-openshift-default-5f8f5599d5-x2l7h"
Oct 30 19:11:35.570944 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:11:35.570909    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-praxis-a-f554bb-openshift-default-5f8f5599d5-x2l7h"
Oct 30 19:11:35.571922 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:11:35.571902    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-praxis-a-f554bb-openshift-default-5f8f5599d5-x2l7h"
Oct 30 19:11:35.620039 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:11:35.620004    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-jbsw6" containerName="extractor"
Oct 30 19:12:05.618301 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:12:05.618206    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-jbsw6" containerName="extractor"
Oct 30 19:12:16.277935 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:12:16.277902    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-rjmdz"]
Oct 30 19:12:35.618430 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:12:35.618394    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-jbsw6" containerName="extractor"
Oct 30 19:12:44.551396 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:12:44.551362    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-52d383-6f847499f-hmbjs"]
Oct 30 19:12:44.555697 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:12:44.555677    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-52d383-6f847499f-hmbjs"
Oct 30 19:12:44.609316 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:12:44.609267    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-52d383-serving-cert\""
Oct 30 19:12:44.664081 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:12:44.664048    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-52d383-6f847499f-hmbjs"]
Oct 30 19:12:44.704789 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:12:44.704750    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fvgfm\" (UniqueName: \"kubernetes.io/projected/dd3ecf9f-3563-4640-aec5-bb83ed70f726-kube-api-access-fvgfm\") pod \"maas-api-e2e-ipp-a-52d383-6f847499f-hmbjs\" (UID: \"dd3ecf9f-3563-4640-aec5-bb83ed70f726\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-52d383-6f847499f-hmbjs"
Oct 30 19:12:44.705017 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:12:44.704988    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dd3ecf9f-3563-4640-aec5-bb83ed70f726-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-52d383-6f847499f-hmbjs\" (UID: \"dd3ecf9f-3563-4640-aec5-bb83ed70f726\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-52d383-6f847499f-hmbjs"
Oct 30 19:12:44.705141 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:12:44.705123    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dd3ecf9f-3563-4640-aec5-bb83ed70f726-metrics-certs\") pod \"maas-api-e2e-ipp-a-52d383-6f847499f-hmbjs\" (UID: \"dd3ecf9f-3563-4640-aec5-bb83ed70f726\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-52d383-6f847499f-hmbjs"
Oct 30 19:12:44.705272 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:12:44.705250    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dd3ecf9f-3563-4640-aec5-bb83ed70f726-maas-api-tls\") pod \"maas-api-e2e-ipp-a-52d383-6f847499f-hmbjs\" (UID: \"dd3ecf9f-3563-4640-aec5-bb83ed70f726\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-52d383-6f847499f-hmbjs"
Oct 30 19:12:44.809113 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:12:44.808966    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dd3ecf9f-3563-4640-aec5-bb83ed70f726-metrics-certs\") pod \"maas-api-e2e-ipp-a-52d383-6f847499f-hmbjs\" (UID: \"dd3ecf9f-3563-4640-aec5-bb83ed70f726\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-52d383-6f847499f-hmbjs"
Oct 30 19:12:44.809113 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:12:44.809087    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dd3ecf9f-3563-4640-aec5-bb83ed70f726-maas-api-tls\") pod \"maas-api-e2e-ipp-a-52d383-6f847499f-hmbjs\" (UID: \"dd3ecf9f-3563-4640-aec5-bb83ed70f726\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-52d383-6f847499f-hmbjs"
Oct 30 19:12:44.809810 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:12:44.809773    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fvgfm\" (UniqueName: \"kubernetes.io/projected/dd3ecf9f-3563-4640-aec5-bb83ed70f726-kube-api-access-fvgfm\") pod \"maas-api-e2e-ipp-a-52d383-6f847499f-hmbjs\" (UID: \"dd3ecf9f-3563-4640-aec5-bb83ed70f726\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-52d383-6f847499f-hmbjs"
Oct 30 19:12:44.809936 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:12:44.809905    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dd3ecf9f-3563-4640-aec5-bb83ed70f726-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-52d383-6f847499f-hmbjs\" (UID: \"dd3ecf9f-3563-4640-aec5-bb83ed70f726\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-52d383-6f847499f-hmbjs"
Oct 30 19:12:44.810672 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:12:44.810634    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dd3ecf9f-3563-4640-aec5-bb83ed70f726-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-52d383-6f847499f-hmbjs\" (UID: \"dd3ecf9f-3563-4640-aec5-bb83ed70f726\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-52d383-6f847499f-hmbjs"
Oct 30 19:12:44.810984 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:12:44.810966    4888 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-a-52d383-serving-cert: secret "maas-api-e2e-ipp-a-52d383-serving-cert" not found
Oct 30 19:12:44.811088 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:12:44.811028    4888 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dd3ecf9f-3563-4640-aec5-bb83ed70f726-maas-api-tls podName:dd3ecf9f-3563-4640-aec5-bb83ed70f726 nodeName:}" failed. No retries permitted until 2026-10-30 19:12:45.311010873 +0000 UTC m=+2329.558416891 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/dd3ecf9f-3563-4640-aec5-bb83ed70f726-maas-api-tls") pod "maas-api-e2e-ipp-a-52d383-6f847499f-hmbjs" (UID: "dd3ecf9f-3563-4640-aec5-bb83ed70f726") : secret "maas-api-e2e-ipp-a-52d383-serving-cert" not found
Oct 30 19:12:44.816370 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:12:44.816085    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dd3ecf9f-3563-4640-aec5-bb83ed70f726-metrics-certs\") pod \"maas-api-e2e-ipp-a-52d383-6f847499f-hmbjs\" (UID: \"dd3ecf9f-3563-4640-aec5-bb83ed70f726\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-52d383-6f847499f-hmbjs"
Oct 30 19:12:44.890148 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:12:44.890118    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fvgfm\" (UniqueName: \"kubernetes.io/projected/dd3ecf9f-3563-4640-aec5-bb83ed70f726-kube-api-access-fvgfm\") pod \"maas-api-e2e-ipp-a-52d383-6f847499f-hmbjs\" (UID: \"dd3ecf9f-3563-4640-aec5-bb83ed70f726\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-52d383-6f847499f-hmbjs"
Oct 30 19:12:45.317544 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:12:45.317504    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dd3ecf9f-3563-4640-aec5-bb83ed70f726-maas-api-tls\") pod \"maas-api-e2e-ipp-a-52d383-6f847499f-hmbjs\" (UID: \"dd3ecf9f-3563-4640-aec5-bb83ed70f726\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-52d383-6f847499f-hmbjs"
Oct 30 19:12:45.320045 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:12:45.320026    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dd3ecf9f-3563-4640-aec5-bb83ed70f726-maas-api-tls\") pod \"maas-api-e2e-ipp-a-52d383-6f847499f-hmbjs\" (UID: \"dd3ecf9f-3563-4640-aec5-bb83ed70f726\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-52d383-6f847499f-hmbjs"
Oct 30 19:12:45.470118 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:12:45.470093    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-52d383-6f847499f-hmbjs"
Oct 30 19:12:45.860194 ip-10-0-134-46 kubenswrapper[4888]: W1030 19:12:45.860165    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd3ecf9f_3563_4640_aec5_bb83ed70f726.slice/crio-9890b56c31b427f9d0684073d09f1782c9baa50207a157e011e2afe40762f313 WatchSource:0}: Error finding container 9890b56c31b427f9d0684073d09f1782c9baa50207a157e011e2afe40762f313: Status 404 returned error can't find the container with id 9890b56c31b427f9d0684073d09f1782c9baa50207a157e011e2afe40762f313
Oct 30 19:12:45.861590 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:12:45.861567    4888 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 19:12:45.861966 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:12:45.861943    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-52d383-6f847499f-hmbjs"]
Oct 30 19:12:46.145187 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:12:46.145102    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-52d383-6f847499f-hmbjs" event={"ID":"dd3ecf9f-3563-4640-aec5-bb83ed70f726","Type":"ContainerStarted","Data":"9890b56c31b427f9d0684073d09f1782c9baa50207a157e011e2afe40762f313"}
Oct 30 19:12:47.153368 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:12:47.153336    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-52d383-6f847499f-hmbjs" event={"ID":"dd3ecf9f-3563-4640-aec5-bb83ed70f726","Type":"ContainerStarted","Data":"5f507b2ca96a58906a94d576f81fea6a4e09dbf32668f4a28998a476420155ab"}
Oct 30 19:12:47.153851 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:12:47.153425    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-52d383-6f847499f-hmbjs"
Oct 30 19:12:47.327155 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:12:47.327090    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-52d383-6f847499f-hmbjs" podStartSLOduration=2.538621406 podStartE2EDuration="3.327075784s" totalImagesPullingTime="788.454378ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:12:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:12:47.277260299 +0000 UTC m=+2331.524666325" watchObservedRunningTime="2026-10-30 19:12:47.327075784 +0000 UTC m=+2331.574481808"
Oct 30 19:12:53.165880 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:12:53.165849    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-52d383-6f847499f-hmbjs"
Oct 30 19:12:55.371699 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:12:55.371667    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-rjmdz"]
Oct 30 19:13:05.620204 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:13:05.620169    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-jbsw6" containerName="extractor"
Oct 30 19:13:06.258424 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:13:06.258049    4888 patch_prober.go:29] interesting pod/e2e-praxis-a-f554bb-openshift-default-5f8f5599d5-x2l7h container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.125:15021/healthz/ready\": read tcp 10.132.0.2:57318->10.132.0.125:15021: read: connection reset by peer" start-of-body=
Oct 30 19:13:06.258424 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:13:06.258125    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-praxis-a-f554bb-openshift-default-5f8f5599d5-x2l7h" podUID="0c0bee10-4345-43a8-9008-b60054dc6850" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.125:15021/healthz/ready\": read tcp 10.132.0.2:57318->10.132.0.125:15021: read: connection reset by peer"
Oct 30 19:13:06.339222 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:13:06.339194    4888 generic.go:356] "Generic (PLEG): container finished" podID="0c0bee10-4345-43a8-9008-b60054dc6850" containerID="7c68227f98c1f0fb0f8bdd8e71b946c750cb37fc4c933852dba832d92a90c7e1" exitCode=0
Oct 30 19:13:06.339390 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:13:06.339272    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-f554bb-openshift-default-5f8f5599d5-x2l7h" event={"ID":"0c0bee10-4345-43a8-9008-b60054dc6850","Type":"ContainerDied","Data":"7c68227f98c1f0fb0f8bdd8e71b946c750cb37fc4c933852dba832d92a90c7e1"}
Oct 30 19:13:06.339722 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:13:06.339706    4888 scope.go:122] "RemoveContainer" containerID="7c68227f98c1f0fb0f8bdd8e71b946c750cb37fc4c933852dba832d92a90c7e1"
Oct 30 19:13:06.341143 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:13:06.341114    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:13:06.341217 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:13:06.341193    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:13:06.341262 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:13:06.341226    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:13:07.348140 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:13:07.348109    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-f554bb-openshift-default-5f8f5599d5-x2l7h" event={"ID":"0c0bee10-4345-43a8-9008-b60054dc6850","Type":"ContainerStarted","Data":"4096fb8ce41b1b43297c7e16718d23dd852fa2009a7420f4e5a9d190da9f1843"}
Oct 30 19:13:07.588434 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:13:07.588395    4888 patch_prober.go:29] interesting pod/e2e-praxis-a-f554bb-openshift-default-5f8f5599d5-x2l7h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.125:15021/healthz/ready\": dial tcp 10.132.0.125:15021: connect: connection refused" start-of-body=
Oct 30 19:13:07.588792 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:13:07.588748    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-f554bb-openshift-default-5f8f5599d5-x2l7h" podUID="0c0bee10-4345-43a8-9008-b60054dc6850" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.125:15021/healthz/ready\": dial tcp 10.132.0.125:15021: connect: connection refused"
Oct 30 19:13:07.589636 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:13:07.588689    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-praxis-a-f554bb-openshift-default-5f8f5599d5-x2l7h"
Oct 30 19:13:08.581750 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:13:08.581712    4888 patch_prober.go:29] interesting pod/e2e-praxis-a-f554bb-openshift-default-5f8f5599d5-x2l7h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.125:15021/healthz/ready\": dial tcp 10.132.0.125:15021: connect: connection refused" start-of-body=
Oct 30 19:13:08.582379 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:13:08.582346    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-f554bb-openshift-default-5f8f5599d5-x2l7h" podUID="0c0bee10-4345-43a8-9008-b60054dc6850" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.125:15021/healthz/ready\": dial tcp 10.132.0.125:15021: connect: connection refused"
Oct 30 19:13:09.581673 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:13:09.581635    4888 patch_prober.go:29] interesting pod/e2e-praxis-a-f554bb-openshift-default-5f8f5599d5-x2l7h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.125:15021/healthz/ready\": dial tcp 10.132.0.125:15021: connect: connection refused" start-of-body=
Oct 30 19:13:09.581957 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:13:09.581928    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-f554bb-openshift-default-5f8f5599d5-x2l7h" podUID="0c0bee10-4345-43a8-9008-b60054dc6850" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.125:15021/healthz/ready\": dial tcp 10.132.0.125:15021: connect: connection refused"
Oct 30 19:13:10.427155 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:13:10.427122    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-x959p"]
Oct 30 19:13:10.427472 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:13:10.427440    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-x959p" podUID="ff78b45c-608b-4562-99d5-d4cf35e59c71" containerName="discovery" containerID="cri-o://7820fbcdf5394f25ef2fc531ceb664c18fdcf0a10a25f309ff353f2aeaa0642e" gracePeriod=30
Oct 30 19:13:10.585310 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:13:10.584698    4888 patch_prober.go:29] interesting pod/e2e-praxis-a-f554bb-openshift-default-5f8f5599d5-x2l7h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.125:15021/healthz/ready\": dial tcp 10.132.0.125:15021: connect: connection refused" start-of-body=
Oct 30 19:13:10.585310 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:13:10.584766    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-f554bb-openshift-default-5f8f5599d5-x2l7h" podUID="0c0bee10-4345-43a8-9008-b60054dc6850" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.125:15021/healthz/ready\": dial tcp 10.132.0.125:15021: connect: connection refused"
Oct 30 19:13:10.816670 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:13:10.816353    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-x959p"
Oct 30 19:13:10.915552 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:13:10.915521    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ff78b45c-608b-4562-99d5-d4cf35e59c71-local-certs\" (UniqueName: \"kubernetes.io/empty-dir/ff78b45c-608b-4562-99d5-d4cf35e59c71-local-certs\") pod \"ff78b45c-608b-4562-99d5-d4cf35e59c71\" (UID: \"ff78b45c-608b-4562-99d5-d4cf35e59c71\") "
Oct 30 19:13:10.915751 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:13:10.915603    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ff78b45c-608b-4562-99d5-d4cf35e59c71-istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ff78b45c-608b-4562-99d5-d4cf35e59c71-istio-kubeconfig\") pod \"ff78b45c-608b-4562-99d5-d4cf35e59c71\" (UID: \"ff78b45c-608b-4562-99d5-d4cf35e59c71\") "
Oct 30 19:13:10.915751 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:13:10.915649    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ff78b45c-608b-4562-99d5-d4cf35e59c71-istio-token\" (UniqueName: \"kubernetes.io/projected/ff78b45c-608b-4562-99d5-d4cf35e59c71-istio-token\") pod \"ff78b45c-608b-4562-99d5-d4cf35e59c71\" (UID: \"ff78b45c-608b-4562-99d5-d4cf35e59c71\") "
Oct 30 19:13:10.915884 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:13:10.915695    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ff78b45c-608b-4562-99d5-d4cf35e59c71-kube-api-access-qd2xz\" (UniqueName: \"kubernetes.io/projected/ff78b45c-608b-4562-99d5-d4cf35e59c71-kube-api-access-qd2xz\") pod \"ff78b45c-608b-4562-99d5-d4cf35e59c71\" (UID: \"ff78b45c-608b-4562-99d5-d4cf35e59c71\") "
Oct 30 19:13:10.916100 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:13:10.916081    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ff78b45c-608b-4562-99d5-d4cf35e59c71-cacerts\" (UniqueName: \"kubernetes.io/secret/ff78b45c-608b-4562-99d5-d4cf35e59c71-cacerts\") pod \"ff78b45c-608b-4562-99d5-d4cf35e59c71\" (UID: \"ff78b45c-608b-4562-99d5-d4cf35e59c71\") "
Oct 30 19:13:10.916182 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:13:10.916167    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ff78b45c-608b-4562-99d5-d4cf35e59c71-istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/ff78b45c-608b-4562-99d5-d4cf35e59c71-istio-csr-dns-cert\") pod \"ff78b45c-608b-4562-99d5-d4cf35e59c71\" (UID: \"ff78b45c-608b-4562-99d5-d4cf35e59c71\") "
Oct 30 19:13:10.916784 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:13:10.916757    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ff78b45c-608b-4562-99d5-d4cf35e59c71-istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/ff78b45c-608b-4562-99d5-d4cf35e59c71-istio-csr-ca-configmap\") pod \"ff78b45c-608b-4562-99d5-d4cf35e59c71\" (UID: \"ff78b45c-608b-4562-99d5-d4cf35e59c71\") "
Oct 30 19:13:10.917314 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:13:10.917267    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ff78b45c-608b-4562-99d5-d4cf35e59c71-istio-csr-ca-configmap" pod "ff78b45c-608b-4562-99d5-d4cf35e59c71" (UID: "ff78b45c-608b-4562-99d5-d4cf35e59c71"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:13:10.917498 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:13:10.917471    4888 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/ff78b45c-608b-4562-99d5-d4cf35e59c71-istio-csr-ca-configmap\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:13:10.918858 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:13:10.918828    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff78b45c-608b-4562-99d5-d4cf35e59c71-local-certs" pod "ff78b45c-608b-4562-99d5-d4cf35e59c71" (UID: "ff78b45c-608b-4562-99d5-d4cf35e59c71"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:13:10.918961 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:13:10.918931    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ff78b45c-608b-4562-99d5-d4cf35e59c71-kube-api-access-qd2xz" pod "ff78b45c-608b-4562-99d5-d4cf35e59c71" (UID: "ff78b45c-608b-4562-99d5-d4cf35e59c71"). InnerVolumeSpecName "kube-api-access-qd2xz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:13:10.919066 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:13:10.919047    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ff78b45c-608b-4562-99d5-d4cf35e59c71-istio-token" pod "ff78b45c-608b-4562-99d5-d4cf35e59c71" (UID: "ff78b45c-608b-4562-99d5-d4cf35e59c71"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:13:10.920111 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:13:10.920069    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ff78b45c-608b-4562-99d5-d4cf35e59c71-istio-kubeconfig" pod "ff78b45c-608b-4562-99d5-d4cf35e59c71" (UID: "ff78b45c-608b-4562-99d5-d4cf35e59c71"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:13:10.920111 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:13:10.920096    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ff78b45c-608b-4562-99d5-d4cf35e59c71-istio-csr-dns-cert" pod "ff78b45c-608b-4562-99d5-d4cf35e59c71" (UID: "ff78b45c-608b-4562-99d5-d4cf35e59c71"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:13:10.920347 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:13:10.920322    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ff78b45c-608b-4562-99d5-d4cf35e59c71-cacerts" pod "ff78b45c-608b-4562-99d5-d4cf35e59c71" (UID: "ff78b45c-608b-4562-99d5-d4cf35e59c71"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:13:11.018360 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:13:11.018263    4888 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ff78b45c-608b-4562-99d5-d4cf35e59c71-istio-token\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:13:11.018360 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:13:11.018316    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qd2xz\" (UniqueName: \"kubernetes.io/projected/ff78b45c-608b-4562-99d5-d4cf35e59c71-kube-api-access-qd2xz\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:13:11.018360 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:13:11.018328    4888 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/ff78b45c-608b-4562-99d5-d4cf35e59c71-cacerts\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:13:11.018360 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:13:11.018338    4888 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/ff78b45c-608b-4562-99d5-d4cf35e59c71-istio-csr-dns-cert\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:13:11.018360 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:13:11.018347    4888 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/ff78b45c-608b-4562-99d5-d4cf35e59c71-local-certs\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:13:11.018360 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:13:11.018355    4888 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ff78b45c-608b-4562-99d5-d4cf35e59c71-istio-kubeconfig\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:13:11.403703 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:13:11.403658    4888 generic.go:356] "Generic (PLEG): container finished" podID="ff78b45c-608b-4562-99d5-d4cf35e59c71" containerID="7820fbcdf5394f25ef2fc531ceb664c18fdcf0a10a25f309ff353f2aeaa0642e" exitCode=0
Oct 30 19:13:11.403919 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:13:11.403800    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-x959p" event={"ID":"ff78b45c-608b-4562-99d5-d4cf35e59c71","Type":"ContainerDied","Data":"7820fbcdf5394f25ef2fc531ceb664c18fdcf0a10a25f309ff353f2aeaa0642e"}
Oct 30 19:13:11.403919 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:13:11.403849    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-x959p" event={"ID":"ff78b45c-608b-4562-99d5-d4cf35e59c71","Type":"ContainerDied","Data":"97f2abd085766f36b69571f4ce0c4aa9d1d147e8937d3d8963a3d32527aea750"}
Oct 30 19:13:11.403919 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:13:11.403874    4888 scope.go:122] "RemoveContainer" containerID="7820fbcdf5394f25ef2fc531ceb664c18fdcf0a10a25f309ff353f2aeaa0642e"
Oct 30 19:13:11.404113 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:13:11.404032    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-x959p"
Oct 30 19:13:11.420212 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:13:11.420138    4888 scope.go:122] "RemoveContainer" containerID="7820fbcdf5394f25ef2fc531ceb664c18fdcf0a10a25f309ff353f2aeaa0642e"
Oct 30 19:13:11.420563 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:13:11.420526    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7820fbcdf5394f25ef2fc531ceb664c18fdcf0a10a25f309ff353f2aeaa0642e\": container with ID starting with 7820fbcdf5394f25ef2fc531ceb664c18fdcf0a10a25f309ff353f2aeaa0642e not found: ID does not exist" containerID="7820fbcdf5394f25ef2fc531ceb664c18fdcf0a10a25f309ff353f2aeaa0642e"
Oct 30 19:13:11.420673 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:13:11.420568    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7820fbcdf5394f25ef2fc531ceb664c18fdcf0a10a25f309ff353f2aeaa0642e"} err="failed to get container status \"7820fbcdf5394f25ef2fc531ceb664c18fdcf0a10a25f309ff353f2aeaa0642e\": rpc error: code = NotFound desc = could not find container \"7820fbcdf5394f25ef2fc531ceb664c18fdcf0a10a25f309ff353f2aeaa0642e\": container with ID starting with 7820fbcdf5394f25ef2fc531ceb664c18fdcf0a10a25f309ff353f2aeaa0642e not found: ID does not exist"
Oct 30 19:13:11.583751 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:13:11.583710    4888 patch_prober.go:29] interesting pod/e2e-praxis-a-f554bb-openshift-default-5f8f5599d5-x2l7h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.125:15021/healthz/ready\": dial tcp 10.132.0.125:15021: connect: connection refused" start-of-body=
Oct 30 19:13:11.604345 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:13:11.584043    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-f554bb-openshift-default-5f8f5599d5-x2l7h" podUID="0c0bee10-4345-43a8-9008-b60054dc6850" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.125:15021/healthz/ready\": dial tcp 10.132.0.125:15021: connect: connection refused"
Oct 30 19:13:11.606017 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:13:11.605989    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-x959p"]
Oct 30 19:13:11.654500 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:13:11.654414    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-x959p"]
Oct 30 19:13:11.906676 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:13:11.906584    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ff78b45c-608b-4562-99d5-d4cf35e59c71" path="/var/lib/kubelet/pods/ff78b45c-608b-4562-99d5-d4cf35e59c71/volumes"
Oct 30 19:13:12.586145 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:13:12.585666    4888 patch_prober.go:29] interesting pod/e2e-praxis-a-f554bb-openshift-default-5f8f5599d5-x2l7h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.125:15021/healthz/ready\": dial tcp 10.132.0.125:15021: connect: connection refused" start-of-body=
Oct 30 19:13:12.586145 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:13:12.585745    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-f554bb-openshift-default-5f8f5599d5-x2l7h" podUID="0c0bee10-4345-43a8-9008-b60054dc6850" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.125:15021/healthz/ready\": dial tcp 10.132.0.125:15021: connect: connection refused"
Oct 30 19:13:13.586221 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:13:13.586188    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-praxis-a-f554bb-openshift-default-5f8f5599d5-x2l7h"
Oct 30 19:13:13.586660 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:13:13.586640    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-praxis-a-f554bb-openshift-default-5f8f5599d5-x2l7h"
Oct 30 19:13:13.587307 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:13:13.587267    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-praxis-a-f554bb-openshift-default-5f8f5599d5-x2l7h"
Oct 30 19:13:31.747258 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:13:31.747219    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-rjmdz"]
Oct 30 19:13:35.619222 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:13:35.619138    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-jbsw6" containerName="extractor"
Oct 30 19:13:56.599528 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:13:56.599490    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-7x4rb_382e8b31-a7fa-43f3-9f15-4c61627b0c04/console-operator/2.log"
Oct 30 19:13:56.604757 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:13:56.604733    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-7x4rb_382e8b31-a7fa-43f3-9f15-4c61627b0c04/console-operator/2.log"
Oct 30 19:13:56.609471 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:13:56.609448    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xfcdl_3f3fe764-2652-46c9-8e54-7882c0e6bcf1/ovn-acl-logging/0.log"
Oct 30 19:13:56.613592 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:13:56.613568    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xfcdl_3f3fe764-2652-46c9-8e54-7882c0e6bcf1/ovn-acl-logging/0.log"
Oct 30 19:14:05.620889 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:14:05.620851    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-jbsw6" containerName="extractor"
Oct 30 19:14:08.443999 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:14:08.443966    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-b-6d96a6-66f76bf9b7-rfhcw"]
Oct 30 19:14:08.444607 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:14:08.444591    4888 cpu_manager.go:414] "containerMap: removing container" podUID="ff78b45c-608b-4562-99d5-d4cf35e59c71" containerName="discovery"
Oct 30 19:14:08.444653 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:14:08.444609    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ff78b45c-608b-4562-99d5-d4cf35e59c71" containerName="discovery"
Oct 30 19:14:08.444709 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:14:08.444700    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="ff78b45c-608b-4562-99d5-d4cf35e59c71" containerName="discovery"
Oct 30 19:14:08.449331 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:14:08.449303    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-6d96a6-66f76bf9b7-rfhcw"
Oct 30 19:14:08.506573 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:14:08.506547    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-b-6d96a6-66f76bf9b7-rfhcw"]
Oct 30 19:14:08.520933 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:14:08.520889    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b62e784d-c5de-4e4d-ba8a-96c955b1a431-plugins-config-volume\") pod \"payload-pre-processing-e2e-praxis-b-6d96a6-66f76bf9b7-rfhcw\" (UID: \"b62e784d-c5de-4e4d-ba8a-96c955b1a431\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-6d96a6-66f76bf9b7-rfhcw"
Oct 30 19:14:08.521094 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:14:08.521067    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b28nl\" (UniqueName: \"kubernetes.io/projected/b62e784d-c5de-4e4d-ba8a-96c955b1a431-kube-api-access-b28nl\") pod \"payload-pre-processing-e2e-praxis-b-6d96a6-66f76bf9b7-rfhcw\" (UID: \"b62e784d-c5de-4e4d-ba8a-96c955b1a431\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-6d96a6-66f76bf9b7-rfhcw"
Oct 30 19:14:08.560510 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:14:08.560478    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-praxis-b-6d96a6\""
Oct 30 19:14:08.579675 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:14:08.578093    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-praxis-b-6d96a6-dockercfg-8m2wz\""
Oct 30 19:14:08.621303 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:14:08.619447    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-b-6d96a6-7447cf6d8b-zwp8g"]
Oct 30 19:14:08.626932 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:14:08.625901    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-praxis-b-6d96a6-7447cf6d8b-zwp8g"
Oct 30 19:14:08.628657 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:14:08.628631    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b28nl\" (UniqueName: \"kubernetes.io/projected/b62e784d-c5de-4e4d-ba8a-96c955b1a431-kube-api-access-b28nl\") pod \"payload-pre-processing-e2e-praxis-b-6d96a6-66f76bf9b7-rfhcw\" (UID: \"b62e784d-c5de-4e4d-ba8a-96c955b1a431\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-6d96a6-66f76bf9b7-rfhcw"
Oct 30 19:14:08.629524 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:14:08.629500    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b62e784d-c5de-4e4d-ba8a-96c955b1a431-plugins-config-volume\") pod \"payload-pre-processing-e2e-praxis-b-6d96a6-66f76bf9b7-rfhcw\" (UID: \"b62e784d-c5de-4e4d-ba8a-96c955b1a431\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-6d96a6-66f76bf9b7-rfhcw"
Oct 30 19:14:08.630751 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:14:08.630565    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b62e784d-c5de-4e4d-ba8a-96c955b1a431-plugins-config-volume\") pod \"payload-pre-processing-e2e-praxis-b-6d96a6-66f76bf9b7-rfhcw\" (UID: \"b62e784d-c5de-4e4d-ba8a-96c955b1a431\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-6d96a6-66f76bf9b7-rfhcw"
Oct 30 19:14:08.670498 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:14:08.670468    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-b-6d96a6-7447cf6d8b-zwp8g"]
Oct 30 19:14:08.689980 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:14:08.689955    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b28nl\" (UniqueName: \"kubernetes.io/projected/b62e784d-c5de-4e4d-ba8a-96c955b1a431-kube-api-access-b28nl\") pod \"payload-pre-processing-e2e-praxis-b-6d96a6-66f76bf9b7-rfhcw\" (UID: \"b62e784d-c5de-4e4d-ba8a-96c955b1a431\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-6d96a6-66f76bf9b7-rfhcw"
Oct 30 19:14:08.732197 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:14:08.732121    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/30332320-0fa8-4c94-ba9f-42cbacfdd0d9-plugins-config-volume\") pod \"payload-processing-e2e-praxis-b-6d96a6-7447cf6d8b-zwp8g\" (UID: \"30332320-0fa8-4c94-ba9f-42cbacfdd0d9\") " pod="openshift-ingress/payload-processing-e2e-praxis-b-6d96a6-7447cf6d8b-zwp8g"
Oct 30 19:14:08.732356 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:14:08.732195    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lrtp7\" (UniqueName: \"kubernetes.io/projected/30332320-0fa8-4c94-ba9f-42cbacfdd0d9-kube-api-access-lrtp7\") pod \"payload-processing-e2e-praxis-b-6d96a6-7447cf6d8b-zwp8g\" (UID: \"30332320-0fa8-4c94-ba9f-42cbacfdd0d9\") " pod="openshift-ingress/payload-processing-e2e-praxis-b-6d96a6-7447cf6d8b-zwp8g"
Oct 30 19:14:08.762877 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:14:08.762835    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-6d96a6-66f76bf9b7-rfhcw"
Oct 30 19:14:08.835946 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:14:08.833448    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/30332320-0fa8-4c94-ba9f-42cbacfdd0d9-plugins-config-volume\") pod \"payload-processing-e2e-praxis-b-6d96a6-7447cf6d8b-zwp8g\" (UID: \"30332320-0fa8-4c94-ba9f-42cbacfdd0d9\") " pod="openshift-ingress/payload-processing-e2e-praxis-b-6d96a6-7447cf6d8b-zwp8g"
Oct 30 19:14:08.835946 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:14:08.833522    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lrtp7\" (UniqueName: \"kubernetes.io/projected/30332320-0fa8-4c94-ba9f-42cbacfdd0d9-kube-api-access-lrtp7\") pod \"payload-processing-e2e-praxis-b-6d96a6-7447cf6d8b-zwp8g\" (UID: \"30332320-0fa8-4c94-ba9f-42cbacfdd0d9\") " pod="openshift-ingress/payload-processing-e2e-praxis-b-6d96a6-7447cf6d8b-zwp8g"
Oct 30 19:14:08.835946 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:14:08.834937    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/30332320-0fa8-4c94-ba9f-42cbacfdd0d9-plugins-config-volume\") pod \"payload-processing-e2e-praxis-b-6d96a6-7447cf6d8b-zwp8g\" (UID: \"30332320-0fa8-4c94-ba9f-42cbacfdd0d9\") " pod="openshift-ingress/payload-processing-e2e-praxis-b-6d96a6-7447cf6d8b-zwp8g"
Oct 30 19:14:08.898896 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:14:08.898832    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lrtp7\" (UniqueName: \"kubernetes.io/projected/30332320-0fa8-4c94-ba9f-42cbacfdd0d9-kube-api-access-lrtp7\") pod \"payload-processing-e2e-praxis-b-6d96a6-7447cf6d8b-zwp8g\" (UID: \"30332320-0fa8-4c94-ba9f-42cbacfdd0d9\") " pod="openshift-ingress/payload-processing-e2e-praxis-b-6d96a6-7447cf6d8b-zwp8g"
Oct 30 19:14:08.947268 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:14:08.947239    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-praxis-b-6d96a6-7447cf6d8b-zwp8g"
Oct 30 19:14:09.160588 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:14:09.160557    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-b-6d96a6-66f76bf9b7-rfhcw"]
Oct 30 19:14:09.162879 ip-10-0-134-46 kubenswrapper[4888]: W1030 19:14:09.162850    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb62e784d_c5de_4e4d_ba8a_96c955b1a431.slice/crio-a633cfd3a4b5fd453e62d9ae1a6d772725ed6134bf4ea5ed290d7e21523bf635 WatchSource:0}: Error finding container a633cfd3a4b5fd453e62d9ae1a6d772725ed6134bf4ea5ed290d7e21523bf635: Status 404 returned error can't find the container with id a633cfd3a4b5fd453e62d9ae1a6d772725ed6134bf4ea5ed290d7e21523bf635
Oct 30 19:14:09.339565 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:14:09.339534    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-b-6d96a6-7447cf6d8b-zwp8g"]
Oct 30 19:14:09.342219 ip-10-0-134-46 kubenswrapper[4888]: W1030 19:14:09.342194    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30332320_0fa8_4c94_ba9f_42cbacfdd0d9.slice/crio-120ee6d532be653ad7bef6a7954460cf0fa597cd71a43bbc56a7628cded17b5c WatchSource:0}: Error finding container 120ee6d532be653ad7bef6a7954460cf0fa597cd71a43bbc56a7628cded17b5c: Status 404 returned error can't find the container with id 120ee6d532be653ad7bef6a7954460cf0fa597cd71a43bbc56a7628cded17b5c
Oct 30 19:14:09.846541 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:14:09.846508    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-6d96a6-66f76bf9b7-rfhcw" event={"ID":"b62e784d-c5de-4e4d-ba8a-96c955b1a431","Type":"ContainerStarted","Data":"c78c980ef3720093be418d943a91869a25ab1e9d26bade5492dc8afcde18d958"}
Oct 30 19:14:09.847036 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:14:09.846564    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-6d96a6-66f76bf9b7-rfhcw" event={"ID":"b62e784d-c5de-4e4d-ba8a-96c955b1a431","Type":"ContainerStarted","Data":"a633cfd3a4b5fd453e62d9ae1a6d772725ed6134bf4ea5ed290d7e21523bf635"}
Oct 30 19:14:09.847036 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:14:09.846716    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-6d96a6-66f76bf9b7-rfhcw"
Oct 30 19:14:09.848826 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:14:09.848800    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-praxis-b-6d96a6-7447cf6d8b-zwp8g" event={"ID":"30332320-0fa8-4c94-ba9f-42cbacfdd0d9","Type":"ContainerStarted","Data":"5e689a83fe59077292f3b6f1fa1dede7908656ecf6af62c3ec4c5086aaf14b85"}
Oct 30 19:14:09.848959 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:14:09.848841    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-praxis-b-6d96a6-7447cf6d8b-zwp8g" event={"ID":"30332320-0fa8-4c94-ba9f-42cbacfdd0d9","Type":"ContainerStarted","Data":"120ee6d532be653ad7bef6a7954460cf0fa597cd71a43bbc56a7628cded17b5c"}
Oct 30 19:14:09.848959 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:14:09.848862    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-praxis-b-6d96a6-7447cf6d8b-zwp8g"
Oct 30 19:14:09.975794 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:14:09.975743    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-6d96a6-66f76bf9b7-rfhcw" podStartSLOduration=1.975729518 podStartE2EDuration="1.975729518s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:14:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:14:09.971887485 +0000 UTC m=+2414.219293513" watchObservedRunningTime="2026-10-30 19:14:09.975729518 +0000 UTC m=+2414.223135544"
Oct 30 19:14:10.111776 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:14:10.111590    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-praxis-b-6d96a6-7447cf6d8b-zwp8g" podStartSLOduration=2.111567311 podStartE2EDuration="2.111567311s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:14:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:14:10.101803709 +0000 UTC m=+2414.349209735" watchObservedRunningTime="2026-10-30 19:14:10.111567311 +0000 UTC m=+2414.358973337"
Oct 30 19:14:19.451602 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:14:19.451573    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-5a035d-865f889459-xx7rn"]
Oct 30 19:14:19.452029 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:14:19.451779    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-5a035d-865f889459-xx7rn" podUID="b141f1fa-213b-4816-b5c4-420693daba2f" containerName="maas-api" containerID="cri-o://e54895f2dbbba86baecdd6192bd9f0d429fa7cb6acf79e77fb95b940f31cfe06" gracePeriod=30
Oct 30 19:14:19.766880 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:14:19.766858    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-5a035d-865f889459-xx7rn"
Oct 30 19:14:19.874776 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:14:19.874738    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b141f1fa-213b-4816-b5c4-420693daba2f-metrics-certs\" (UniqueName: \"kubernetes.io/secret/b141f1fa-213b-4816-b5c4-420693daba2f-metrics-certs\") pod \"b141f1fa-213b-4816-b5c4-420693daba2f\" (UID: \"b141f1fa-213b-4816-b5c4-420693daba2f\") "
Oct 30 19:14:19.875020 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:14:19.874816    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b141f1fa-213b-4816-b5c4-420693daba2f-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b141f1fa-213b-4816-b5c4-420693daba2f-maas-api-tls\") pod \"b141f1fa-213b-4816-b5c4-420693daba2f\" (UID: \"b141f1fa-213b-4816-b5c4-420693daba2f\") "
Oct 30 19:14:19.875524 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:14:19.875211    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b141f1fa-213b-4816-b5c4-420693daba2f-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b141f1fa-213b-4816-b5c4-420693daba2f-openshift-service-ca\") pod \"b141f1fa-213b-4816-b5c4-420693daba2f\" (UID: \"b141f1fa-213b-4816-b5c4-420693daba2f\") "
Oct 30 19:14:19.875524 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:14:19.875260    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b141f1fa-213b-4816-b5c4-420693daba2f-kube-api-access-vrkfz\" (UniqueName: \"kubernetes.io/projected/b141f1fa-213b-4816-b5c4-420693daba2f-kube-api-access-vrkfz\") pod \"b141f1fa-213b-4816-b5c4-420693daba2f\" (UID: \"b141f1fa-213b-4816-b5c4-420693daba2f\") "
Oct 30 19:14:19.876947 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:14:19.876871    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b141f1fa-213b-4816-b5c4-420693daba2f-openshift-service-ca" pod "b141f1fa-213b-4816-b5c4-420693daba2f" (UID: "b141f1fa-213b-4816-b5c4-420693daba2f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:14:19.914033 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:14:19.901003    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b141f1fa-213b-4816-b5c4-420693daba2f-kube-api-access-vrkfz" pod "b141f1fa-213b-4816-b5c4-420693daba2f" (UID: "b141f1fa-213b-4816-b5c4-420693daba2f"). InnerVolumeSpecName "kube-api-access-vrkfz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:14:19.914033 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:14:19.912426    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b141f1fa-213b-4816-b5c4-420693daba2f-maas-api-tls" pod "b141f1fa-213b-4816-b5c4-420693daba2f" (UID: "b141f1fa-213b-4816-b5c4-420693daba2f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:14:19.914033 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:14:19.913825    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b141f1fa-213b-4816-b5c4-420693daba2f-metrics-certs" pod "b141f1fa-213b-4816-b5c4-420693daba2f" (UID: "b141f1fa-213b-4816-b5c4-420693daba2f"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:14:19.925425 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:14:19.925391    4888 generic.go:356] "Generic (PLEG): container finished" podID="b141f1fa-213b-4816-b5c4-420693daba2f" containerID="e54895f2dbbba86baecdd6192bd9f0d429fa7cb6acf79e77fb95b940f31cfe06" exitCode=0
Oct 30 19:14:19.925599 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:14:19.925580    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-5a035d-865f889459-xx7rn"
Oct 30 19:14:19.934234 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:14:19.934121    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-5a035d-865f889459-xx7rn" event={"ID":"b141f1fa-213b-4816-b5c4-420693daba2f","Type":"ContainerDied","Data":"e54895f2dbbba86baecdd6192bd9f0d429fa7cb6acf79e77fb95b940f31cfe06"}
Oct 30 19:14:19.934234 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:14:19.934183    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-5a035d-865f889459-xx7rn" event={"ID":"b141f1fa-213b-4816-b5c4-420693daba2f","Type":"ContainerDied","Data":"73790fe8809d496735c1919190e4dec2c7c669b8911e15a143083df4099e6894"}
Oct 30 19:14:19.934234 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:14:19.934212    4888 scope.go:122] "RemoveContainer" containerID="e54895f2dbbba86baecdd6192bd9f0d429fa7cb6acf79e77fb95b940f31cfe06"
Oct 30 19:14:19.948100 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:14:19.948076    4888 scope.go:122] "RemoveContainer" containerID="e54895f2dbbba86baecdd6192bd9f0d429fa7cb6acf79e77fb95b940f31cfe06"
Oct 30 19:14:19.948401 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:14:19.948381    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e54895f2dbbba86baecdd6192bd9f0d429fa7cb6acf79e77fb95b940f31cfe06\": container with ID starting with e54895f2dbbba86baecdd6192bd9f0d429fa7cb6acf79e77fb95b940f31cfe06 not found: ID does not exist" containerID="e54895f2dbbba86baecdd6192bd9f0d429fa7cb6acf79e77fb95b940f31cfe06"
Oct 30 19:14:19.948486 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:14:19.948411    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e54895f2dbbba86baecdd6192bd9f0d429fa7cb6acf79e77fb95b940f31cfe06"} err="failed to get container status \"e54895f2dbbba86baecdd6192bd9f0d429fa7cb6acf79e77fb95b940f31cfe06\": rpc error: code = NotFound desc = could not find container \"e54895f2dbbba86baecdd6192bd9f0d429fa7cb6acf79e77fb95b940f31cfe06\": container with ID starting with e54895f2dbbba86baecdd6192bd9f0d429fa7cb6acf79e77fb95b940f31cfe06 not found: ID does not exist"
Oct 30 19:14:19.977586 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:14:19.977561    4888 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b141f1fa-213b-4816-b5c4-420693daba2f-metrics-certs\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:14:19.977720 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:14:19.977603    4888 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b141f1fa-213b-4816-b5c4-420693daba2f-maas-api-tls\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:14:19.977720 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:14:19.977622    4888 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b141f1fa-213b-4816-b5c4-420693daba2f-openshift-service-ca\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:14:19.977720 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:14:19.977635    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vrkfz\" (UniqueName: \"kubernetes.io/projected/b141f1fa-213b-4816-b5c4-420693daba2f-kube-api-access-vrkfz\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:14:20.103238 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:14:20.103206    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-5a035d-865f889459-xx7rn"]
Oct 30 19:14:20.106395 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:14:20.106358    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-5a035d-865f889459-xx7rn"]
Oct 30 19:14:20.859676 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:14:20.859647    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-praxis-b-6d96a6-7447cf6d8b-zwp8g"
Oct 30 19:14:20.860121 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:14:20.859754    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-6d96a6-66f76bf9b7-rfhcw"
Oct 30 19:14:21.891308 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:14:21.891247    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b141f1fa-213b-4816-b5c4-420693daba2f" path="/var/lib/kubelet/pods/b141f1fa-213b-4816-b5c4-420693daba2f/volumes"
Oct 30 19:14:35.617801 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:14:35.617769    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-jbsw6" containerName="extractor"
Oct 30 19:14:49.800891 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:14:49.800858    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-rjmdz"]
Oct 30 19:15:00.190849 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:00.190813    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-2ade09-29889795-9tk7q"]
Oct 30 19:15:00.191729 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:00.191700    4888 cpu_manager.go:414] "containerMap: removing container" podUID="b141f1fa-213b-4816-b5c4-420693daba2f" containerName="maas-api"
Oct 30 19:15:00.191729 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:00.191729    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b141f1fa-213b-4816-b5c4-420693daba2f" containerName="maas-api"
Oct 30 19:15:00.191944 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:00.191926    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="b141f1fa-213b-4816-b5c4-420693daba2f" containerName="maas-api"
Oct 30 19:15:00.197081 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:00.197059    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-2ade09-29889795-9tk7q"
Oct 30 19:15:00.214537 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:00.214506    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889795-j9vgs"]
Oct 30 19:15:00.219848 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:00.219826    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-52d383-29889795-dxc57"]
Oct 30 19:15:00.219999 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:00.219968    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889795-j9vgs"
Oct 30 19:15:00.224183 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:00.224162    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-52d383-29889795-dxc57"
Oct 30 19:15:00.294857 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:00.294815    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-986xn\" (UniqueName: \"kubernetes.io/projected/7208215b-516e-47d6-bb61-27e1031c7953-kube-api-access-986xn\") pod \"maas-api-key-cleanup-e2e-ipp-a-52d383-29889795-dxc57\" (UID: \"7208215b-516e-47d6-bb61-27e1031c7953\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-52d383-29889795-dxc57"
Oct 30 19:15:00.295052 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:00.295018    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7208215b-516e-47d6-bb61-27e1031c7953-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-ipp-a-52d383-29889795-dxc57\" (UID: \"7208215b-516e-47d6-bb61-27e1031c7953\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-52d383-29889795-dxc57"
Oct 30 19:15:00.295136 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:00.295081    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9fe0e09a-5f21-4382-8153-542047006bb6-maas-api-service-ca\") pod \"maas-api-key-cleanup-29889795-j9vgs\" (UID: \"9fe0e09a-5f21-4382-8153-542047006bb6\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889795-j9vgs"
Oct 30 19:15:00.295232 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:00.295208    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8xpzz\" (UniqueName: \"kubernetes.io/projected/bbc9f851-b372-4f93-a7a8-42f67d93093c-kube-api-access-8xpzz\") pod \"maas-api-key-cleanup-e2e-worker-w0-2ade09-29889795-9tk7q\" (UID: \"bbc9f851-b372-4f93-a7a8-42f67d93093c\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-2ade09-29889795-9tk7q"
Oct 30 19:15:00.295341 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:00.295324    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6tlfn\" (UniqueName: \"kubernetes.io/projected/9fe0e09a-5f21-4382-8153-542047006bb6-kube-api-access-6tlfn\") pod \"maas-api-key-cleanup-29889795-j9vgs\" (UID: \"9fe0e09a-5f21-4382-8153-542047006bb6\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889795-j9vgs"
Oct 30 19:15:00.295424 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:00.295374    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bbc9f851-b372-4f93-a7a8-42f67d93093c-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w0-2ade09-29889795-9tk7q\" (UID: \"bbc9f851-b372-4f93-a7a8-42f67d93093c\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-2ade09-29889795-9tk7q"
Oct 30 19:15:00.315790 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:00.315760    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-2ade09-29889795-9tk7q"]
Oct 30 19:15:00.322938 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:00.322900    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-2qg7n\""
Oct 30 19:15:00.364846 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:00.364815    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889795-j9vgs"]
Oct 30 19:15:00.371922 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:00.371891    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-52d383-29889795-dxc57"]
Oct 30 19:15:00.396571 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:00.396545    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7208215b-516e-47d6-bb61-27e1031c7953-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-ipp-a-52d383-29889795-dxc57\" (UID: \"7208215b-516e-47d6-bb61-27e1031c7953\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-52d383-29889795-dxc57"
Oct 30 19:15:00.396729 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:00.396584    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9fe0e09a-5f21-4382-8153-542047006bb6-maas-api-service-ca\") pod \"maas-api-key-cleanup-29889795-j9vgs\" (UID: \"9fe0e09a-5f21-4382-8153-542047006bb6\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889795-j9vgs"
Oct 30 19:15:00.396729 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:00.396629    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8xpzz\" (UniqueName: \"kubernetes.io/projected/bbc9f851-b372-4f93-a7a8-42f67d93093c-kube-api-access-8xpzz\") pod \"maas-api-key-cleanup-e2e-worker-w0-2ade09-29889795-9tk7q\" (UID: \"bbc9f851-b372-4f93-a7a8-42f67d93093c\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-2ade09-29889795-9tk7q"
Oct 30 19:15:00.396729 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:00.396678    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6tlfn\" (UniqueName: \"kubernetes.io/projected/9fe0e09a-5f21-4382-8153-542047006bb6-kube-api-access-6tlfn\") pod \"maas-api-key-cleanup-29889795-j9vgs\" (UID: \"9fe0e09a-5f21-4382-8153-542047006bb6\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889795-j9vgs"
Oct 30 19:15:00.396729 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:00.396714    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bbc9f851-b372-4f93-a7a8-42f67d93093c-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w0-2ade09-29889795-9tk7q\" (UID: \"bbc9f851-b372-4f93-a7a8-42f67d93093c\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-2ade09-29889795-9tk7q"
Oct 30 19:15:00.396991 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:00.396747    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-986xn\" (UniqueName: \"kubernetes.io/projected/7208215b-516e-47d6-bb61-27e1031c7953-kube-api-access-986xn\") pod \"maas-api-key-cleanup-e2e-ipp-a-52d383-29889795-dxc57\" (UID: \"7208215b-516e-47d6-bb61-27e1031c7953\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-52d383-29889795-dxc57"
Oct 30 19:15:00.397449 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:00.397425    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7208215b-516e-47d6-bb61-27e1031c7953-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-ipp-a-52d383-29889795-dxc57\" (UID: \"7208215b-516e-47d6-bb61-27e1031c7953\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-52d383-29889795-dxc57"
Oct 30 19:15:00.397648 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:00.397630    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bbc9f851-b372-4f93-a7a8-42f67d93093c-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w0-2ade09-29889795-9tk7q\" (UID: \"bbc9f851-b372-4f93-a7a8-42f67d93093c\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-2ade09-29889795-9tk7q"
Oct 30 19:15:00.397994 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:00.397970    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9fe0e09a-5f21-4382-8153-542047006bb6-maas-api-service-ca\") pod \"maas-api-key-cleanup-29889795-j9vgs\" (UID: \"9fe0e09a-5f21-4382-8153-542047006bb6\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889795-j9vgs"
Oct 30 19:15:00.459437 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:00.459370    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8xpzz\" (UniqueName: \"kubernetes.io/projected/bbc9f851-b372-4f93-a7a8-42f67d93093c-kube-api-access-8xpzz\") pod \"maas-api-key-cleanup-e2e-worker-w0-2ade09-29889795-9tk7q\" (UID: \"bbc9f851-b372-4f93-a7a8-42f67d93093c\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-2ade09-29889795-9tk7q"
Oct 30 19:15:00.459437 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:00.459426    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6tlfn\" (UniqueName: \"kubernetes.io/projected/9fe0e09a-5f21-4382-8153-542047006bb6-kube-api-access-6tlfn\") pod \"maas-api-key-cleanup-29889795-j9vgs\" (UID: \"9fe0e09a-5f21-4382-8153-542047006bb6\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889795-j9vgs"
Oct 30 19:15:00.459625 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:00.459425    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-986xn\" (UniqueName: \"kubernetes.io/projected/7208215b-516e-47d6-bb61-27e1031c7953-kube-api-access-986xn\") pod \"maas-api-key-cleanup-e2e-ipp-a-52d383-29889795-dxc57\" (UID: \"7208215b-516e-47d6-bb61-27e1031c7953\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-52d383-29889795-dxc57"
Oct 30 19:15:00.520157 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:00.520127    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-2ade09-29889795-9tk7q"
Oct 30 19:15:00.537068 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:00.537041    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889795-j9vgs"
Oct 30 19:15:00.545210 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:00.545187    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-52d383-29889795-dxc57"
Oct 30 19:15:00.960433 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:00.960403    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-2ade09-29889795-9tk7q"]
Oct 30 19:15:00.960929 ip-10-0-134-46 kubenswrapper[4888]: W1030 19:15:00.960902    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbbc9f851_b372_4f93_a7a8_42f67d93093c.slice/crio-d8dfd39e9f6a43750a572b6d34cd5de535a204f3fdfba96746f04ce80a7b5ad6 WatchSource:0}: Error finding container d8dfd39e9f6a43750a572b6d34cd5de535a204f3fdfba96746f04ce80a7b5ad6: Status 404 returned error can't find the container with id d8dfd39e9f6a43750a572b6d34cd5de535a204f3fdfba96746f04ce80a7b5ad6
Oct 30 19:15:00.987045 ip-10-0-134-46 kubenswrapper[4888]: W1030 19:15:00.987011    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9fe0e09a_5f21_4382_8153_542047006bb6.slice/crio-7da4f88103faeed1e7a51daf7370782596e7bfddbdd3ddcc257f3c18a288c532 WatchSource:0}: Error finding container 7da4f88103faeed1e7a51daf7370782596e7bfddbdd3ddcc257f3c18a288c532: Status 404 returned error can't find the container with id 7da4f88103faeed1e7a51daf7370782596e7bfddbdd3ddcc257f3c18a288c532
Oct 30 19:15:00.989166 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:00.988065    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889795-j9vgs"]
Oct 30 19:15:00.993258 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:00.993231    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-52d383-29889795-dxc57"]
Oct 30 19:15:00.999131 ip-10-0-134-46 kubenswrapper[4888]: W1030 19:15:00.997897    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7208215b_516e_47d6_bb61_27e1031c7953.slice/crio-340570533ca4cd0fec749c9f48646c7cc168b783c42092d8b8409d6ebeea06c7 WatchSource:0}: Error finding container 340570533ca4cd0fec749c9f48646c7cc168b783c42092d8b8409d6ebeea06c7: Status 404 returned error can't find the container with id 340570533ca4cd0fec749c9f48646c7cc168b783c42092d8b8409d6ebeea06c7
Oct 30 19:15:01.222555 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:01.222453    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889795-j9vgs" event={"ID":"9fe0e09a-5f21-4382-8153-542047006bb6","Type":"ContainerStarted","Data":"7da4f88103faeed1e7a51daf7370782596e7bfddbdd3ddcc257f3c18a288c532"}
Oct 30 19:15:01.224266 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:01.224237    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-52d383-29889795-dxc57" event={"ID":"7208215b-516e-47d6-bb61-27e1031c7953","Type":"ContainerStarted","Data":"340570533ca4cd0fec749c9f48646c7cc168b783c42092d8b8409d6ebeea06c7"}
Oct 30 19:15:01.225540 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:01.225520    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-2ade09-29889795-9tk7q" event={"ID":"bbc9f851-b372-4f93-a7a8-42f67d93093c","Type":"ContainerStarted","Data":"d8dfd39e9f6a43750a572b6d34cd5de535a204f3fdfba96746f04ce80a7b5ad6"}
Oct 30 19:15:02.235829 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:02.235742    4888 generic.go:356] "Generic (PLEG): container finished" podID="bbc9f851-b372-4f93-a7a8-42f67d93093c" containerID="17a28d6869888af6feaf82518f7f312277202a6d7186a8365fcbd5e7a230bcad" exitCode=60
Oct 30 19:15:02.236263 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:02.235818    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-2ade09-29889795-9tk7q" event={"ID":"bbc9f851-b372-4f93-a7a8-42f67d93093c","Type":"ContainerDied","Data":"17a28d6869888af6feaf82518f7f312277202a6d7186a8365fcbd5e7a230bcad"}
Oct 30 19:15:02.236263 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:02.236093    4888 scope.go:122] "RemoveContainer" containerID="17a28d6869888af6feaf82518f7f312277202a6d7186a8365fcbd5e7a230bcad"
Oct 30 19:15:02.237818 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:02.237714    4888 generic.go:356] "Generic (PLEG): container finished" podID="9fe0e09a-5f21-4382-8153-542047006bb6" containerID="ed189946d0b0899b1f44c052e5aca398dda52d8404a5d4f066dfa34a5571a2a1" exitCode=60
Oct 30 19:15:02.237818 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:02.237793    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889795-j9vgs" event={"ID":"9fe0e09a-5f21-4382-8153-542047006bb6","Type":"ContainerDied","Data":"ed189946d0b0899b1f44c052e5aca398dda52d8404a5d4f066dfa34a5571a2a1"}
Oct 30 19:15:02.238001 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:02.237985    4888 scope.go:122] "RemoveContainer" containerID="ed189946d0b0899b1f44c052e5aca398dda52d8404a5d4f066dfa34a5571a2a1"
Oct 30 19:15:02.239746 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:02.239726    4888 generic.go:356] "Generic (PLEG): container finished" podID="7208215b-516e-47d6-bb61-27e1031c7953" containerID="a836e96ddc3abf34faff13e0db5e29a7952e8957202274016d875b9e536ca3a1" exitCode=60
Oct 30 19:15:02.239852 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:02.239763    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-52d383-29889795-dxc57" event={"ID":"7208215b-516e-47d6-bb61-27e1031c7953","Type":"ContainerDied","Data":"a836e96ddc3abf34faff13e0db5e29a7952e8957202274016d875b9e536ca3a1"}
Oct 30 19:15:02.239988 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:02.239974    4888 scope.go:122] "RemoveContainer" containerID="a836e96ddc3abf34faff13e0db5e29a7952e8957202274016d875b9e536ca3a1"
Oct 30 19:15:02.911752 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:02.911723    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-52d383-5b7f46bf4c-m9fjk"]
Oct 30 19:15:02.915991 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:02.915969    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-52d383-5b7f46bf4c-m9fjk"
Oct 30 19:15:02.994826 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:02.994789    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-52d383-5b7f46bf4c-m9fjk"]
Oct 30 19:15:03.012810 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:03.012766    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-52d383\""
Oct 30 19:15:03.015196 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:03.015173    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-52d383-dockercfg-hs478\""
Oct 30 19:15:03.031994 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:03.031964    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t9zwc\" (UniqueName: \"kubernetes.io/projected/ade6242e-b404-446e-8474-6e64b818a3f1-kube-api-access-t9zwc\") pod \"payload-pre-processing-e2e-ipp-a-52d383-5b7f46bf4c-m9fjk\" (UID: \"ade6242e-b404-446e-8474-6e64b818a3f1\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-52d383-5b7f46bf4c-m9fjk"
Oct 30 19:15:03.032217 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:03.032192    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ade6242e-b404-446e-8474-6e64b818a3f1-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-52d383-5b7f46bf4c-m9fjk\" (UID: \"ade6242e-b404-446e-8474-6e64b818a3f1\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-52d383-5b7f46bf4c-m9fjk"
Oct 30 19:15:03.134196 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:03.134144    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ade6242e-b404-446e-8474-6e64b818a3f1-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-52d383-5b7f46bf4c-m9fjk\" (UID: \"ade6242e-b404-446e-8474-6e64b818a3f1\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-52d383-5b7f46bf4c-m9fjk"
Oct 30 19:15:03.134431 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:03.134324    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t9zwc\" (UniqueName: \"kubernetes.io/projected/ade6242e-b404-446e-8474-6e64b818a3f1-kube-api-access-t9zwc\") pod \"payload-pre-processing-e2e-ipp-a-52d383-5b7f46bf4c-m9fjk\" (UID: \"ade6242e-b404-446e-8474-6e64b818a3f1\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-52d383-5b7f46bf4c-m9fjk"
Oct 30 19:15:03.135134 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:03.135107    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ade6242e-b404-446e-8474-6e64b818a3f1-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-52d383-5b7f46bf4c-m9fjk\" (UID: \"ade6242e-b404-446e-8474-6e64b818a3f1\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-52d383-5b7f46bf4c-m9fjk"
Oct 30 19:15:03.197490 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:03.197450    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t9zwc\" (UniqueName: \"kubernetes.io/projected/ade6242e-b404-446e-8474-6e64b818a3f1-kube-api-access-t9zwc\") pod \"payload-pre-processing-e2e-ipp-a-52d383-5b7f46bf4c-m9fjk\" (UID: \"ade6242e-b404-446e-8474-6e64b818a3f1\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-52d383-5b7f46bf4c-m9fjk"
Oct 30 19:15:03.231947 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:03.231912    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-52d383-5b7f46bf4c-m9fjk"
Oct 30 19:15:03.266823 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:03.266544    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-52d383-29889795-dxc57" event={"ID":"7208215b-516e-47d6-bb61-27e1031c7953","Type":"ContainerStarted","Data":"703da68fc83c676cc0e331e4ee42f5f3a6e71650da8b784858ce41305b11543b"}
Oct 30 19:15:03.274656 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:03.274623    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-2ade09-29889795-9tk7q" event={"ID":"bbc9f851-b372-4f93-a7a8-42f67d93093c","Type":"ContainerStarted","Data":"c97a46e96bf2271f333f1ff6e5590d71b9ee7f1c3fa67f9c098194ef7e37f56f"}
Oct 30 19:15:03.277963 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:03.277934    4888 generic.go:356] "Generic (PLEG): container finished" podID="9fe0e09a-5f21-4382-8153-542047006bb6" containerID="9cc8b34961cb8cb607b87f99d9bf233a1431b723ec66a1d88f125a4359df2ad6" exitCode=60
Oct 30 19:15:03.278107 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:03.278023    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889795-j9vgs" event={"ID":"9fe0e09a-5f21-4382-8153-542047006bb6","Type":"ContainerDied","Data":"9cc8b34961cb8cb607b87f99d9bf233a1431b723ec66a1d88f125a4359df2ad6"}
Oct 30 19:15:03.278107 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:03.278102    4888 scope.go:122] "RemoveContainer" containerID="ed189946d0b0899b1f44c052e5aca398dda52d8404a5d4f066dfa34a5571a2a1"
Oct 30 19:15:03.279620 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:03.279601    4888 scope.go:122] "RemoveContainer" containerID="9cc8b34961cb8cb607b87f99d9bf233a1431b723ec66a1d88f125a4359df2ad6"
Oct 30 19:15:03.279899 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:15:03.279879    4888 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-29889795-j9vgs_odh-ai-gateway-infra(9fe0e09a-5f21-4382-8153-542047006bb6)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889795-j9vgs" podUID="9fe0e09a-5f21-4382-8153-542047006bb6"
Oct 30 19:15:03.392240 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:03.392177    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-52d383-29889795-dxc57" podStartSLOduration=3.392158817 podStartE2EDuration="3.392158817s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:15:03.39025335 +0000 UTC m=+2467.637659375" watchObservedRunningTime="2026-10-30 19:15:03.392158817 +0000 UTC m=+2467.639564843"
Oct 30 19:15:03.630997 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:03.630942    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-2ade09-29889795-9tk7q" podStartSLOduration=3.630929383 podStartE2EDuration="3.630929383s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:15:03.628613227 +0000 UTC m=+2467.876019253" watchObservedRunningTime="2026-10-30 19:15:03.630929383 +0000 UTC m=+2467.878335407"
Oct 30 19:15:03.657635 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:03.657607    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-52d383-5b7f46bf4c-m9fjk"]
Oct 30 19:15:03.659581 ip-10-0-134-46 kubenswrapper[4888]: W1030 19:15:03.659540    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podade6242e_b404_446e_8474_6e64b818a3f1.slice/crio-751fc4c52572da253a5418662e10993672f0ddf83c47d6f7b1d8b3518cbef646 WatchSource:0}: Error finding container 751fc4c52572da253a5418662e10993672f0ddf83c47d6f7b1d8b3518cbef646: Status 404 returned error can't find the container with id 751fc4c52572da253a5418662e10993672f0ddf83c47d6f7b1d8b3518cbef646
Oct 30 19:15:04.290568 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:04.290539    4888 scope.go:122] "RemoveContainer" containerID="9cc8b34961cb8cb607b87f99d9bf233a1431b723ec66a1d88f125a4359df2ad6"
Oct 30 19:15:04.291052 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:15:04.290864    4888 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-29889795-j9vgs_odh-ai-gateway-infra(9fe0e09a-5f21-4382-8153-542047006bb6)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889795-j9vgs" podUID="9fe0e09a-5f21-4382-8153-542047006bb6"
Oct 30 19:15:04.292322 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:04.292266    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-52d383-5b7f46bf4c-m9fjk" event={"ID":"ade6242e-b404-446e-8474-6e64b818a3f1","Type":"ContainerStarted","Data":"751fc4c52572da253a5418662e10993672f0ddf83c47d6f7b1d8b3518cbef646"}
Oct 30 19:15:04.294616 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:04.294594    4888 generic.go:356] "Generic (PLEG): container finished" podID="7208215b-516e-47d6-bb61-27e1031c7953" containerID="703da68fc83c676cc0e331e4ee42f5f3a6e71650da8b784858ce41305b11543b" exitCode=60
Oct 30 19:15:04.294735 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:04.294643    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-52d383-29889795-dxc57" event={"ID":"7208215b-516e-47d6-bb61-27e1031c7953","Type":"ContainerDied","Data":"703da68fc83c676cc0e331e4ee42f5f3a6e71650da8b784858ce41305b11543b"}
Oct 30 19:15:04.294735 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:04.294704    4888 scope.go:122] "RemoveContainer" containerID="a836e96ddc3abf34faff13e0db5e29a7952e8957202274016d875b9e536ca3a1"
Oct 30 19:15:04.294986 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:04.294968    4888 scope.go:122] "RemoveContainer" containerID="703da68fc83c676cc0e331e4ee42f5f3a6e71650da8b784858ce41305b11543b"
Oct 30 19:15:04.295323 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:15:04.295299    4888 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-ipp-a-52d383-29889795-dxc57_odh-ai-gateway-infra(7208215b-516e-47d6-bb61-27e1031c7953)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-52d383-29889795-dxc57" podUID="7208215b-516e-47d6-bb61-27e1031c7953"
Oct 30 19:15:04.296893 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:04.296876    4888 generic.go:356] "Generic (PLEG): container finished" podID="bbc9f851-b372-4f93-a7a8-42f67d93093c" containerID="c97a46e96bf2271f333f1ff6e5590d71b9ee7f1c3fa67f9c098194ef7e37f56f" exitCode=60
Oct 30 19:15:04.297005 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:04.296947    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-2ade09-29889795-9tk7q" event={"ID":"bbc9f851-b372-4f93-a7a8-42f67d93093c","Type":"ContainerDied","Data":"c97a46e96bf2271f333f1ff6e5590d71b9ee7f1c3fa67f9c098194ef7e37f56f"}
Oct 30 19:15:04.297237 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:04.297223    4888 scope.go:122] "RemoveContainer" containerID="c97a46e96bf2271f333f1ff6e5590d71b9ee7f1c3fa67f9c098194ef7e37f56f"
Oct 30 19:15:04.297516 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:15:04.297488    4888 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w0-2ade09-29889795-9tk7q_odh-ai-gateway-infra(bbc9f851-b372-4f93-a7a8-42f67d93093c)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-2ade09-29889795-9tk7q" podUID="bbc9f851-b372-4f93-a7a8-42f67d93093c"
Oct 30 19:15:04.311646 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:04.311624    4888 scope.go:122] "RemoveContainer" containerID="17a28d6869888af6feaf82518f7f312277202a6d7186a8365fcbd5e7a230bcad"
Oct 30 19:15:05.307591 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:05.307561    4888 scope.go:122] "RemoveContainer" containerID="c97a46e96bf2271f333f1ff6e5590d71b9ee7f1c3fa67f9c098194ef7e37f56f"
Oct 30 19:15:05.308120 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:15:05.307912    4888 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w0-2ade09-29889795-9tk7q_odh-ai-gateway-infra(bbc9f851-b372-4f93-a7a8-42f67d93093c)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-2ade09-29889795-9tk7q" podUID="bbc9f851-b372-4f93-a7a8-42f67d93093c"
Oct 30 19:15:05.309771 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:05.309750    4888 scope.go:122] "RemoveContainer" containerID="703da68fc83c676cc0e331e4ee42f5f3a6e71650da8b784858ce41305b11543b"
Oct 30 19:15:05.310044 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:15:05.310018    4888 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-ipp-a-52d383-29889795-dxc57_odh-ai-gateway-infra(7208215b-516e-47d6-bb61-27e1031c7953)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-52d383-29889795-dxc57" podUID="7208215b-516e-47d6-bb61-27e1031c7953"
Oct 30 19:15:05.623892 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:15:05.623859    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-jbsw6" containerName="extractor"
Oct 30 19:15:07.363313 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:07.347538    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-52d383-5b7f46bf4c-m9fjk" event={"ID":"ade6242e-b404-446e-8474-6e64b818a3f1","Type":"ContainerStarted","Data":"bb33afc32974c4f2bc57e31e225b7ff2088147066067827f6165c3fcebfa6b4b"}
Oct 30 19:15:07.363313 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:07.347869    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-52d383-5b7f46bf4c-m9fjk"
Oct 30 19:15:07.505840 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:07.505778    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-52d383-5b7f46bf4c-m9fjk" podStartSLOduration=2.743249711 podStartE2EDuration="5.50575865s" totalImagesPullingTime="2.762508939s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:15:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:15:07.503006438 +0000 UTC m=+2471.750412466" watchObservedRunningTime="2026-10-30 19:15:07.50575865 +0000 UTC m=+2471.753164676"
Oct 30 19:15:12.061988 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:12.061946    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-0b653520-fpmp4"]
Oct 30 19:15:12.066632 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:12.066608    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0b653520-fpmp4"
Oct 30 19:15:12.120972 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:12.120934    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 19:15:12.163243 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:12.163210    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/91ee5bf1-aa98-44bd-b9da-2ce9631dfe49-maas-api-xks-ca\") pod \"maas-api-revoke-sub-0b653520-fpmp4\" (UID: \"91ee5bf1-aa98-44bd-b9da-2ce9631dfe49\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0b653520-fpmp4"
Oct 30 19:15:12.163243 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:12.163245    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6n6f4\" (UniqueName: \"kubernetes.io/projected/91ee5bf1-aa98-44bd-b9da-2ce9631dfe49-kube-api-access-6n6f4\") pod \"maas-api-revoke-sub-0b653520-fpmp4\" (UID: \"91ee5bf1-aa98-44bd-b9da-2ce9631dfe49\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0b653520-fpmp4"
Oct 30 19:15:12.163513 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:12.163423    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/91ee5bf1-aa98-44bd-b9da-2ce9631dfe49-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-0b653520-fpmp4\" (UID: \"91ee5bf1-aa98-44bd-b9da-2ce9631dfe49\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0b653520-fpmp4"
Oct 30 19:15:12.180881 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:12.180828    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-0b653520-fpmp4"]
Oct 30 19:15:12.264779 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:12.264747    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/91ee5bf1-aa98-44bd-b9da-2ce9631dfe49-maas-api-xks-ca\") pod \"maas-api-revoke-sub-0b653520-fpmp4\" (UID: \"91ee5bf1-aa98-44bd-b9da-2ce9631dfe49\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0b653520-fpmp4"
Oct 30 19:15:12.264779 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:12.264780    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6n6f4\" (UniqueName: \"kubernetes.io/projected/91ee5bf1-aa98-44bd-b9da-2ce9631dfe49-kube-api-access-6n6f4\") pod \"maas-api-revoke-sub-0b653520-fpmp4\" (UID: \"91ee5bf1-aa98-44bd-b9da-2ce9631dfe49\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0b653520-fpmp4"
Oct 30 19:15:12.265040 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:12.264837    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/91ee5bf1-aa98-44bd-b9da-2ce9631dfe49-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-0b653520-fpmp4\" (UID: \"91ee5bf1-aa98-44bd-b9da-2ce9631dfe49\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0b653520-fpmp4"
Oct 30 19:15:12.265610 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:12.265589    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/91ee5bf1-aa98-44bd-b9da-2ce9631dfe49-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-0b653520-fpmp4\" (UID: \"91ee5bf1-aa98-44bd-b9da-2ce9631dfe49\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0b653520-fpmp4"
Oct 30 19:15:12.267300 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:12.267267    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/91ee5bf1-aa98-44bd-b9da-2ce9631dfe49-maas-api-xks-ca\") pod \"maas-api-revoke-sub-0b653520-fpmp4\" (UID: \"91ee5bf1-aa98-44bd-b9da-2ce9631dfe49\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0b653520-fpmp4"
Oct 30 19:15:12.329244 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:12.329203    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6n6f4\" (UniqueName: \"kubernetes.io/projected/91ee5bf1-aa98-44bd-b9da-2ce9631dfe49-kube-api-access-6n6f4\") pod \"maas-api-revoke-sub-0b653520-fpmp4\" (UID: \"91ee5bf1-aa98-44bd-b9da-2ce9631dfe49\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0b653520-fpmp4"
Oct 30 19:15:12.385671 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:12.385633    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0b653520-fpmp4"
Oct 30 19:15:12.775271 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:12.775244    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-0b653520-fpmp4"]
Oct 30 19:15:12.775888 ip-10-0-134-46 kubenswrapper[4888]: W1030 19:15:12.775865    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91ee5bf1_aa98_44bd_b9da_2ce9631dfe49.slice/crio-2b948de6a3fda1bebba286eef6bd17886af39847a7667ffbdc598e05068e1a60 WatchSource:0}: Error finding container 2b948de6a3fda1bebba286eef6bd17886af39847a7667ffbdc598e05068e1a60: Status 404 returned error can't find the container with id 2b948de6a3fda1bebba286eef6bd17886af39847a7667ffbdc598e05068e1a60
Oct 30 19:15:13.401681 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:13.401605    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0b653520-fpmp4" event={"ID":"91ee5bf1-aa98-44bd-b9da-2ce9631dfe49","Type":"ContainerStarted","Data":"1ca565f82145a722b9ea8c57d1c00fc342746c79052c6653e8f5389d566f7531"}
Oct 30 19:15:13.401681 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:13.401660    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0b653520-fpmp4" event={"ID":"91ee5bf1-aa98-44bd-b9da-2ce9631dfe49","Type":"ContainerStarted","Data":"2b948de6a3fda1bebba286eef6bd17886af39847a7667ffbdc598e05068e1a60"}
Oct 30 19:15:13.521791 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:13.521745    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0b653520-fpmp4" podStartSLOduration=2.521729542 podStartE2EDuration="2.521729542s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:15:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:15:13.519777104 +0000 UTC m=+2477.767183133" watchObservedRunningTime="2026-10-30 19:15:13.521729542 +0000 UTC m=+2477.769135583"
Oct 30 19:15:13.883983 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:13.883938    4888 scope.go:122] "RemoveContainer" containerID="703da68fc83c676cc0e331e4ee42f5f3a6e71650da8b784858ce41305b11543b"
Oct 30 19:15:14.411118 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:14.411031    4888 generic.go:356] "Generic (PLEG): container finished" podID="91ee5bf1-aa98-44bd-b9da-2ce9631dfe49" containerID="1ca565f82145a722b9ea8c57d1c00fc342746c79052c6653e8f5389d566f7531" exitCode=0
Oct 30 19:15:14.411613 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:14.411117    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0b653520-fpmp4" event={"ID":"91ee5bf1-aa98-44bd-b9da-2ce9631dfe49","Type":"ContainerDied","Data":"1ca565f82145a722b9ea8c57d1c00fc342746c79052c6653e8f5389d566f7531"}
Oct 30 19:15:14.413355 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:14.413325    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-52d383-29889795-dxc57" event={"ID":"7208215b-516e-47d6-bb61-27e1031c7953","Type":"ContainerStarted","Data":"182c0db00967b0e3fcf562f13a20d71766b8e7b900c541e24f4461c2c413564c"}
Oct 30 19:15:15.423241 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:15.423206    4888 generic.go:356] "Generic (PLEG): container finished" podID="7208215b-516e-47d6-bb61-27e1031c7953" containerID="182c0db00967b0e3fcf562f13a20d71766b8e7b900c541e24f4461c2c413564c" exitCode=60
Oct 30 19:15:15.423727 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:15.423241    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-52d383-29889795-dxc57" event={"ID":"7208215b-516e-47d6-bb61-27e1031c7953","Type":"ContainerDied","Data":"182c0db00967b0e3fcf562f13a20d71766b8e7b900c541e24f4461c2c413564c"}
Oct 30 19:15:15.423727 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:15.423317    4888 scope.go:122] "RemoveContainer" containerID="703da68fc83c676cc0e331e4ee42f5f3a6e71650da8b784858ce41305b11543b"
Oct 30 19:15:15.423727 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:15.423673    4888 scope.go:122] "RemoveContainer" containerID="182c0db00967b0e3fcf562f13a20d71766b8e7b900c541e24f4461c2c413564c"
Oct 30 19:15:15.424230 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:15:15.423965    4888 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-ipp-a-52d383-29889795-dxc57_odh-ai-gateway-infra(7208215b-516e-47d6-bb61-27e1031c7953)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-52d383-29889795-dxc57" podUID="7208215b-516e-47d6-bb61-27e1031c7953"
Oct 30 19:15:15.585812 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:15.585780    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0b653520-fpmp4"
Oct 30 19:15:15.679437 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:15.678033    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-52d383-29889795-dxc57"]
Oct 30 19:15:15.706746 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:15.706713    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/91ee5bf1-aa98-44bd-b9da-2ce9631dfe49-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/91ee5bf1-aa98-44bd-b9da-2ce9631dfe49-maas-api-xks-ca\") pod \"91ee5bf1-aa98-44bd-b9da-2ce9631dfe49\" (UID: \"91ee5bf1-aa98-44bd-b9da-2ce9631dfe49\") "
Oct 30 19:15:15.706958 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:15.706789    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/91ee5bf1-aa98-44bd-b9da-2ce9631dfe49-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/91ee5bf1-aa98-44bd-b9da-2ce9631dfe49-maas-api-openshift-ca\") pod \"91ee5bf1-aa98-44bd-b9da-2ce9631dfe49\" (UID: \"91ee5bf1-aa98-44bd-b9da-2ce9631dfe49\") "
Oct 30 19:15:15.706958 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:15.706897    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/91ee5bf1-aa98-44bd-b9da-2ce9631dfe49-kube-api-access-6n6f4\" (UniqueName: \"kubernetes.io/projected/91ee5bf1-aa98-44bd-b9da-2ce9631dfe49-kube-api-access-6n6f4\") pod \"91ee5bf1-aa98-44bd-b9da-2ce9631dfe49\" (UID: \"91ee5bf1-aa98-44bd-b9da-2ce9631dfe49\") "
Oct 30 19:15:15.709026 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:15.708961    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/91ee5bf1-aa98-44bd-b9da-2ce9631dfe49-maas-api-openshift-ca" pod "91ee5bf1-aa98-44bd-b9da-2ce9631dfe49" (UID: "91ee5bf1-aa98-44bd-b9da-2ce9631dfe49"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:15:15.710431 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:15.710401    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/91ee5bf1-aa98-44bd-b9da-2ce9631dfe49-kube-api-access-6n6f4" pod "91ee5bf1-aa98-44bd-b9da-2ce9631dfe49" (UID: "91ee5bf1-aa98-44bd-b9da-2ce9631dfe49"). InnerVolumeSpecName "kube-api-access-6n6f4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:15:15.713985 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:15.711069    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/91ee5bf1-aa98-44bd-b9da-2ce9631dfe49-maas-api-xks-ca" pod "91ee5bf1-aa98-44bd-b9da-2ce9631dfe49" (UID: "91ee5bf1-aa98-44bd-b9da-2ce9631dfe49"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:15:15.808974 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:15.808937    4888 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/91ee5bf1-aa98-44bd-b9da-2ce9631dfe49-maas-api-xks-ca\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:15:15.808974 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:15.808976    4888 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/91ee5bf1-aa98-44bd-b9da-2ce9631dfe49-maas-api-openshift-ca\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:15:15.809234 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:15.809009    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6n6f4\" (UniqueName: \"kubernetes.io/projected/91ee5bf1-aa98-44bd-b9da-2ce9631dfe49-kube-api-access-6n6f4\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:15:15.890131 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:15.889963    4888 scope.go:122] "RemoveContainer" containerID="9cc8b34961cb8cb607b87f99d9bf233a1431b723ec66a1d88f125a4359df2ad6"
Oct 30 19:15:15.890676 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:15.890653    4888 scope.go:122] "RemoveContainer" containerID="c97a46e96bf2271f333f1ff6e5590d71b9ee7f1c3fa67f9c098194ef7e37f56f"
Oct 30 19:15:16.431160 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:16.431123    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0b653520-fpmp4" event={"ID":"91ee5bf1-aa98-44bd-b9da-2ce9631dfe49","Type":"ContainerDied","Data":"2b948de6a3fda1bebba286eef6bd17886af39847a7667ffbdc598e05068e1a60"}
Oct 30 19:15:16.431160 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:16.431164    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0b653520-fpmp4"
Oct 30 19:15:16.431743 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:16.431189    4888 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2b948de6a3fda1bebba286eef6bd17886af39847a7667ffbdc598e05068e1a60"
Oct 30 19:15:16.435932 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:16.435912    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-2ade09-29889795-9tk7q" event={"ID":"bbc9f851-b372-4f93-a7a8-42f67d93093c","Type":"ContainerStarted","Data":"b1678573ef9e81a9cb1e13cd8035fa9db78f6141e8ee1e78058f605f3b28cac0"}
Oct 30 19:15:16.438049 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:16.438029    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889795-j9vgs" event={"ID":"9fe0e09a-5f21-4382-8153-542047006bb6","Type":"ContainerStarted","Data":"dd95fc088136e813673f355bca4f64649cf4b5a2a839c65a0f71a4f2be76e98f"}
Oct 30 19:15:16.592651 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:16.592627    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-52d383-29889795-dxc57"
Oct 30 19:15:16.725629 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:16.725585    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889795-j9vgs" podStartSLOduration=16.72557129 podStartE2EDuration="16.72557129s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:15:16.724994551 +0000 UTC m=+2480.972400577" watchObservedRunningTime="2026-10-30 19:15:16.72557129 +0000 UTC m=+2480.972977316"
Oct 30 19:15:16.726321 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:16.726301    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7208215b-516e-47d6-bb61-27e1031c7953-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7208215b-516e-47d6-bb61-27e1031c7953-maas-api-service-ca\") pod \"7208215b-516e-47d6-bb61-27e1031c7953\" (UID: \"7208215b-516e-47d6-bb61-27e1031c7953\") "
Oct 30 19:15:16.726403 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:16.726388    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7208215b-516e-47d6-bb61-27e1031c7953-kube-api-access-986xn\" (UniqueName: \"kubernetes.io/projected/7208215b-516e-47d6-bb61-27e1031c7953-kube-api-access-986xn\") pod \"7208215b-516e-47d6-bb61-27e1031c7953\" (UID: \"7208215b-516e-47d6-bb61-27e1031c7953\") "
Oct 30 19:15:16.726673 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:16.726639    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7208215b-516e-47d6-bb61-27e1031c7953-maas-api-service-ca" pod "7208215b-516e-47d6-bb61-27e1031c7953" (UID: "7208215b-516e-47d6-bb61-27e1031c7953"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:15:16.726927 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:16.726905    4888 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7208215b-516e-47d6-bb61-27e1031c7953-maas-api-service-ca\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:15:16.768173 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:16.767713    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7208215b-516e-47d6-bb61-27e1031c7953-kube-api-access-986xn" pod "7208215b-516e-47d6-bb61-27e1031c7953" (UID: "7208215b-516e-47d6-bb61-27e1031c7953"). InnerVolumeSpecName "kube-api-access-986xn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:15:16.828776 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:16.828731    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-986xn\" (UniqueName: \"kubernetes.io/projected/7208215b-516e-47d6-bb61-27e1031c7953-kube-api-access-986xn\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:15:17.447881 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:17.447851    4888 generic.go:356] "Generic (PLEG): container finished" podID="bbc9f851-b372-4f93-a7a8-42f67d93093c" containerID="b1678573ef9e81a9cb1e13cd8035fa9db78f6141e8ee1e78058f605f3b28cac0" exitCode=60
Oct 30 19:15:17.448362 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:17.447932    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-2ade09-29889795-9tk7q" event={"ID":"bbc9f851-b372-4f93-a7a8-42f67d93093c","Type":"ContainerDied","Data":"b1678573ef9e81a9cb1e13cd8035fa9db78f6141e8ee1e78058f605f3b28cac0"}
Oct 30 19:15:17.448362 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:17.448006    4888 scope.go:122] "RemoveContainer" containerID="c97a46e96bf2271f333f1ff6e5590d71b9ee7f1c3fa67f9c098194ef7e37f56f"
Oct 30 19:15:17.448463 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:17.448369    4888 scope.go:122] "RemoveContainer" containerID="b1678573ef9e81a9cb1e13cd8035fa9db78f6141e8ee1e78058f605f3b28cac0"
Oct 30 19:15:17.448733 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:15:17.448697    4888 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w0-2ade09-29889795-9tk7q_odh-ai-gateway-infra(bbc9f851-b372-4f93-a7a8-42f67d93093c)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-2ade09-29889795-9tk7q" podUID="bbc9f851-b372-4f93-a7a8-42f67d93093c"
Oct 30 19:15:17.450169 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:17.450132    4888 generic.go:356] "Generic (PLEG): container finished" podID="9fe0e09a-5f21-4382-8153-542047006bb6" containerID="dd95fc088136e813673f355bca4f64649cf4b5a2a839c65a0f71a4f2be76e98f" exitCode=60
Oct 30 19:15:17.450313 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:17.450257    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889795-j9vgs" event={"ID":"9fe0e09a-5f21-4382-8153-542047006bb6","Type":"ContainerDied","Data":"dd95fc088136e813673f355bca4f64649cf4b5a2a839c65a0f71a4f2be76e98f"}
Oct 30 19:15:17.450640 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:17.450617    4888 scope.go:122] "RemoveContainer" containerID="dd95fc088136e813673f355bca4f64649cf4b5a2a839c65a0f71a4f2be76e98f"
Oct 30 19:15:17.450904 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:15:17.450879    4888 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-29889795-j9vgs_odh-ai-gateway-infra(9fe0e09a-5f21-4382-8153-542047006bb6)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889795-j9vgs" podUID="9fe0e09a-5f21-4382-8153-542047006bb6"
Oct 30 19:15:17.451660 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:17.451640    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-52d383-29889795-dxc57" event={"ID":"7208215b-516e-47d6-bb61-27e1031c7953","Type":"ContainerDied","Data":"340570533ca4cd0fec749c9f48646c7cc168b783c42092d8b8409d6ebeea06c7"}
Oct 30 19:15:17.451738 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:17.451671    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-52d383-29889795-dxc57"
Oct 30 19:15:17.461917 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:17.461892    4888 scope.go:122] "RemoveContainer" containerID="9cc8b34961cb8cb607b87f99d9bf233a1431b723ec66a1d88f125a4359df2ad6"
Oct 30 19:15:17.476582 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:17.476563    4888 scope.go:122] "RemoveContainer" containerID="182c0db00967b0e3fcf562f13a20d71766b8e7b900c541e24f4461c2c413564c"
Oct 30 19:15:17.740608 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:17.740528    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889795-j9vgs"]
Oct 30 19:15:17.819305 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:17.819259    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-52d383-29889795-dxc57"]
Oct 30 19:15:17.830066 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:17.830032    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ipp-a-52d383-29889795-dxc57"]
Oct 30 19:15:17.855506 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:17.855482    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-2ade09-29889795-9tk7q"]
Oct 30 19:15:17.903344 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:17.900150    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7208215b-516e-47d6-bb61-27e1031c7953" path="/var/lib/kubelet/pods/7208215b-516e-47d6-bb61-27e1031c7953/volumes"
Oct 30 19:15:18.357194 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:18.357167    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-52d383-5b7f46bf4c-m9fjk"
Oct 30 19:15:18.658575 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:18.658549    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889795-j9vgs"
Oct 30 19:15:18.705777 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:18.705746    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-2ade09-29889795-9tk7q"
Oct 30 19:15:18.754206 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:18.754178    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9fe0e09a-5f21-4382-8153-542047006bb6-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9fe0e09a-5f21-4382-8153-542047006bb6-maas-api-service-ca\") pod \"9fe0e09a-5f21-4382-8153-542047006bb6\" (UID: \"9fe0e09a-5f21-4382-8153-542047006bb6\") "
Oct 30 19:15:18.754428 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:18.754400    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9fe0e09a-5f21-4382-8153-542047006bb6-kube-api-access-6tlfn\" (UniqueName: \"kubernetes.io/projected/9fe0e09a-5f21-4382-8153-542047006bb6-kube-api-access-6tlfn\") pod \"9fe0e09a-5f21-4382-8153-542047006bb6\" (UID: \"9fe0e09a-5f21-4382-8153-542047006bb6\") "
Oct 30 19:15:18.754704 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:18.754674    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9fe0e09a-5f21-4382-8153-542047006bb6-maas-api-service-ca" pod "9fe0e09a-5f21-4382-8153-542047006bb6" (UID: "9fe0e09a-5f21-4382-8153-542047006bb6"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:15:18.754848 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:18.754830    4888 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9fe0e09a-5f21-4382-8153-542047006bb6-maas-api-service-ca\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:15:18.756792 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:18.756765    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9fe0e09a-5f21-4382-8153-542047006bb6-kube-api-access-6tlfn" pod "9fe0e09a-5f21-4382-8153-542047006bb6" (UID: "9fe0e09a-5f21-4382-8153-542047006bb6"). InnerVolumeSpecName "kube-api-access-6tlfn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:15:18.855459 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:18.855429    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bbc9f851-b372-4f93-a7a8-42f67d93093c-kube-api-access-8xpzz\" (UniqueName: \"kubernetes.io/projected/bbc9f851-b372-4f93-a7a8-42f67d93093c-kube-api-access-8xpzz\") pod \"bbc9f851-b372-4f93-a7a8-42f67d93093c\" (UID: \"bbc9f851-b372-4f93-a7a8-42f67d93093c\") "
Oct 30 19:15:18.855632 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:18.855569    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/bbc9f851-b372-4f93-a7a8-42f67d93093c-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bbc9f851-b372-4f93-a7a8-42f67d93093c-maas-api-service-ca\") pod \"bbc9f851-b372-4f93-a7a8-42f67d93093c\" (UID: \"bbc9f851-b372-4f93-a7a8-42f67d93093c\") "
Oct 30 19:15:18.855939 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:18.855911    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bbc9f851-b372-4f93-a7a8-42f67d93093c-maas-api-service-ca" pod "bbc9f851-b372-4f93-a7a8-42f67d93093c" (UID: "bbc9f851-b372-4f93-a7a8-42f67d93093c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:15:18.856055 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:18.856041    4888 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bbc9f851-b372-4f93-a7a8-42f67d93093c-maas-api-service-ca\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:15:18.856126 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:18.856058    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6tlfn\" (UniqueName: \"kubernetes.io/projected/9fe0e09a-5f21-4382-8153-542047006bb6-kube-api-access-6tlfn\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:15:18.857859 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:18.857830    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bbc9f851-b372-4f93-a7a8-42f67d93093c-kube-api-access-8xpzz" pod "bbc9f851-b372-4f93-a7a8-42f67d93093c" (UID: "bbc9f851-b372-4f93-a7a8-42f67d93093c"). InnerVolumeSpecName "kube-api-access-8xpzz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:15:18.957861 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:18.957783    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8xpzz\" (UniqueName: \"kubernetes.io/projected/bbc9f851-b372-4f93-a7a8-42f67d93093c-kube-api-access-8xpzz\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:15:19.481338 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:19.481276    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889795-j9vgs"
Oct 30 19:15:19.481338 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:19.481327    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889795-j9vgs" event={"ID":"9fe0e09a-5f21-4382-8153-542047006bb6","Type":"ContainerDied","Data":"7da4f88103faeed1e7a51daf7370782596e7bfddbdd3ddcc257f3c18a288c532"}
Oct 30 19:15:19.481581 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:19.481393    4888 scope.go:122] "RemoveContainer" containerID="dd95fc088136e813673f355bca4f64649cf4b5a2a839c65a0f71a4f2be76e98f"
Oct 30 19:15:19.483120 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:19.483015    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-2ade09-29889795-9tk7q"
Oct 30 19:15:19.483120 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:19.483041    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-2ade09-29889795-9tk7q" event={"ID":"bbc9f851-b372-4f93-a7a8-42f67d93093c","Type":"ContainerDied","Data":"d8dfd39e9f6a43750a572b6d34cd5de535a204f3fdfba96746f04ce80a7b5ad6"}
Oct 30 19:15:19.493060 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:19.492953    4888 scope.go:122] "RemoveContainer" containerID="b1678573ef9e81a9cb1e13cd8035fa9db78f6141e8ee1e78058f605f3b28cac0"
Oct 30 19:15:19.679711 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:19.679679    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889795-j9vgs"]
Oct 30 19:15:19.691577 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:19.691545    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889795-j9vgs"]
Oct 30 19:15:19.826134 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:19.823264    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-2ade09-29889795-9tk7q"]
Oct 30 19:15:19.828854 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:19.828824    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-2ade09-29889795-9tk7q"]
Oct 30 19:15:19.891468 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:19.891440    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9fe0e09a-5f21-4382-8153-542047006bb6" path="/var/lib/kubelet/pods/9fe0e09a-5f21-4382-8153-542047006bb6/volumes"
Oct 30 19:15:19.891867 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:19.891854    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="bbc9f851-b372-4f93-a7a8-42f67d93093c" path="/var/lib/kubelet/pods/bbc9f851-b372-4f93-a7a8-42f67d93093c/volumes"
Oct 30 19:15:35.623190 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:15:35.623157    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-jbsw6" containerName="extractor"
Oct 30 19:15:37.533783 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:37.533743    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-rjmdz"]
Oct 30 19:15:57.327522 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:57.327492    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-0c9a2b-5f4f444dc9-rvxxc"]
Oct 30 19:15:57.327935 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:57.327728    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0c9a2b-5f4f444dc9-rvxxc" podUID="108b1dc2-3b32-4ede-b94c-403cc9ffa73e" containerName="maas-api" containerID="cri-o://73a9718c2012a19fb7f62b4ad93945282dfe35028379a584fcb9f044ce36e578" gracePeriod=30
Oct 30 19:15:57.619694 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:57.619669    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0c9a2b-5f4f444dc9-rvxxc"
Oct 30 19:15:57.765532 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:57.765499    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/108b1dc2-3b32-4ede-b94c-403cc9ffa73e-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/108b1dc2-3b32-4ede-b94c-403cc9ffa73e-maas-api-tls\") pod \"108b1dc2-3b32-4ede-b94c-403cc9ffa73e\" (UID: \"108b1dc2-3b32-4ede-b94c-403cc9ffa73e\") "
Oct 30 19:15:57.765750 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:57.765573    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/108b1dc2-3b32-4ede-b94c-403cc9ffa73e-kube-api-access-b2prb\" (UniqueName: \"kubernetes.io/projected/108b1dc2-3b32-4ede-b94c-403cc9ffa73e-kube-api-access-b2prb\") pod \"108b1dc2-3b32-4ede-b94c-403cc9ffa73e\" (UID: \"108b1dc2-3b32-4ede-b94c-403cc9ffa73e\") "
Oct 30 19:15:57.765750 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:57.765603    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/108b1dc2-3b32-4ede-b94c-403cc9ffa73e-metrics-certs\" (UniqueName: \"kubernetes.io/secret/108b1dc2-3b32-4ede-b94c-403cc9ffa73e-metrics-certs\") pod \"108b1dc2-3b32-4ede-b94c-403cc9ffa73e\" (UID: \"108b1dc2-3b32-4ede-b94c-403cc9ffa73e\") "
Oct 30 19:15:57.765750 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:57.765664    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/108b1dc2-3b32-4ede-b94c-403cc9ffa73e-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/108b1dc2-3b32-4ede-b94c-403cc9ffa73e-openshift-service-ca\") pod \"108b1dc2-3b32-4ede-b94c-403cc9ffa73e\" (UID: \"108b1dc2-3b32-4ede-b94c-403cc9ffa73e\") "
Oct 30 19:15:57.766455 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:57.766352    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/108b1dc2-3b32-4ede-b94c-403cc9ffa73e-openshift-service-ca" pod "108b1dc2-3b32-4ede-b94c-403cc9ffa73e" (UID: "108b1dc2-3b32-4ede-b94c-403cc9ffa73e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:15:57.768642 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:57.768608    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/108b1dc2-3b32-4ede-b94c-403cc9ffa73e-kube-api-access-b2prb" pod "108b1dc2-3b32-4ede-b94c-403cc9ffa73e" (UID: "108b1dc2-3b32-4ede-b94c-403cc9ffa73e"). InnerVolumeSpecName "kube-api-access-b2prb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:15:57.768760 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:57.768742    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/108b1dc2-3b32-4ede-b94c-403cc9ffa73e-maas-api-tls" pod "108b1dc2-3b32-4ede-b94c-403cc9ffa73e" (UID: "108b1dc2-3b32-4ede-b94c-403cc9ffa73e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:15:57.768843 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:57.768811    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/108b1dc2-3b32-4ede-b94c-403cc9ffa73e-metrics-certs" pod "108b1dc2-3b32-4ede-b94c-403cc9ffa73e" (UID: "108b1dc2-3b32-4ede-b94c-403cc9ffa73e"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:15:57.823635 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:57.823604    4888 generic.go:356] "Generic (PLEG): container finished" podID="108b1dc2-3b32-4ede-b94c-403cc9ffa73e" containerID="73a9718c2012a19fb7f62b4ad93945282dfe35028379a584fcb9f044ce36e578" exitCode=0
Oct 30 19:15:57.823808 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:57.823653    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0c9a2b-5f4f444dc9-rvxxc"
Oct 30 19:15:57.823808 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:57.823687    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0c9a2b-5f4f444dc9-rvxxc" event={"ID":"108b1dc2-3b32-4ede-b94c-403cc9ffa73e","Type":"ContainerDied","Data":"73a9718c2012a19fb7f62b4ad93945282dfe35028379a584fcb9f044ce36e578"}
Oct 30 19:15:57.823808 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:57.823747    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0c9a2b-5f4f444dc9-rvxxc" event={"ID":"108b1dc2-3b32-4ede-b94c-403cc9ffa73e","Type":"ContainerDied","Data":"70954ee5c18d4bb5ac4168611dd702634df23c0abbdaeca1ca4fe694b227cf4e"}
Oct 30 19:15:57.823808 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:57.823774    4888 scope.go:122] "RemoveContainer" containerID="73a9718c2012a19fb7f62b4ad93945282dfe35028379a584fcb9f044ce36e578"
Oct 30 19:15:57.849629 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:57.849605    4888 scope.go:122] "RemoveContainer" containerID="73a9718c2012a19fb7f62b4ad93945282dfe35028379a584fcb9f044ce36e578"
Oct 30 19:15:57.850050 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:15:57.850008    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"73a9718c2012a19fb7f62b4ad93945282dfe35028379a584fcb9f044ce36e578\": container with ID starting with 73a9718c2012a19fb7f62b4ad93945282dfe35028379a584fcb9f044ce36e578 not found: ID does not exist" containerID="73a9718c2012a19fb7f62b4ad93945282dfe35028379a584fcb9f044ce36e578"
Oct 30 19:15:57.850109 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:57.850076    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"73a9718c2012a19fb7f62b4ad93945282dfe35028379a584fcb9f044ce36e578"} err="failed to get container status \"73a9718c2012a19fb7f62b4ad93945282dfe35028379a584fcb9f044ce36e578\": rpc error: code = NotFound desc = could not find container \"73a9718c2012a19fb7f62b4ad93945282dfe35028379a584fcb9f044ce36e578\": container with ID starting with 73a9718c2012a19fb7f62b4ad93945282dfe35028379a584fcb9f044ce36e578 not found: ID does not exist"
Oct 30 19:15:57.866926 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:57.866902    4888 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/108b1dc2-3b32-4ede-b94c-403cc9ffa73e-maas-api-tls\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:15:57.866926 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:57.866927    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b2prb\" (UniqueName: \"kubernetes.io/projected/108b1dc2-3b32-4ede-b94c-403cc9ffa73e-kube-api-access-b2prb\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:15:57.867137 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:57.866937    4888 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/108b1dc2-3b32-4ede-b94c-403cc9ffa73e-metrics-certs\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:15:57.867137 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:57.866947    4888 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/108b1dc2-3b32-4ede-b94c-403cc9ffa73e-openshift-service-ca\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:15:57.994005 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:57.993976    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-0c9a2b-5f4f444dc9-rvxxc"]
Oct 30 19:15:58.001000 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:58.000967    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-0c9a2b-5f4f444dc9-rvxxc"]
Oct 30 19:15:59.890760 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:15:59.890725    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="108b1dc2-3b32-4ede-b94c-403cc9ffa73e" path="/var/lib/kubelet/pods/108b1dc2-3b32-4ede-b94c-403cc9ffa73e/volumes"
Oct 30 19:16:05.619172 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:16:05.619133    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-jbsw6" containerName="extractor"
Oct 30 19:16:23.615189 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:16:23.615157    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-vdg7j"]
Oct 30 19:16:23.615988 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:16:23.615968    4888 cpu_manager.go:414] "containerMap: removing container" podUID="7208215b-516e-47d6-bb61-27e1031c7953" containerName="cleanup"
Oct 30 19:16:23.616091 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:16:23.615991    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7208215b-516e-47d6-bb61-27e1031c7953" containerName="cleanup"
Oct 30 19:16:23.616091 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:16:23.616007    4888 cpu_manager.go:414] "containerMap: removing container" podUID="108b1dc2-3b32-4ede-b94c-403cc9ffa73e" containerName="maas-api"
Oct 30 19:16:23.616091 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:16:23.616014    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="108b1dc2-3b32-4ede-b94c-403cc9ffa73e" containerName="maas-api"
Oct 30 19:16:23.616091 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:16:23.616032    4888 cpu_manager.go:414] "containerMap: removing container" podUID="bbc9f851-b372-4f93-a7a8-42f67d93093c" containerName="cleanup"
Oct 30 19:16:23.616091 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:16:23.616040    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bbc9f851-b372-4f93-a7a8-42f67d93093c" containerName="cleanup"
Oct 30 19:16:23.616091 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:16:23.616064    4888 cpu_manager.go:414] "containerMap: removing container" podUID="9fe0e09a-5f21-4382-8153-542047006bb6" containerName="cleanup"
Oct 30 19:16:23.616091 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:16:23.616070    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9fe0e09a-5f21-4382-8153-542047006bb6" containerName="cleanup"
Oct 30 19:16:23.616091 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:16:23.616088    4888 cpu_manager.go:414] "containerMap: removing container" podUID="7208215b-516e-47d6-bb61-27e1031c7953" containerName="cleanup"
Oct 30 19:16:23.616549 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:16:23.616095    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7208215b-516e-47d6-bb61-27e1031c7953" containerName="cleanup"
Oct 30 19:16:23.616549 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:16:23.616117    4888 cpu_manager.go:414] "containerMap: removing container" podUID="9fe0e09a-5f21-4382-8153-542047006bb6" containerName="cleanup"
Oct 30 19:16:23.616549 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:16:23.616124    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9fe0e09a-5f21-4382-8153-542047006bb6" containerName="cleanup"
Oct 30 19:16:23.616549 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:16:23.616134    4888 cpu_manager.go:414] "containerMap: removing container" podUID="bbc9f851-b372-4f93-a7a8-42f67d93093c" containerName="cleanup"
Oct 30 19:16:23.616549 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:16:23.616141    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bbc9f851-b372-4f93-a7a8-42f67d93093c" containerName="cleanup"
Oct 30 19:16:23.616549 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:16:23.616160    4888 cpu_manager.go:414] "containerMap: removing container" podUID="91ee5bf1-aa98-44bd-b9da-2ce9631dfe49" containerName="revoke-keys"
Oct 30 19:16:23.616549 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:16:23.616168    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="91ee5bf1-aa98-44bd-b9da-2ce9631dfe49" containerName="revoke-keys"
Oct 30 19:16:23.616549 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:16:23.616354    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="9fe0e09a-5f21-4382-8153-542047006bb6" containerName="cleanup"
Oct 30 19:16:23.616549 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:16:23.616369    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="bbc9f851-b372-4f93-a7a8-42f67d93093c" containerName="cleanup"
Oct 30 19:16:23.616549 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:16:23.616381    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="91ee5bf1-aa98-44bd-b9da-2ce9631dfe49" containerName="revoke-keys"
Oct 30 19:16:23.616549 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:16:23.616396    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="7208215b-516e-47d6-bb61-27e1031c7953" containerName="cleanup"
Oct 30 19:16:23.616549 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:16:23.616405    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="bbc9f851-b372-4f93-a7a8-42f67d93093c" containerName="cleanup"
Oct 30 19:16:23.616549 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:16:23.616417    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="7208215b-516e-47d6-bb61-27e1031c7953" containerName="cleanup"
Oct 30 19:16:23.616549 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:16:23.616426    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="108b1dc2-3b32-4ede-b94c-403cc9ffa73e" containerName="maas-api"
Oct 30 19:16:23.616549 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:16:23.616437    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="bbc9f851-b372-4f93-a7a8-42f67d93093c" containerName="cleanup"
Oct 30 19:16:23.616549 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:16:23.616451    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="9fe0e09a-5f21-4382-8153-542047006bb6" containerName="cleanup"
Oct 30 19:16:23.619732 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:16:23.619712    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-vdg7j"
Oct 30 19:16:23.691327 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:16:23.691270    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c841723f-12c0-449f-bd04-96b459feb185-workload-socket\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-vdg7j\" (UID: \"c841723f-12c0-449f-bd04-96b459feb185\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-vdg7j"
Oct 30 19:16:23.691532 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:16:23.691452    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c841723f-12c0-449f-bd04-96b459feb185-workload-certs\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-vdg7j\" (UID: \"c841723f-12c0-449f-bd04-96b459feb185\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-vdg7j"
Oct 30 19:16:23.691532 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:16:23.691524    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c841723f-12c0-449f-bd04-96b459feb185-istiod-ca-cert\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-vdg7j\" (UID: \"c841723f-12c0-449f-bd04-96b459feb185\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-vdg7j"
Oct 30 19:16:23.691671 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:16:23.691605    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c841723f-12c0-449f-bd04-96b459feb185-credential-socket\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-vdg7j\" (UID: \"c841723f-12c0-449f-bd04-96b459feb185\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-vdg7j"
Oct 30 19:16:23.691837 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:16:23.691807    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c841723f-12c0-449f-bd04-96b459feb185-istio-token\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-vdg7j\" (UID: \"c841723f-12c0-449f-bd04-96b459feb185\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-vdg7j"
Oct 30 19:16:23.691919 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:16:23.691850    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gzh4t\" (UniqueName: \"kubernetes.io/projected/c841723f-12c0-449f-bd04-96b459feb185-kube-api-access-gzh4t\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-vdg7j\" (UID: \"c841723f-12c0-449f-bd04-96b459feb185\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-vdg7j"
Oct 30 19:16:23.691981 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:16:23.691952    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c841723f-12c0-449f-bd04-96b459feb185-istio-data\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-vdg7j\" (UID: \"c841723f-12c0-449f-bd04-96b459feb185\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-vdg7j"
Oct 30 19:16:23.692069 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:16:23.692035    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c841723f-12c0-449f-bd04-96b459feb185-istio-envoy\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-vdg7j\" (UID: \"c841723f-12c0-449f-bd04-96b459feb185\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-vdg7j"
Oct 30 19:16:23.692130 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:16:23.692110    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c841723f-12c0-449f-bd04-96b459feb185-istio-podinfo\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-vdg7j\" (UID: \"c841723f-12c0-449f-bd04-96b459feb185\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-vdg7j"
Oct 30 19:16:23.728178 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:16:23.728136    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-vdg7j"]
Oct 30 19:16:23.792891 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:16:23.792856    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c841723f-12c0-449f-bd04-96b459feb185-istio-token\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-vdg7j\" (UID: \"c841723f-12c0-449f-bd04-96b459feb185\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-vdg7j"
Oct 30 19:16:23.792891 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:16:23.792896    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gzh4t\" (UniqueName: \"kubernetes.io/projected/c841723f-12c0-449f-bd04-96b459feb185-kube-api-access-gzh4t\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-vdg7j\" (UID: \"c841723f-12c0-449f-bd04-96b459feb185\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-vdg7j"
Oct 30 19:16:23.793148 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:16:23.792929    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c841723f-12c0-449f-bd04-96b459feb185-istio-data\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-vdg7j\" (UID: \"c841723f-12c0-449f-bd04-96b459feb185\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-vdg7j"
Oct 30 19:16:23.793148 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:16:23.792950    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c841723f-12c0-449f-bd04-96b459feb185-istio-envoy\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-vdg7j\" (UID: \"c841723f-12c0-449f-bd04-96b459feb185\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-vdg7j"
Oct 30 19:16:23.793148 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:16:23.792981    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c841723f-12c0-449f-bd04-96b459feb185-istio-podinfo\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-vdg7j\" (UID: \"c841723f-12c0-449f-bd04-96b459feb185\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-vdg7j"
Oct 30 19:16:23.793148 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:16:23.793005    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c841723f-12c0-449f-bd04-96b459feb185-workload-socket\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-vdg7j\" (UID: \"c841723f-12c0-449f-bd04-96b459feb185\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-vdg7j"
Oct 30 19:16:23.793148 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:16:23.793057    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c841723f-12c0-449f-bd04-96b459feb185-workload-certs\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-vdg7j\" (UID: \"c841723f-12c0-449f-bd04-96b459feb185\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-vdg7j"
Oct 30 19:16:23.793148 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:16:23.793095    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c841723f-12c0-449f-bd04-96b459feb185-istiod-ca-cert\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-vdg7j\" (UID: \"c841723f-12c0-449f-bd04-96b459feb185\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-vdg7j"
Oct 30 19:16:23.793148 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:16:23.793123    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c841723f-12c0-449f-bd04-96b459feb185-credential-socket\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-vdg7j\" (UID: \"c841723f-12c0-449f-bd04-96b459feb185\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-vdg7j"
Oct 30 19:16:23.793529 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:16:23.793467    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c841723f-12c0-449f-bd04-96b459feb185-istio-data\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-vdg7j\" (UID: \"c841723f-12c0-449f-bd04-96b459feb185\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-vdg7j"
Oct 30 19:16:23.793812 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:16:23.793787    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c841723f-12c0-449f-bd04-96b459feb185-workload-socket\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-vdg7j\" (UID: \"c841723f-12c0-449f-bd04-96b459feb185\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-vdg7j"
Oct 30 19:16:23.793924 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:16:23.793841    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c841723f-12c0-449f-bd04-96b459feb185-istiod-ca-cert\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-vdg7j\" (UID: \"c841723f-12c0-449f-bd04-96b459feb185\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-vdg7j"
Oct 30 19:16:23.793924 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:16:23.793889    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c841723f-12c0-449f-bd04-96b459feb185-workload-certs\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-vdg7j\" (UID: \"c841723f-12c0-449f-bd04-96b459feb185\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-vdg7j"
Oct 30 19:16:23.794007 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:16:23.793958    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c841723f-12c0-449f-bd04-96b459feb185-credential-socket\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-vdg7j\" (UID: \"c841723f-12c0-449f-bd04-96b459feb185\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-vdg7j"
Oct 30 19:16:23.795772 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:16:23.795755    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c841723f-12c0-449f-bd04-96b459feb185-istio-envoy\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-vdg7j\" (UID: \"c841723f-12c0-449f-bd04-96b459feb185\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-vdg7j"
Oct 30 19:16:23.795902 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:16:23.795882    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c841723f-12c0-449f-bd04-96b459feb185-istio-podinfo\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-vdg7j\" (UID: \"c841723f-12c0-449f-bd04-96b459feb185\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-vdg7j"
Oct 30 19:16:23.856140 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:16:23.856109    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c841723f-12c0-449f-bd04-96b459feb185-istio-token\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-vdg7j\" (UID: \"c841723f-12c0-449f-bd04-96b459feb185\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-vdg7j"
Oct 30 19:16:23.856322 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:16:23.856242    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gzh4t\" (UniqueName: \"kubernetes.io/projected/c841723f-12c0-449f-bd04-96b459feb185-kube-api-access-gzh4t\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-vdg7j\" (UID: \"c841723f-12c0-449f-bd04-96b459feb185\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-vdg7j"
Oct 30 19:16:23.933260 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:16:23.933173    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-vdg7j"
Oct 30 19:16:24.368087 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:16:24.368056    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-vdg7j"]
Oct 30 19:16:24.368450 ip-10-0-134-46 kubenswrapper[4888]: W1030 19:16:24.368431    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc841723f_12c0_449f_bd04_96b459feb185.slice/crio-f195e8146d4e0a2c8ee44f7043d4eafec38506546836881826e6eb45cbed7c53 WatchSource:0}: Error finding container f195e8146d4e0a2c8ee44f7043d4eafec38506546836881826e6eb45cbed7c53: Status 404 returned error can't find the container with id f195e8146d4e0a2c8ee44f7043d4eafec38506546836881826e6eb45cbed7c53
Oct 30 19:16:24.370956 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:16:24.370925    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:16:24.371032 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:16:24.370990    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:16:24.371032 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:16:24.371019    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:16:25.029160 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:16:25.029123    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-vdg7j" event={"ID":"c841723f-12c0-449f-bd04-96b459feb185","Type":"ContainerStarted","Data":"64b30a0e50ed9cf9e8046c343673c0b9c9fb27c873852f76a26eabc5c8c17368"}
Oct 30 19:16:25.029602 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:16:25.029184    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-vdg7j" event={"ID":"c841723f-12c0-449f-bd04-96b459feb185","Type":"ContainerStarted","Data":"f195e8146d4e0a2c8ee44f7043d4eafec38506546836881826e6eb45cbed7c53"}
Oct 30 19:16:25.157142 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:16:25.157089    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-vdg7j" podStartSLOduration=2.15707561 podStartE2EDuration="2.15707561s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:16:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:16:25.154305801 +0000 UTC m=+2549.401711819" watchObservedRunningTime="2026-10-30 19:16:25.15707561 +0000 UTC m=+2549.404481673"
Oct 30 19:16:25.933683 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:16:25.933647    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-vdg7j"
Oct 30 19:16:25.935577 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:16:25.935552    4888 patch_prober.go:29] interesting pod/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-vdg7j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.134:15021/healthz/ready\": dial tcp 10.132.0.134:15021: connect: connection refused" start-of-body=
Oct 30 19:16:25.935710 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:16:25.935599    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-vdg7j" podUID="c841723f-12c0-449f-bd04-96b459feb185" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.134:15021/healthz/ready\": dial tcp 10.132.0.134:15021: connect: connection refused"
Oct 30 19:16:26.933989 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:16:26.933956    4888 patch_prober.go:29] interesting pod/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-vdg7j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.134:15021/healthz/ready\": dial tcp 10.132.0.134:15021: connect: connection refused" start-of-body=
Oct 30 19:16:26.934464 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:16:26.934022    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-vdg7j" podUID="c841723f-12c0-449f-bd04-96b459feb185" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.134:15021/healthz/ready\": dial tcp 10.132.0.134:15021: connect: connection refused"
Oct 30 19:16:27.934457 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:16:27.934413    4888 patch_prober.go:29] interesting pod/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-vdg7j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.134:15021/healthz/ready\": dial tcp 10.132.0.134:15021: connect: connection refused" start-of-body=
Oct 30 19:16:27.934865 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:16:27.934475    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-vdg7j" podUID="c841723f-12c0-449f-bd04-96b459feb185" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.134:15021/healthz/ready\": dial tcp 10.132.0.134:15021: connect: connection refused"
Oct 30 19:16:28.933917 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:16:28.933877    4888 patch_prober.go:29] interesting pod/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-vdg7j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.134:15021/healthz/ready\": dial tcp 10.132.0.134:15021: connect: connection refused" start-of-body=
Oct 30 19:16:28.934110 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:16:28.933954    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-vdg7j" podUID="c841723f-12c0-449f-bd04-96b459feb185" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.134:15021/healthz/ready\": dial tcp 10.132.0.134:15021: connect: connection refused"
Oct 30 19:16:29.933999 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:16:29.933968    4888 patch_prober.go:29] interesting pod/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-vdg7j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.134:15021/healthz/ready\": dial tcp 10.132.0.134:15021: connect: connection refused" start-of-body=
Oct 30 19:16:29.934524 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:16:29.934016    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-vdg7j" podUID="c841723f-12c0-449f-bd04-96b459feb185" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.134:15021/healthz/ready\": dial tcp 10.132.0.134:15021: connect: connection refused"
Oct 30 19:16:30.936823 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:16:30.936787    4888 patch_prober.go:29] interesting pod/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-vdg7j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.134:15021/healthz/ready\": dial tcp 10.132.0.134:15021: connect: connection refused" start-of-body=
Oct 30 19:16:30.937428 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:16:30.937394    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-vdg7j" podUID="c841723f-12c0-449f-bd04-96b459feb185" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.134:15021/healthz/ready\": dial tcp 10.132.0.134:15021: connect: connection refused"
Oct 30 19:16:31.937852 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:16:31.937822    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-vdg7j"
Oct 30 19:16:31.938257 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:16:31.938207    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-vdg7j"
Oct 30 19:16:31.938941 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:16:31.938921    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-vdg7j"
Oct 30 19:16:35.619570 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:16:35.619481    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-jbsw6" containerName="extractor"
Oct 30 19:16:36.043631 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:16:36.043533    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-rjmdz"]
Oct 30 19:16:49.893319 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:16:49.893267    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-0c9a2b-openshift-default-85695844fc-wsjcl"]
Oct 30 19:16:49.893885 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:16:49.893849    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-a-0c9a2b-openshift-default-85695844fc-wsjcl" podUID="3df2a357-4896-4c99-831b-96fe4cd18fb5" containerName="istio-proxy" containerID="cri-o://5d1b1eb66907ff2011cf4aac8c7e07bdb8cc86674b20b3a6011c689327fc5729" gracePeriod=30
Oct 30 19:16:55.203406 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:16:55.203377    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-0c9a2b-openshift-default-85695844fc-wsjcl"
Oct 30 19:16:55.234881 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:16:55.234849    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3df2a357-4896-4c99-831b-96fe4cd18fb5-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3df2a357-4896-4c99-831b-96fe4cd18fb5-credential-socket\") pod \"3df2a357-4896-4c99-831b-96fe4cd18fb5\" (UID: \"3df2a357-4896-4c99-831b-96fe4cd18fb5\") "
Oct 30 19:16:55.235055 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:16:55.234904    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/3df2a357-4896-4c99-831b-96fe4cd18fb5-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3df2a357-4896-4c99-831b-96fe4cd18fb5-istio-podinfo\") pod \"3df2a357-4896-4c99-831b-96fe4cd18fb5\" (UID: \"3df2a357-4896-4c99-831b-96fe4cd18fb5\") "
Oct 30 19:16:55.235055 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:16:55.234941    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3df2a357-4896-4c99-831b-96fe4cd18fb5-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3df2a357-4896-4c99-831b-96fe4cd18fb5-istio-data\") pod \"3df2a357-4896-4c99-831b-96fe4cd18fb5\" (UID: \"3df2a357-4896-4c99-831b-96fe4cd18fb5\") "
Oct 30 19:16:55.235055 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:16:55.235010    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3df2a357-4896-4c99-831b-96fe4cd18fb5-kube-api-access-r9m7n\" (UniqueName: \"kubernetes.io/projected/3df2a357-4896-4c99-831b-96fe4cd18fb5-kube-api-access-r9m7n\") pod \"3df2a357-4896-4c99-831b-96fe4cd18fb5\" (UID: \"3df2a357-4896-4c99-831b-96fe4cd18fb5\") "
Oct 30 19:16:55.235055 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:16:55.235041    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3df2a357-4896-4c99-831b-96fe4cd18fb5-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3df2a357-4896-4c99-831b-96fe4cd18fb5-istio-envoy\") pod \"3df2a357-4896-4c99-831b-96fe4cd18fb5\" (UID: \"3df2a357-4896-4c99-831b-96fe4cd18fb5\") "
Oct 30 19:16:55.235408 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:16:55.235065    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3df2a357-4896-4c99-831b-96fe4cd18fb5-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3df2a357-4896-4c99-831b-96fe4cd18fb5-istiod-ca-cert\") pod \"3df2a357-4896-4c99-831b-96fe4cd18fb5\" (UID: \"3df2a357-4896-4c99-831b-96fe4cd18fb5\") "
Oct 30 19:16:55.235408 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:16:55.235101    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3df2a357-4896-4c99-831b-96fe4cd18fb5-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3df2a357-4896-4c99-831b-96fe4cd18fb5-workload-socket\") pod \"3df2a357-4896-4c99-831b-96fe4cd18fb5\" (UID: \"3df2a357-4896-4c99-831b-96fe4cd18fb5\") "
Oct 30 19:16:55.235408 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:16:55.235222    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3df2a357-4896-4c99-831b-96fe4cd18fb5-credential-socket" pod "3df2a357-4896-4c99-831b-96fe4cd18fb5" (UID: "3df2a357-4896-4c99-831b-96fe4cd18fb5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:16:55.235408 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:16:55.235330    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3df2a357-4896-4c99-831b-96fe4cd18fb5-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3df2a357-4896-4c99-831b-96fe4cd18fb5-workload-certs\") pod \"3df2a357-4896-4c99-831b-96fe4cd18fb5\" (UID: \"3df2a357-4896-4c99-831b-96fe4cd18fb5\") "
Oct 30 19:16:55.235408 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:16:55.235367    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3df2a357-4896-4c99-831b-96fe4cd18fb5-istio-token\" (UniqueName: \"kubernetes.io/projected/3df2a357-4896-4c99-831b-96fe4cd18fb5-istio-token\") pod \"3df2a357-4896-4c99-831b-96fe4cd18fb5\" (UID: \"3df2a357-4896-4c99-831b-96fe4cd18fb5\") "
Oct 30 19:16:55.235675 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:16:55.235481    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3df2a357-4896-4c99-831b-96fe4cd18fb5-workload-socket" pod "3df2a357-4896-4c99-831b-96fe4cd18fb5" (UID: "3df2a357-4896-4c99-831b-96fe4cd18fb5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:16:55.235675 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:16:55.235605    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3df2a357-4896-4c99-831b-96fe4cd18fb5-workload-certs" pod "3df2a357-4896-4c99-831b-96fe4cd18fb5" (UID: "3df2a357-4896-4c99-831b-96fe4cd18fb5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:16:55.235675 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:16:55.235647    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3df2a357-4896-4c99-831b-96fe4cd18fb5-istio-data" pod "3df2a357-4896-4c99-831b-96fe4cd18fb5" (UID: "3df2a357-4896-4c99-831b-96fe4cd18fb5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:16:55.235825 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:16:55.235779    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3df2a357-4896-4c99-831b-96fe4cd18fb5-istiod-ca-cert" pod "3df2a357-4896-4c99-831b-96fe4cd18fb5" (UID: "3df2a357-4896-4c99-831b-96fe4cd18fb5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:16:55.236195 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:16:55.236161    4888 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3df2a357-4896-4c99-831b-96fe4cd18fb5-workload-certs\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:16:55.236195 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:16:55.236186    4888 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3df2a357-4896-4c99-831b-96fe4cd18fb5-credential-socket\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:16:55.236350 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:16:55.236200    4888 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3df2a357-4896-4c99-831b-96fe4cd18fb5-istio-data\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:16:55.236350 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:16:55.236214    4888 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3df2a357-4896-4c99-831b-96fe4cd18fb5-istiod-ca-cert\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:16:55.236350 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:16:55.236226    4888 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3df2a357-4896-4c99-831b-96fe4cd18fb5-workload-socket\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:16:55.237826 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:16:55.237791    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3df2a357-4896-4c99-831b-96fe4cd18fb5-istio-token" pod "3df2a357-4896-4c99-831b-96fe4cd18fb5" (UID: "3df2a357-4896-4c99-831b-96fe4cd18fb5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:16:55.237930 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:16:55.237890    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3df2a357-4896-4c99-831b-96fe4cd18fb5-istio-envoy" pod "3df2a357-4896-4c99-831b-96fe4cd18fb5" (UID: "3df2a357-4896-4c99-831b-96fe4cd18fb5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:16:55.238072 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:16:55.238051    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3df2a357-4896-4c99-831b-96fe4cd18fb5-kube-api-access-r9m7n" pod "3df2a357-4896-4c99-831b-96fe4cd18fb5" (UID: "3df2a357-4896-4c99-831b-96fe4cd18fb5"). InnerVolumeSpecName "kube-api-access-r9m7n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:16:55.238072 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:16:55.238048    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3df2a357-4896-4c99-831b-96fe4cd18fb5-istio-podinfo" pod "3df2a357-4896-4c99-831b-96fe4cd18fb5" (UID: "3df2a357-4896-4c99-831b-96fe4cd18fb5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:16:55.295837 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:16:55.295787    4888 generic.go:356] "Generic (PLEG): container finished" podID="3df2a357-4896-4c99-831b-96fe4cd18fb5" containerID="5d1b1eb66907ff2011cf4aac8c7e07bdb8cc86674b20b3a6011c689327fc5729" exitCode=0
Oct 30 19:16:55.296028 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:16:55.295862    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-0c9a2b-openshift-default-85695844fc-wsjcl" event={"ID":"3df2a357-4896-4c99-831b-96fe4cd18fb5","Type":"ContainerDied","Data":"5d1b1eb66907ff2011cf4aac8c7e07bdb8cc86674b20b3a6011c689327fc5729"}
Oct 30 19:16:55.296028 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:16:55.295913    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-0c9a2b-openshift-default-85695844fc-wsjcl" event={"ID":"3df2a357-4896-4c99-831b-96fe4cd18fb5","Type":"ContainerDied","Data":"70aadca04473b75faa967ba029c393237997501d6ebae28d53bb2e3cf44195bc"}
Oct 30 19:16:55.296028 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:16:55.295941    4888 scope.go:122] "RemoveContainer" containerID="5d1b1eb66907ff2011cf4aac8c7e07bdb8cc86674b20b3a6011c689327fc5729"
Oct 30 19:16:55.296225 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:16:55.296204    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-0c9a2b-openshift-default-85695844fc-wsjcl"
Oct 30 19:16:55.321993 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:16:55.321971    4888 scope.go:122] "RemoveContainer" containerID="503b82a28a2b628b8837da4256013f10eb9afed70a785e50e74b1daa6ae0a8fe"
Oct 30 19:16:55.332419 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:16:55.332400    4888 scope.go:122] "RemoveContainer" containerID="5d1b1eb66907ff2011cf4aac8c7e07bdb8cc86674b20b3a6011c689327fc5729"
Oct 30 19:16:55.332716 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:16:55.332700    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5d1b1eb66907ff2011cf4aac8c7e07bdb8cc86674b20b3a6011c689327fc5729\": container with ID starting with 5d1b1eb66907ff2011cf4aac8c7e07bdb8cc86674b20b3a6011c689327fc5729 not found: ID does not exist" containerID="5d1b1eb66907ff2011cf4aac8c7e07bdb8cc86674b20b3a6011c689327fc5729"
Oct 30 19:16:55.332794 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:16:55.332728    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5d1b1eb66907ff2011cf4aac8c7e07bdb8cc86674b20b3a6011c689327fc5729"} err="failed to get container status \"5d1b1eb66907ff2011cf4aac8c7e07bdb8cc86674b20b3a6011c689327fc5729\": rpc error: code = NotFound desc = could not find container \"5d1b1eb66907ff2011cf4aac8c7e07bdb8cc86674b20b3a6011c689327fc5729\": container with ID starting with 5d1b1eb66907ff2011cf4aac8c7e07bdb8cc86674b20b3a6011c689327fc5729 not found: ID does not exist"
Oct 30 19:16:55.332794 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:16:55.332752    4888 scope.go:122] "RemoveContainer" containerID="503b82a28a2b628b8837da4256013f10eb9afed70a785e50e74b1daa6ae0a8fe"
Oct 30 19:16:55.332997 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:16:55.332978    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"503b82a28a2b628b8837da4256013f10eb9afed70a785e50e74b1daa6ae0a8fe\": container with ID starting with 503b82a28a2b628b8837da4256013f10eb9afed70a785e50e74b1daa6ae0a8fe not found: ID does not exist" containerID="503b82a28a2b628b8837da4256013f10eb9afed70a785e50e74b1daa6ae0a8fe"
Oct 30 19:16:55.333052 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:16:55.333006    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"503b82a28a2b628b8837da4256013f10eb9afed70a785e50e74b1daa6ae0a8fe"} err="failed to get container status \"503b82a28a2b628b8837da4256013f10eb9afed70a785e50e74b1daa6ae0a8fe\": rpc error: code = NotFound desc = could not find container \"503b82a28a2b628b8837da4256013f10eb9afed70a785e50e74b1daa6ae0a8fe\": container with ID starting with 503b82a28a2b628b8837da4256013f10eb9afed70a785e50e74b1daa6ae0a8fe not found: ID does not exist"
Oct 30 19:16:55.337893 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:16:55.337873    4888 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3df2a357-4896-4c99-831b-96fe4cd18fb5-istio-podinfo\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:16:55.337970 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:16:55.337897    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r9m7n\" (UniqueName: \"kubernetes.io/projected/3df2a357-4896-4c99-831b-96fe4cd18fb5-kube-api-access-r9m7n\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:16:55.337970 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:16:55.337908    4888 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3df2a357-4896-4c99-831b-96fe4cd18fb5-istio-envoy\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:16:55.337970 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:16:55.337916    4888 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3df2a357-4896-4c99-831b-96fe4cd18fb5-istio-token\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:16:55.517456 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:16:55.517372    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-0c9a2b-openshift-default-85695844fc-wsjcl"]
Oct 30 19:16:55.520528 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:16:55.520505    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-a-0c9a2b-openshift-default-85695844fc-wsjcl"]
Oct 30 19:16:55.906103 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:16:55.906069    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3df2a357-4896-4c99-831b-96fe4cd18fb5" path="/var/lib/kubelet/pods/3df2a357-4896-4c99-831b-96fe4cd18fb5/volumes"
Oct 30 19:17:01.667133 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:01.667103    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-26cqt"]
Oct 30 19:17:01.667710 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:01.667321    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-26cqt" podUID="93de7d1c-bc7a-499d-97f6-3320e2e4e948" containerName="istio-proxy" containerID="cri-o://3e98e7afa364c552b0b15f972202160c04c2b8deae9efa1e9e19fef9b57dc4ed" gracePeriod=30
Oct 30 19:17:03.977750 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:03.977713    4888 patch_prober.go:29] interesting pod/e2e-worker-w0-2ade09-openshift-default-785b7cf675-26cqt container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 19:17:03.978141 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:03.977782    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-26cqt" podUID="93de7d1c-bc7a-499d-97f6-3320e2e4e948" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 19:17:04.376566 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:04.376530    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-praxis-a-f554bb-openshift-default-5f8f5599d5-x2l7h"]
Oct 30 19:17:04.376827 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:04.376781    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-praxis-a-f554bb-openshift-default-5f8f5599d5-x2l7h" podUID="0c0bee10-4345-43a8-9008-b60054dc6850" containerName="istio-proxy" containerID="cri-o://4096fb8ce41b1b43297c7e16718d23dd852fa2009a7420f4e5a9d190da9f1843" gracePeriod=30
Oct 30 19:17:05.619707 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:17:05.619668    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-jbsw6" containerName="extractor"
Oct 30 19:17:06.958031 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:06.958006    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-26cqt"
Oct 30 19:17:07.115313 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:07.115268    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/93de7d1c-bc7a-499d-97f6-3320e2e4e948-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/93de7d1c-bc7a-499d-97f6-3320e2e4e948-workload-socket\") pod \"93de7d1c-bc7a-499d-97f6-3320e2e4e948\" (UID: \"93de7d1c-bc7a-499d-97f6-3320e2e4e948\") "
Oct 30 19:17:07.115496 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:07.115346    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/93de7d1c-bc7a-499d-97f6-3320e2e4e948-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/93de7d1c-bc7a-499d-97f6-3320e2e4e948-istio-podinfo\") pod \"93de7d1c-bc7a-499d-97f6-3320e2e4e948\" (UID: \"93de7d1c-bc7a-499d-97f6-3320e2e4e948\") "
Oct 30 19:17:07.115496 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:07.115368    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/93de7d1c-bc7a-499d-97f6-3320e2e4e948-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/93de7d1c-bc7a-499d-97f6-3320e2e4e948-credential-socket\") pod \"93de7d1c-bc7a-499d-97f6-3320e2e4e948\" (UID: \"93de7d1c-bc7a-499d-97f6-3320e2e4e948\") "
Oct 30 19:17:07.115496 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:07.115397    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/93de7d1c-bc7a-499d-97f6-3320e2e4e948-kube-api-access-dqkzq\" (UniqueName: \"kubernetes.io/projected/93de7d1c-bc7a-499d-97f6-3320e2e4e948-kube-api-access-dqkzq\") pod \"93de7d1c-bc7a-499d-97f6-3320e2e4e948\" (UID: \"93de7d1c-bc7a-499d-97f6-3320e2e4e948\") "
Oct 30 19:17:07.115680 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:07.115510    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93de7d1c-bc7a-499d-97f6-3320e2e4e948-workload-socket" pod "93de7d1c-bc7a-499d-97f6-3320e2e4e948" (UID: "93de7d1c-bc7a-499d-97f6-3320e2e4e948"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:17:07.115680 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:07.115560    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/93de7d1c-bc7a-499d-97f6-3320e2e4e948-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/93de7d1c-bc7a-499d-97f6-3320e2e4e948-istio-envoy\") pod \"93de7d1c-bc7a-499d-97f6-3320e2e4e948\" (UID: \"93de7d1c-bc7a-499d-97f6-3320e2e4e948\") "
Oct 30 19:17:07.115680 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:07.115634    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/93de7d1c-bc7a-499d-97f6-3320e2e4e948-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/93de7d1c-bc7a-499d-97f6-3320e2e4e948-workload-certs\") pod \"93de7d1c-bc7a-499d-97f6-3320e2e4e948\" (UID: \"93de7d1c-bc7a-499d-97f6-3320e2e4e948\") "
Oct 30 19:17:07.115852 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:07.115687    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93de7d1c-bc7a-499d-97f6-3320e2e4e948-credential-socket" pod "93de7d1c-bc7a-499d-97f6-3320e2e4e948" (UID: "93de7d1c-bc7a-499d-97f6-3320e2e4e948"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:17:07.115852 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:07.115696    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/93de7d1c-bc7a-499d-97f6-3320e2e4e948-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/93de7d1c-bc7a-499d-97f6-3320e2e4e948-istio-data\") pod \"93de7d1c-bc7a-499d-97f6-3320e2e4e948\" (UID: \"93de7d1c-bc7a-499d-97f6-3320e2e4e948\") "
Oct 30 19:17:07.115852 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:07.115762    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/93de7d1c-bc7a-499d-97f6-3320e2e4e948-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/93de7d1c-bc7a-499d-97f6-3320e2e4e948-istiod-ca-cert\") pod \"93de7d1c-bc7a-499d-97f6-3320e2e4e948\" (UID: \"93de7d1c-bc7a-499d-97f6-3320e2e4e948\") "
Oct 30 19:17:07.115852 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:07.115827    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/93de7d1c-bc7a-499d-97f6-3320e2e4e948-istio-token\" (UniqueName: \"kubernetes.io/projected/93de7d1c-bc7a-499d-97f6-3320e2e4e948-istio-token\") pod \"93de7d1c-bc7a-499d-97f6-3320e2e4e948\" (UID: \"93de7d1c-bc7a-499d-97f6-3320e2e4e948\") "
Oct 30 19:17:07.116050 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:07.115904    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93de7d1c-bc7a-499d-97f6-3320e2e4e948-istio-data" pod "93de7d1c-bc7a-499d-97f6-3320e2e4e948" (UID: "93de7d1c-bc7a-499d-97f6-3320e2e4e948"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:17:07.116050 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:07.115936    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93de7d1c-bc7a-499d-97f6-3320e2e4e948-workload-certs" pod "93de7d1c-bc7a-499d-97f6-3320e2e4e948" (UID: "93de7d1c-bc7a-499d-97f6-3320e2e4e948"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:17:07.116396 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:07.116377    4888 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/93de7d1c-bc7a-499d-97f6-3320e2e4e948-workload-socket\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:17:07.116501 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:07.116399    4888 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/93de7d1c-bc7a-499d-97f6-3320e2e4e948-credential-socket\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:17:07.116501 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:07.116412    4888 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/93de7d1c-bc7a-499d-97f6-3320e2e4e948-workload-certs\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:17:07.116501 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:07.116423    4888 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/93de7d1c-bc7a-499d-97f6-3320e2e4e948-istio-data\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:17:07.116501 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:07.116419    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/93de7d1c-bc7a-499d-97f6-3320e2e4e948-istiod-ca-cert" pod "93de7d1c-bc7a-499d-97f6-3320e2e4e948" (UID: "93de7d1c-bc7a-499d-97f6-3320e2e4e948"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:17:07.117799 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:07.117775    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/93de7d1c-bc7a-499d-97f6-3320e2e4e948-istio-podinfo" pod "93de7d1c-bc7a-499d-97f6-3320e2e4e948" (UID: "93de7d1c-bc7a-499d-97f6-3320e2e4e948"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:17:07.117905 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:07.117814    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93de7d1c-bc7a-499d-97f6-3320e2e4e948-istio-envoy" pod "93de7d1c-bc7a-499d-97f6-3320e2e4e948" (UID: "93de7d1c-bc7a-499d-97f6-3320e2e4e948"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:17:07.118250 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:07.118236    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/93de7d1c-bc7a-499d-97f6-3320e2e4e948-kube-api-access-dqkzq" pod "93de7d1c-bc7a-499d-97f6-3320e2e4e948" (UID: "93de7d1c-bc7a-499d-97f6-3320e2e4e948"). InnerVolumeSpecName "kube-api-access-dqkzq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:17:07.118486 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:07.118463    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/93de7d1c-bc7a-499d-97f6-3320e2e4e948-istio-token" pod "93de7d1c-bc7a-499d-97f6-3320e2e4e948" (UID: "93de7d1c-bc7a-499d-97f6-3320e2e4e948"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:17:07.217270 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:07.217180    4888 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/93de7d1c-bc7a-499d-97f6-3320e2e4e948-istio-podinfo\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:17:07.217270 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:07.217204    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dqkzq\" (UniqueName: \"kubernetes.io/projected/93de7d1c-bc7a-499d-97f6-3320e2e4e948-kube-api-access-dqkzq\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:17:07.217270 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:07.217215    4888 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/93de7d1c-bc7a-499d-97f6-3320e2e4e948-istio-envoy\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:17:07.217270 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:07.217230    4888 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/93de7d1c-bc7a-499d-97f6-3320e2e4e948-istiod-ca-cert\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:17:07.217270 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:07.217242    4888 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/93de7d1c-bc7a-499d-97f6-3320e2e4e948-istio-token\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:17:07.424171 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:07.424141    4888 generic.go:356] "Generic (PLEG): container finished" podID="93de7d1c-bc7a-499d-97f6-3320e2e4e948" containerID="3e98e7afa364c552b0b15f972202160c04c2b8deae9efa1e9e19fef9b57dc4ed" exitCode=0
Oct 30 19:17:07.424389 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:07.424184    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-26cqt"
Oct 30 19:17:07.424389 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:07.424223    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-26cqt" event={"ID":"93de7d1c-bc7a-499d-97f6-3320e2e4e948","Type":"ContainerDied","Data":"3e98e7afa364c552b0b15f972202160c04c2b8deae9efa1e9e19fef9b57dc4ed"}
Oct 30 19:17:07.424389 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:07.424302    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-26cqt" event={"ID":"93de7d1c-bc7a-499d-97f6-3320e2e4e948","Type":"ContainerDied","Data":"2015051f1f4c05f64b2a3a9459b5b65cf7d87ecdb219f150cb0c5f4d95951307"}
Oct 30 19:17:07.424389 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:07.424325    4888 scope.go:122] "RemoveContainer" containerID="3e98e7afa364c552b0b15f972202160c04c2b8deae9efa1e9e19fef9b57dc4ed"
Oct 30 19:17:07.445670 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:07.445647    4888 scope.go:122] "RemoveContainer" containerID="3e98e7afa364c552b0b15f972202160c04c2b8deae9efa1e9e19fef9b57dc4ed"
Oct 30 19:17:07.446025 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:17:07.445998    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3e98e7afa364c552b0b15f972202160c04c2b8deae9efa1e9e19fef9b57dc4ed\": container with ID starting with 3e98e7afa364c552b0b15f972202160c04c2b8deae9efa1e9e19fef9b57dc4ed not found: ID does not exist" containerID="3e98e7afa364c552b0b15f972202160c04c2b8deae9efa1e9e19fef9b57dc4ed"
Oct 30 19:17:07.446071 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:07.446035    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3e98e7afa364c552b0b15f972202160c04c2b8deae9efa1e9e19fef9b57dc4ed"} err="failed to get container status \"3e98e7afa364c552b0b15f972202160c04c2b8deae9efa1e9e19fef9b57dc4ed\": rpc error: code = NotFound desc = could not find container \"3e98e7afa364c552b0b15f972202160c04c2b8deae9efa1e9e19fef9b57dc4ed\": container with ID starting with 3e98e7afa364c552b0b15f972202160c04c2b8deae9efa1e9e19fef9b57dc4ed not found: ID does not exist"
Oct 30 19:17:07.595839 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:07.595806    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-26cqt"]
Oct 30 19:17:07.647107 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:07.647073    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-26cqt"]
Oct 30 19:17:07.890426 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:07.890339    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="93de7d1c-bc7a-499d-97f6-3320e2e4e948" path="/var/lib/kubelet/pods/93de7d1c-bc7a-499d-97f6-3320e2e4e948/volumes"
Oct 30 19:17:08.994520 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:08.994485    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-jhlsk"]
Oct 30 19:17:08.995069 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:08.995055    4888 cpu_manager.go:414] "containerMap: removing container" podUID="9fe0e09a-5f21-4382-8153-542047006bb6" containerName="cleanup"
Oct 30 19:17:08.995114 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:08.995070    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9fe0e09a-5f21-4382-8153-542047006bb6" containerName="cleanup"
Oct 30 19:17:08.995114 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:08.995088    4888 cpu_manager.go:414] "containerMap: removing container" podUID="3df2a357-4896-4c99-831b-96fe4cd18fb5" containerName="istio-proxy"
Oct 30 19:17:08.995114 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:08.995094    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3df2a357-4896-4c99-831b-96fe4cd18fb5" containerName="istio-proxy"
Oct 30 19:17:08.995114 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:08.995106    4888 cpu_manager.go:414] "containerMap: removing container" podUID="93de7d1c-bc7a-499d-97f6-3320e2e4e948" containerName="istio-proxy"
Oct 30 19:17:08.995114 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:08.995112    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="93de7d1c-bc7a-499d-97f6-3320e2e4e948" containerName="istio-proxy"
Oct 30 19:17:08.995267 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:08.995132    4888 cpu_manager.go:414] "containerMap: removing container" podUID="7208215b-516e-47d6-bb61-27e1031c7953" containerName="cleanup"
Oct 30 19:17:08.995267 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:08.995137    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7208215b-516e-47d6-bb61-27e1031c7953" containerName="cleanup"
Oct 30 19:17:08.995267 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:08.995148    4888 cpu_manager.go:414] "containerMap: removing container" podUID="bbc9f851-b372-4f93-a7a8-42f67d93093c" containerName="cleanup"
Oct 30 19:17:08.995267 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:08.995153    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bbc9f851-b372-4f93-a7a8-42f67d93093c" containerName="cleanup"
Oct 30 19:17:08.995267 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:08.995251    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="3df2a357-4896-4c99-831b-96fe4cd18fb5" containerName="istio-proxy"
Oct 30 19:17:08.995267 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:08.995262    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="9fe0e09a-5f21-4382-8153-542047006bb6" containerName="cleanup"
Oct 30 19:17:08.995465 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:08.995271    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="3df2a357-4896-4c99-831b-96fe4cd18fb5" containerName="istio-proxy"
Oct 30 19:17:08.995465 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:08.995294    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="93de7d1c-bc7a-499d-97f6-3320e2e4e948" containerName="istio-proxy"
Oct 30 19:17:08.995465 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:08.995305    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="7208215b-516e-47d6-bb61-27e1031c7953" containerName="cleanup"
Oct 30 19:17:09.000235 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:09.000207    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-jhlsk"
Oct 30 19:17:09.138830 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:09.138798    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nrbcf\" (UniqueName: \"kubernetes.io/projected/b43a09c5-9f21-4ac7-99cd-25300651d3a8-kube-api-access-nrbcf\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-jhlsk\" (UID: \"b43a09c5-9f21-4ac7-99cd-25300651d3a8\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-jhlsk"
Oct 30 19:17:09.139009 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:09.138851    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b43a09c5-9f21-4ac7-99cd-25300651d3a8-workload-certs\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-jhlsk\" (UID: \"b43a09c5-9f21-4ac7-99cd-25300651d3a8\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-jhlsk"
Oct 30 19:17:09.139062 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:09.139004    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b43a09c5-9f21-4ac7-99cd-25300651d3a8-istio-token\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-jhlsk\" (UID: \"b43a09c5-9f21-4ac7-99cd-25300651d3a8\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-jhlsk"
Oct 30 19:17:09.139108 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:09.139058    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b43a09c5-9f21-4ac7-99cd-25300651d3a8-istio-envoy\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-jhlsk\" (UID: \"b43a09c5-9f21-4ac7-99cd-25300651d3a8\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-jhlsk"
Oct 30 19:17:09.139154 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:09.139124    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b43a09c5-9f21-4ac7-99cd-25300651d3a8-istiod-ca-cert\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-jhlsk\" (UID: \"b43a09c5-9f21-4ac7-99cd-25300651d3a8\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-jhlsk"
Oct 30 19:17:09.139200 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:09.139171    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b43a09c5-9f21-4ac7-99cd-25300651d3a8-credential-socket\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-jhlsk\" (UID: \"b43a09c5-9f21-4ac7-99cd-25300651d3a8\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-jhlsk"
Oct 30 19:17:09.139245 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:09.139216    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b43a09c5-9f21-4ac7-99cd-25300651d3a8-istio-podinfo\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-jhlsk\" (UID: \"b43a09c5-9f21-4ac7-99cd-25300651d3a8\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-jhlsk"
Oct 30 19:17:09.139316 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:09.139244    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b43a09c5-9f21-4ac7-99cd-25300651d3a8-istio-data\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-jhlsk\" (UID: \"b43a09c5-9f21-4ac7-99cd-25300651d3a8\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-jhlsk"
Oct 30 19:17:09.139377 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:09.139344    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b43a09c5-9f21-4ac7-99cd-25300651d3a8-workload-socket\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-jhlsk\" (UID: \"b43a09c5-9f21-4ac7-99cd-25300651d3a8\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-jhlsk"
Oct 30 19:17:09.220800 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:09.220760    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-jhlsk"]
Oct 30 19:17:09.244860 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:09.243372    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b43a09c5-9f21-4ac7-99cd-25300651d3a8-istio-token\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-jhlsk\" (UID: \"b43a09c5-9f21-4ac7-99cd-25300651d3a8\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-jhlsk"
Oct 30 19:17:09.244860 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:09.243443    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b43a09c5-9f21-4ac7-99cd-25300651d3a8-istio-envoy\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-jhlsk\" (UID: \"b43a09c5-9f21-4ac7-99cd-25300651d3a8\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-jhlsk"
Oct 30 19:17:09.244860 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:09.243477    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b43a09c5-9f21-4ac7-99cd-25300651d3a8-istiod-ca-cert\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-jhlsk\" (UID: \"b43a09c5-9f21-4ac7-99cd-25300651d3a8\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-jhlsk"
Oct 30 19:17:09.244860 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:09.243518    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b43a09c5-9f21-4ac7-99cd-25300651d3a8-credential-socket\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-jhlsk\" (UID: \"b43a09c5-9f21-4ac7-99cd-25300651d3a8\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-jhlsk"
Oct 30 19:17:09.244860 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:09.243593    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b43a09c5-9f21-4ac7-99cd-25300651d3a8-istio-podinfo\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-jhlsk\" (UID: \"b43a09c5-9f21-4ac7-99cd-25300651d3a8\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-jhlsk"
Oct 30 19:17:09.244860 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:09.243637    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b43a09c5-9f21-4ac7-99cd-25300651d3a8-istio-data\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-jhlsk\" (UID: \"b43a09c5-9f21-4ac7-99cd-25300651d3a8\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-jhlsk"
Oct 30 19:17:09.244860 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:09.243676    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b43a09c5-9f21-4ac7-99cd-25300651d3a8-workload-socket\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-jhlsk\" (UID: \"b43a09c5-9f21-4ac7-99cd-25300651d3a8\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-jhlsk"
Oct 30 19:17:09.244860 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:09.243740    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nrbcf\" (UniqueName: \"kubernetes.io/projected/b43a09c5-9f21-4ac7-99cd-25300651d3a8-kube-api-access-nrbcf\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-jhlsk\" (UID: \"b43a09c5-9f21-4ac7-99cd-25300651d3a8\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-jhlsk"
Oct 30 19:17:09.244860 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:09.243804    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b43a09c5-9f21-4ac7-99cd-25300651d3a8-workload-certs\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-jhlsk\" (UID: \"b43a09c5-9f21-4ac7-99cd-25300651d3a8\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-jhlsk"
Oct 30 19:17:09.244860 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:09.244316    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b43a09c5-9f21-4ac7-99cd-25300651d3a8-workload-certs\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-jhlsk\" (UID: \"b43a09c5-9f21-4ac7-99cd-25300651d3a8\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-jhlsk"
Oct 30 19:17:09.245860 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:09.245791    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b43a09c5-9f21-4ac7-99cd-25300651d3a8-istiod-ca-cert\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-jhlsk\" (UID: \"b43a09c5-9f21-4ac7-99cd-25300651d3a8\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-jhlsk"
Oct 30 19:17:09.250229 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:09.247810    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b43a09c5-9f21-4ac7-99cd-25300651d3a8-credential-socket\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-jhlsk\" (UID: \"b43a09c5-9f21-4ac7-99cd-25300651d3a8\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-jhlsk"
Oct 30 19:17:09.250229 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:09.248054    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b43a09c5-9f21-4ac7-99cd-25300651d3a8-workload-socket\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-jhlsk\" (UID: \"b43a09c5-9f21-4ac7-99cd-25300651d3a8\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-jhlsk"
Oct 30 19:17:09.250229 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:09.248273    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b43a09c5-9f21-4ac7-99cd-25300651d3a8-istio-data\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-jhlsk\" (UID: \"b43a09c5-9f21-4ac7-99cd-25300651d3a8\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-jhlsk"
Oct 30 19:17:09.263278 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:09.263250    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b43a09c5-9f21-4ac7-99cd-25300651d3a8-istio-envoy\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-jhlsk\" (UID: \"b43a09c5-9f21-4ac7-99cd-25300651d3a8\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-jhlsk"
Oct 30 19:17:09.263926 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:09.263908    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b43a09c5-9f21-4ac7-99cd-25300651d3a8-istio-podinfo\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-jhlsk\" (UID: \"b43a09c5-9f21-4ac7-99cd-25300651d3a8\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-jhlsk"
Oct 30 19:17:09.307436 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:09.307412    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nrbcf\" (UniqueName: \"kubernetes.io/projected/b43a09c5-9f21-4ac7-99cd-25300651d3a8-kube-api-access-nrbcf\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-jhlsk\" (UID: \"b43a09c5-9f21-4ac7-99cd-25300651d3a8\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-jhlsk"
Oct 30 19:17:09.307436 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:09.307427    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b43a09c5-9f21-4ac7-99cd-25300651d3a8-istio-token\") pod \"e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-jhlsk\" (UID: \"b43a09c5-9f21-4ac7-99cd-25300651d3a8\") " pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-jhlsk"
Oct 30 19:17:09.314750 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:09.314717    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-jhlsk"
Oct 30 19:17:09.635193 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:09.635168    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-a-f554bb-openshift-default-5f8f5599d5-x2l7h"
Oct 30 19:17:09.750313 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:09.750271    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0c0bee10-4345-43a8-9008-b60054dc6850-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0c0bee10-4345-43a8-9008-b60054dc6850-workload-certs\") pod \"0c0bee10-4345-43a8-9008-b60054dc6850\" (UID: \"0c0bee10-4345-43a8-9008-b60054dc6850\") "
Oct 30 19:17:09.750516 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:09.750367    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0c0bee10-4345-43a8-9008-b60054dc6850-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0c0bee10-4345-43a8-9008-b60054dc6850-istio-envoy\") pod \"0c0bee10-4345-43a8-9008-b60054dc6850\" (UID: \"0c0bee10-4345-43a8-9008-b60054dc6850\") "
Oct 30 19:17:09.750516 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:09.750440    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0c0bee10-4345-43a8-9008-b60054dc6850-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c0bee10-4345-43a8-9008-b60054dc6850-credential-socket\") pod \"0c0bee10-4345-43a8-9008-b60054dc6850\" (UID: \"0c0bee10-4345-43a8-9008-b60054dc6850\") "
Oct 30 19:17:09.750516 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:09.750485    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0c0bee10-4345-43a8-9008-b60054dc6850-istio-token\" (UniqueName: \"kubernetes.io/projected/0c0bee10-4345-43a8-9008-b60054dc6850-istio-token\") pod \"0c0bee10-4345-43a8-9008-b60054dc6850\" (UID: \"0c0bee10-4345-43a8-9008-b60054dc6850\") "
Oct 30 19:17:09.750706 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:09.750522    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0c0bee10-4345-43a8-9008-b60054dc6850-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c0bee10-4345-43a8-9008-b60054dc6850-workload-socket\") pod \"0c0bee10-4345-43a8-9008-b60054dc6850\" (UID: \"0c0bee10-4345-43a8-9008-b60054dc6850\") "
Oct 30 19:17:09.750706 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:09.750577    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/0c0bee10-4345-43a8-9008-b60054dc6850-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0c0bee10-4345-43a8-9008-b60054dc6850-istio-podinfo\") pod \"0c0bee10-4345-43a8-9008-b60054dc6850\" (UID: \"0c0bee10-4345-43a8-9008-b60054dc6850\") "
Oct 30 19:17:09.750706 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:09.750615    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c0bee10-4345-43a8-9008-b60054dc6850-workload-certs" pod "0c0bee10-4345-43a8-9008-b60054dc6850" (UID: "0c0bee10-4345-43a8-9008-b60054dc6850"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:17:09.750706 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:09.750636    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0c0bee10-4345-43a8-9008-b60054dc6850-kube-api-access-286jf\" (UniqueName: \"kubernetes.io/projected/0c0bee10-4345-43a8-9008-b60054dc6850-kube-api-access-286jf\") pod \"0c0bee10-4345-43a8-9008-b60054dc6850\" (UID: \"0c0bee10-4345-43a8-9008-b60054dc6850\") "
Oct 30 19:17:09.750932 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:09.750760    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0c0bee10-4345-43a8-9008-b60054dc6850-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0c0bee10-4345-43a8-9008-b60054dc6850-istio-data\") pod \"0c0bee10-4345-43a8-9008-b60054dc6850\" (UID: \"0c0bee10-4345-43a8-9008-b60054dc6850\") "
Oct 30 19:17:09.750932 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:09.750794    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0c0bee10-4345-43a8-9008-b60054dc6850-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0c0bee10-4345-43a8-9008-b60054dc6850-istiod-ca-cert\") pod \"0c0bee10-4345-43a8-9008-b60054dc6850\" (UID: \"0c0bee10-4345-43a8-9008-b60054dc6850\") "
Oct 30 19:17:09.750932 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:09.750782    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c0bee10-4345-43a8-9008-b60054dc6850-credential-socket" pod "0c0bee10-4345-43a8-9008-b60054dc6850" (UID: "0c0bee10-4345-43a8-9008-b60054dc6850"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:17:09.750932 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:09.750850    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c0bee10-4345-43a8-9008-b60054dc6850-workload-socket" pod "0c0bee10-4345-43a8-9008-b60054dc6850" (UID: "0c0bee10-4345-43a8-9008-b60054dc6850"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:17:09.751397 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:09.751373    4888 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c0bee10-4345-43a8-9008-b60054dc6850-workload-socket\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:17:09.751533 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:09.751400    4888 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0c0bee10-4345-43a8-9008-b60054dc6850-workload-certs\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:17:09.751533 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:09.751416    4888 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c0bee10-4345-43a8-9008-b60054dc6850-credential-socket\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:17:09.751533 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:09.751412    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c0bee10-4345-43a8-9008-b60054dc6850-istio-data" pod "0c0bee10-4345-43a8-9008-b60054dc6850" (UID: "0c0bee10-4345-43a8-9008-b60054dc6850"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:17:09.751902 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:09.751881    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-jhlsk"]
Oct 30 19:17:09.752010 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:09.751879    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0c0bee10-4345-43a8-9008-b60054dc6850-istiod-ca-cert" pod "0c0bee10-4345-43a8-9008-b60054dc6850" (UID: "0c0bee10-4345-43a8-9008-b60054dc6850"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:17:09.753375 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:09.753347    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0c0bee10-4345-43a8-9008-b60054dc6850-kube-api-access-286jf" pod "0c0bee10-4345-43a8-9008-b60054dc6850" (UID: "0c0bee10-4345-43a8-9008-b60054dc6850"). InnerVolumeSpecName "kube-api-access-286jf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:17:09.753647 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:09.753623    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c0bee10-4345-43a8-9008-b60054dc6850-istio-envoy" pod "0c0bee10-4345-43a8-9008-b60054dc6850" (UID: "0c0bee10-4345-43a8-9008-b60054dc6850"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:17:09.753721 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:09.753698    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0c0bee10-4345-43a8-9008-b60054dc6850-istio-token" pod "0c0bee10-4345-43a8-9008-b60054dc6850" (UID: "0c0bee10-4345-43a8-9008-b60054dc6850"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:17:09.753721 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:09.753704    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0c0bee10-4345-43a8-9008-b60054dc6850-istio-podinfo" pod "0c0bee10-4345-43a8-9008-b60054dc6850" (UID: "0c0bee10-4345-43a8-9008-b60054dc6850"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:17:09.792389 ip-10-0-134-46 kubenswrapper[4888]: W1030 19:17:09.792363    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb43a09c5_9f21_4ac7_99cd_25300651d3a8.slice/crio-6260fe7adcd519aca28cb285325df36753e0a9cfa1012417c6428d675647bd26 WatchSource:0}: Error finding container 6260fe7adcd519aca28cb285325df36753e0a9cfa1012417c6428d675647bd26: Status 404 returned error can't find the container with id 6260fe7adcd519aca28cb285325df36753e0a9cfa1012417c6428d675647bd26
Oct 30 19:17:09.795071 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:09.795040    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:17:09.795148 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:09.795111    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:17:09.795148 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:09.795141    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:17:09.854222 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:09.854199    4888 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0c0bee10-4345-43a8-9008-b60054dc6850-istio-data\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:17:09.854407 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:09.854225    4888 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0c0bee10-4345-43a8-9008-b60054dc6850-istiod-ca-cert\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:17:09.854407 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:09.854241    4888 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0c0bee10-4345-43a8-9008-b60054dc6850-istio-envoy\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:17:09.854407 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:09.854252    4888 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0c0bee10-4345-43a8-9008-b60054dc6850-istio-token\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:17:09.854407 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:09.854260    4888 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0c0bee10-4345-43a8-9008-b60054dc6850-istio-podinfo\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:17:09.854407 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:09.854270    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-286jf\" (UniqueName: \"kubernetes.io/projected/0c0bee10-4345-43a8-9008-b60054dc6850-kube-api-access-286jf\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:17:10.449997 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:10.449967    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-jhlsk" event={"ID":"b43a09c5-9f21-4ac7-99cd-25300651d3a8","Type":"ContainerStarted","Data":"e3cbb8fa9ae750d9f607caadb0d4953bd778a6e860f96e68cbba4d4279b69d95"}
Oct 30 19:17:10.450449 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:10.450025    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-jhlsk" event={"ID":"b43a09c5-9f21-4ac7-99cd-25300651d3a8","Type":"ContainerStarted","Data":"6260fe7adcd519aca28cb285325df36753e0a9cfa1012417c6428d675647bd26"}
Oct 30 19:17:10.451758 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:10.451732    4888 generic.go:356] "Generic (PLEG): container finished" podID="0c0bee10-4345-43a8-9008-b60054dc6850" containerID="4096fb8ce41b1b43297c7e16718d23dd852fa2009a7420f4e5a9d190da9f1843" exitCode=0
Oct 30 19:17:10.451856 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:10.451774    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-a-f554bb-openshift-default-5f8f5599d5-x2l7h"
Oct 30 19:17:10.451856 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:10.451779    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-f554bb-openshift-default-5f8f5599d5-x2l7h" event={"ID":"0c0bee10-4345-43a8-9008-b60054dc6850","Type":"ContainerDied","Data":"4096fb8ce41b1b43297c7e16718d23dd852fa2009a7420f4e5a9d190da9f1843"}
Oct 30 19:17:10.451856 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:10.451819    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-f554bb-openshift-default-5f8f5599d5-x2l7h" event={"ID":"0c0bee10-4345-43a8-9008-b60054dc6850","Type":"ContainerDied","Data":"19afd348df2f13fa7eac18f7dfae5f8dace8bb663d5ad625584285a564fd32fa"}
Oct 30 19:17:10.451856 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:10.451842    4888 scope.go:122] "RemoveContainer" containerID="4096fb8ce41b1b43297c7e16718d23dd852fa2009a7420f4e5a9d190da9f1843"
Oct 30 19:17:10.461841 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:10.461825    4888 scope.go:122] "RemoveContainer" containerID="7c68227f98c1f0fb0f8bdd8e71b946c750cb37fc4c933852dba832d92a90c7e1"
Oct 30 19:17:10.495474 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:10.495450    4888 scope.go:122] "RemoveContainer" containerID="4096fb8ce41b1b43297c7e16718d23dd852fa2009a7420f4e5a9d190da9f1843"
Oct 30 19:17:10.495879 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:17:10.495853    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4096fb8ce41b1b43297c7e16718d23dd852fa2009a7420f4e5a9d190da9f1843\": container with ID starting with 4096fb8ce41b1b43297c7e16718d23dd852fa2009a7420f4e5a9d190da9f1843 not found: ID does not exist" containerID="4096fb8ce41b1b43297c7e16718d23dd852fa2009a7420f4e5a9d190da9f1843"
Oct 30 19:17:10.495989 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:10.495891    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4096fb8ce41b1b43297c7e16718d23dd852fa2009a7420f4e5a9d190da9f1843"} err="failed to get container status \"4096fb8ce41b1b43297c7e16718d23dd852fa2009a7420f4e5a9d190da9f1843\": rpc error: code = NotFound desc = could not find container \"4096fb8ce41b1b43297c7e16718d23dd852fa2009a7420f4e5a9d190da9f1843\": container with ID starting with 4096fb8ce41b1b43297c7e16718d23dd852fa2009a7420f4e5a9d190da9f1843 not found: ID does not exist"
Oct 30 19:17:10.495989 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:10.495913    4888 scope.go:122] "RemoveContainer" containerID="7c68227f98c1f0fb0f8bdd8e71b946c750cb37fc4c933852dba832d92a90c7e1"
Oct 30 19:17:10.496219 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:17:10.496190    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7c68227f98c1f0fb0f8bdd8e71b946c750cb37fc4c933852dba832d92a90c7e1\": container with ID starting with 7c68227f98c1f0fb0f8bdd8e71b946c750cb37fc4c933852dba832d92a90c7e1 not found: ID does not exist" containerID="7c68227f98c1f0fb0f8bdd8e71b946c750cb37fc4c933852dba832d92a90c7e1"
Oct 30 19:17:10.496276 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:10.496230    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7c68227f98c1f0fb0f8bdd8e71b946c750cb37fc4c933852dba832d92a90c7e1"} err="failed to get container status \"7c68227f98c1f0fb0f8bdd8e71b946c750cb37fc4c933852dba832d92a90c7e1\": rpc error: code = NotFound desc = could not find container \"7c68227f98c1f0fb0f8bdd8e71b946c750cb37fc4c933852dba832d92a90c7e1\": container with ID starting with 7c68227f98c1f0fb0f8bdd8e71b946c750cb37fc4c933852dba832d92a90c7e1 not found: ID does not exist"
Oct 30 19:17:10.606071 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:10.600109    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-jhlsk" podStartSLOduration=2.600089721 podStartE2EDuration="2.600089721s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:17:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:17:10.58461093 +0000 UTC m=+2594.832016953" watchObservedRunningTime="2026-10-30 19:17:10.600089721 +0000 UTC m=+2594.847495744"
Oct 30 19:17:10.705798 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:10.705721    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-praxis-a-f554bb-openshift-default-5f8f5599d5-x2l7h"]
Oct 30 19:17:10.712650 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:10.712623    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-praxis-a-f554bb-openshift-default-5f8f5599d5-x2l7h"]
Oct 30 19:17:11.315117 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:11.315089    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-jhlsk"
Oct 30 19:17:11.316848 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:11.316822    4888 patch_prober.go:29] interesting pod/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-jhlsk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.135:15021/healthz/ready\": dial tcp 10.132.0.135:15021: connect: connection refused" start-of-body=
Oct 30 19:17:11.316996 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:11.316882    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-jhlsk" podUID="b43a09c5-9f21-4ac7-99cd-25300651d3a8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.135:15021/healthz/ready\": dial tcp 10.132.0.135:15021: connect: connection refused"
Oct 30 19:17:11.905083 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:11.905048    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0c0bee10-4345-43a8-9008-b60054dc6850" path="/var/lib/kubelet/pods/0c0bee10-4345-43a8-9008-b60054dc6850/volumes"
Oct 30 19:17:12.315684 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:12.315596    4888 patch_prober.go:29] interesting pod/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-jhlsk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.135:15021/healthz/ready\": dial tcp 10.132.0.135:15021: connect: connection refused" start-of-body=
Oct 30 19:17:12.315684 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:12.315653    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-jhlsk" podUID="b43a09c5-9f21-4ac7-99cd-25300651d3a8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.135:15021/healthz/ready\": dial tcp 10.132.0.135:15021: connect: connection refused"
Oct 30 19:17:13.315252 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:13.315215    4888 patch_prober.go:29] interesting pod/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-jhlsk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.135:15021/healthz/ready\": dial tcp 10.132.0.135:15021: connect: connection refused" start-of-body=
Oct 30 19:17:13.315689 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:13.315295    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-jhlsk" podUID="b43a09c5-9f21-4ac7-99cd-25300651d3a8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.135:15021/healthz/ready\": dial tcp 10.132.0.135:15021: connect: connection refused"
Oct 30 19:17:14.315263 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:14.315229    4888 patch_prober.go:29] interesting pod/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-jhlsk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.135:15021/healthz/ready\": dial tcp 10.132.0.135:15021: connect: connection refused" start-of-body=
Oct 30 19:17:14.315692 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:14.315301    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-jhlsk" podUID="b43a09c5-9f21-4ac7-99cd-25300651d3a8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.135:15021/healthz/ready\": dial tcp 10.132.0.135:15021: connect: connection refused"
Oct 30 19:17:15.315362 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:15.315324    4888 patch_prober.go:29] interesting pod/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-jhlsk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.135:15021/healthz/ready\": dial tcp 10.132.0.135:15021: connect: connection refused" start-of-body=
Oct 30 19:17:15.315775 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:15.315400    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-jhlsk" podUID="b43a09c5-9f21-4ac7-99cd-25300651d3a8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.135:15021/healthz/ready\": dial tcp 10.132.0.135:15021: connect: connection refused"
Oct 30 19:17:16.316027 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:16.315983    4888 patch_prober.go:29] interesting pod/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-jhlsk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.135:15021/healthz/ready\": dial tcp 10.132.0.135:15021: connect: connection refused" start-of-body=
Oct 30 19:17:16.316449 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:16.316058    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-jhlsk" podUID="b43a09c5-9f21-4ac7-99cd-25300651d3a8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.135:15021/healthz/ready\": dial tcp 10.132.0.135:15021: connect: connection refused"
Oct 30 19:17:17.320263 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:17.320235    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-jhlsk"
Oct 30 19:17:17.320730 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:17.320698    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-jhlsk"
Oct 30 19:17:17.321561 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:17.321543    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-jhlsk"
Oct 30 19:17:17.553268 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:17.553242    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-b-6d96a6-7447cf6d8b-zwp8g"]
Oct 30 19:17:17.674588 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:17.674552    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-b-6d96a6-66f76bf9b7-rfhcw"]
Oct 30 19:17:22.554530 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:22.554470    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-praxis-b-6d96a6-7447cf6d8b-zwp8g" podUID="30332320-0fa8-4c94-ba9f-42cbacfdd0d9" containerName="payload-processing" containerID="cri-o://5e689a83fe59077292f3b6f1fa1dede7908656ecf6af62c3ec4c5086aaf14b85" gracePeriod=25
Oct 30 19:17:22.675228 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:22.675153    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-6d96a6-66f76bf9b7-rfhcw" podUID="b62e784d-c5de-4e4d-ba8a-96c955b1a431" containerName="payload-pre-processing" containerID="cri-o://c78c980ef3720093be418d943a91869a25ab1e9d26bade5492dc8afcde18d958" gracePeriod=25
Oct 30 19:17:22.863477 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:22.863450    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-praxis-b-6d96a6-7447cf6d8b-zwp8g"
Oct 30 19:17:22.957846 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:22.957809    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/30332320-0fa8-4c94-ba9f-42cbacfdd0d9-kube-api-access-lrtp7\" (UniqueName: \"kubernetes.io/projected/30332320-0fa8-4c94-ba9f-42cbacfdd0d9-kube-api-access-lrtp7\") pod \"30332320-0fa8-4c94-ba9f-42cbacfdd0d9\" (UID: \"30332320-0fa8-4c94-ba9f-42cbacfdd0d9\") "
Oct 30 19:17:22.958027 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:22.957886    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/30332320-0fa8-4c94-ba9f-42cbacfdd0d9-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/30332320-0fa8-4c94-ba9f-42cbacfdd0d9-plugins-config-volume\") pod \"30332320-0fa8-4c94-ba9f-42cbacfdd0d9\" (UID: \"30332320-0fa8-4c94-ba9f-42cbacfdd0d9\") "
Oct 30 19:17:22.958631 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:22.958600    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/30332320-0fa8-4c94-ba9f-42cbacfdd0d9-plugins-config-volume" pod "30332320-0fa8-4c94-ba9f-42cbacfdd0d9" (UID: "30332320-0fa8-4c94-ba9f-42cbacfdd0d9"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:17:22.960677 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:22.960649    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/30332320-0fa8-4c94-ba9f-42cbacfdd0d9-kube-api-access-lrtp7" pod "30332320-0fa8-4c94-ba9f-42cbacfdd0d9" (UID: "30332320-0fa8-4c94-ba9f-42cbacfdd0d9"). InnerVolumeSpecName "kube-api-access-lrtp7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:17:22.986693 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:22.986669    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-6d96a6-66f76bf9b7-rfhcw"
Oct 30 19:17:23.059147 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:23.059116    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b62e784d-c5de-4e4d-ba8a-96c955b1a431-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b62e784d-c5de-4e4d-ba8a-96c955b1a431-plugins-config-volume\") pod \"b62e784d-c5de-4e4d-ba8a-96c955b1a431\" (UID: \"b62e784d-c5de-4e4d-ba8a-96c955b1a431\") "
Oct 30 19:17:23.059331 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:23.059196    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b62e784d-c5de-4e4d-ba8a-96c955b1a431-kube-api-access-b28nl\" (UniqueName: \"kubernetes.io/projected/b62e784d-c5de-4e4d-ba8a-96c955b1a431-kube-api-access-b28nl\") pod \"b62e784d-c5de-4e4d-ba8a-96c955b1a431\" (UID: \"b62e784d-c5de-4e4d-ba8a-96c955b1a431\") "
Oct 30 19:17:23.059527 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:23.059505    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b62e784d-c5de-4e4d-ba8a-96c955b1a431-plugins-config-volume" pod "b62e784d-c5de-4e4d-ba8a-96c955b1a431" (UID: "b62e784d-c5de-4e4d-ba8a-96c955b1a431"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:17:23.059573 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:23.059560    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lrtp7\" (UniqueName: \"kubernetes.io/projected/30332320-0fa8-4c94-ba9f-42cbacfdd0d9-kube-api-access-lrtp7\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:17:23.059622 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:23.059577    4888 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/30332320-0fa8-4c94-ba9f-42cbacfdd0d9-plugins-config-volume\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:17:23.061398 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:23.061378    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b62e784d-c5de-4e4d-ba8a-96c955b1a431-kube-api-access-b28nl" pod "b62e784d-c5de-4e4d-ba8a-96c955b1a431" (UID: "b62e784d-c5de-4e4d-ba8a-96c955b1a431"). InnerVolumeSpecName "kube-api-access-b28nl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:17:23.161443 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:23.161365    4888 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b62e784d-c5de-4e4d-ba8a-96c955b1a431-plugins-config-volume\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:17:23.161443 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:23.161401    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b28nl\" (UniqueName: \"kubernetes.io/projected/b62e784d-c5de-4e4d-ba8a-96c955b1a431-kube-api-access-b28nl\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:17:23.619349 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:23.619277    4888 generic.go:356] "Generic (PLEG): container finished" podID="30332320-0fa8-4c94-ba9f-42cbacfdd0d9" containerID="5e689a83fe59077292f3b6f1fa1dede7908656ecf6af62c3ec4c5086aaf14b85" exitCode=0
Oct 30 19:17:23.619862 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:23.619360    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-praxis-b-6d96a6-7447cf6d8b-zwp8g"
Oct 30 19:17:23.619862 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:23.619375    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-praxis-b-6d96a6-7447cf6d8b-zwp8g" event={"ID":"30332320-0fa8-4c94-ba9f-42cbacfdd0d9","Type":"ContainerDied","Data":"5e689a83fe59077292f3b6f1fa1dede7908656ecf6af62c3ec4c5086aaf14b85"}
Oct 30 19:17:23.619862 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:23.619434    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-praxis-b-6d96a6-7447cf6d8b-zwp8g" event={"ID":"30332320-0fa8-4c94-ba9f-42cbacfdd0d9","Type":"ContainerDied","Data":"120ee6d532be653ad7bef6a7954460cf0fa597cd71a43bbc56a7628cded17b5c"}
Oct 30 19:17:23.619862 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:23.619464    4888 scope.go:122] "RemoveContainer" containerID="5e689a83fe59077292f3b6f1fa1dede7908656ecf6af62c3ec4c5086aaf14b85"
Oct 30 19:17:23.622825 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:23.621723    4888 generic.go:356] "Generic (PLEG): container finished" podID="b62e784d-c5de-4e4d-ba8a-96c955b1a431" containerID="c78c980ef3720093be418d943a91869a25ab1e9d26bade5492dc8afcde18d958" exitCode=0
Oct 30 19:17:23.622825 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:23.621804    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-6d96a6-66f76bf9b7-rfhcw" event={"ID":"b62e784d-c5de-4e4d-ba8a-96c955b1a431","Type":"ContainerDied","Data":"c78c980ef3720093be418d943a91869a25ab1e9d26bade5492dc8afcde18d958"}
Oct 30 19:17:23.622825 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:23.621867    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-6d96a6-66f76bf9b7-rfhcw" event={"ID":"b62e784d-c5de-4e4d-ba8a-96c955b1a431","Type":"ContainerDied","Data":"a633cfd3a4b5fd453e62d9ae1a6d772725ed6134bf4ea5ed290d7e21523bf635"}
Oct 30 19:17:23.622825 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:23.621949    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-6d96a6-66f76bf9b7-rfhcw"
Oct 30 19:17:23.637785 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:23.637765    4888 scope.go:122] "RemoveContainer" containerID="5e689a83fe59077292f3b6f1fa1dede7908656ecf6af62c3ec4c5086aaf14b85"
Oct 30 19:17:23.638137 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:17:23.638118    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5e689a83fe59077292f3b6f1fa1dede7908656ecf6af62c3ec4c5086aaf14b85\": container with ID starting with 5e689a83fe59077292f3b6f1fa1dede7908656ecf6af62c3ec4c5086aaf14b85 not found: ID does not exist" containerID="5e689a83fe59077292f3b6f1fa1dede7908656ecf6af62c3ec4c5086aaf14b85"
Oct 30 19:17:23.638190 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:23.638147    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5e689a83fe59077292f3b6f1fa1dede7908656ecf6af62c3ec4c5086aaf14b85"} err="failed to get container status \"5e689a83fe59077292f3b6f1fa1dede7908656ecf6af62c3ec4c5086aaf14b85\": rpc error: code = NotFound desc = could not find container \"5e689a83fe59077292f3b6f1fa1dede7908656ecf6af62c3ec4c5086aaf14b85\": container with ID starting with 5e689a83fe59077292f3b6f1fa1dede7908656ecf6af62c3ec4c5086aaf14b85 not found: ID does not exist"
Oct 30 19:17:23.638190 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:23.638166    4888 scope.go:122] "RemoveContainer" containerID="c78c980ef3720093be418d943a91869a25ab1e9d26bade5492dc8afcde18d958"
Oct 30 19:17:23.650058 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:23.650031    4888 scope.go:122] "RemoveContainer" containerID="c78c980ef3720093be418d943a91869a25ab1e9d26bade5492dc8afcde18d958"
Oct 30 19:17:23.650417 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:17:23.650393    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c78c980ef3720093be418d943a91869a25ab1e9d26bade5492dc8afcde18d958\": container with ID starting with c78c980ef3720093be418d943a91869a25ab1e9d26bade5492dc8afcde18d958 not found: ID does not exist" containerID="c78c980ef3720093be418d943a91869a25ab1e9d26bade5492dc8afcde18d958"
Oct 30 19:17:23.650533 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:23.650429    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c78c980ef3720093be418d943a91869a25ab1e9d26bade5492dc8afcde18d958"} err="failed to get container status \"c78c980ef3720093be418d943a91869a25ab1e9d26bade5492dc8afcde18d958\": rpc error: code = NotFound desc = could not find container \"c78c980ef3720093be418d943a91869a25ab1e9d26bade5492dc8afcde18d958\": container with ID starting with c78c980ef3720093be418d943a91869a25ab1e9d26bade5492dc8afcde18d958 not found: ID does not exist"
Oct 30 19:17:23.790367 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:23.790332    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-b-6d96a6-66f76bf9b7-rfhcw"]
Oct 30 19:17:23.793407 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:23.793381    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-b-6d96a6-66f76bf9b7-rfhcw"]
Oct 30 19:17:23.893245 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:23.893210    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b62e784d-c5de-4e4d-ba8a-96c955b1a431" path="/var/lib/kubelet/pods/b62e784d-c5de-4e4d-ba8a-96c955b1a431/volumes"
Oct 30 19:17:23.906779 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:23.906752    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-b-6d96a6-7447cf6d8b-zwp8g"]
Oct 30 19:17:23.910542 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:23.910520    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-b-6d96a6-7447cf6d8b-zwp8g"]
Oct 30 19:17:25.142804 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:25.142767    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-rjmdz"]
Oct 30 19:17:25.586508 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:25.586455    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-praxis-a-6d953c-openshift-default-5d96c8ccc7-5bjbr"]
Oct 30 19:17:25.587369 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:25.587346    4888 cpu_manager.go:414] "containerMap: removing container" podUID="0c0bee10-4345-43a8-9008-b60054dc6850" containerName="istio-proxy"
Oct 30 19:17:25.587369 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:25.587371    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0c0bee10-4345-43a8-9008-b60054dc6850" containerName="istio-proxy"
Oct 30 19:17:25.587560 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:25.587387    4888 cpu_manager.go:414] "containerMap: removing container" podUID="30332320-0fa8-4c94-ba9f-42cbacfdd0d9" containerName="payload-processing"
Oct 30 19:17:25.587560 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:25.587395    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="30332320-0fa8-4c94-ba9f-42cbacfdd0d9" containerName="payload-processing"
Oct 30 19:17:25.587560 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:25.587447    4888 cpu_manager.go:414] "containerMap: removing container" podUID="b62e784d-c5de-4e4d-ba8a-96c955b1a431" containerName="payload-pre-processing"
Oct 30 19:17:25.587560 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:25.587456    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b62e784d-c5de-4e4d-ba8a-96c955b1a431" containerName="payload-pre-processing"
Oct 30 19:17:25.587560 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:25.587474    4888 cpu_manager.go:414] "containerMap: removing container" podUID="0c0bee10-4345-43a8-9008-b60054dc6850" containerName="istio-proxy"
Oct 30 19:17:25.587560 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:25.587482    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0c0bee10-4345-43a8-9008-b60054dc6850" containerName="istio-proxy"
Oct 30 19:17:25.587560 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:25.587494    4888 cpu_manager.go:414] "containerMap: removing container" podUID="3df2a357-4896-4c99-831b-96fe4cd18fb5" containerName="istio-proxy"
Oct 30 19:17:25.587560 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:25.587502    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3df2a357-4896-4c99-831b-96fe4cd18fb5" containerName="istio-proxy"
Oct 30 19:17:25.588009 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:25.587670    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="0c0bee10-4345-43a8-9008-b60054dc6850" containerName="istio-proxy"
Oct 30 19:17:25.588009 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:25.587686    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="30332320-0fa8-4c94-ba9f-42cbacfdd0d9" containerName="payload-processing"
Oct 30 19:17:25.588009 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:25.587705    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="b62e784d-c5de-4e4d-ba8a-96c955b1a431" containerName="payload-pre-processing"
Oct 30 19:17:25.600702 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:25.600669    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-a-6d953c-openshift-default-5d96c8ccc7-5bjbr"
Oct 30 19:17:25.662309 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:25.662259    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-praxis-a-6d953c-openshift-default-5d96c8ccc7-5bjbr"]
Oct 30 19:17:25.703328 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:25.700499    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/243ebea3-fb15-455c-8e02-f152b478d25a-istio-envoy\") pod \"e2e-praxis-a-6d953c-openshift-default-5d96c8ccc7-5bjbr\" (UID: \"243ebea3-fb15-455c-8e02-f152b478d25a\") " pod="openshift-ingress/e2e-praxis-a-6d953c-openshift-default-5d96c8ccc7-5bjbr"
Oct 30 19:17:25.703328 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:25.700602    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/243ebea3-fb15-455c-8e02-f152b478d25a-istio-token\") pod \"e2e-praxis-a-6d953c-openshift-default-5d96c8ccc7-5bjbr\" (UID: \"243ebea3-fb15-455c-8e02-f152b478d25a\") " pod="openshift-ingress/e2e-praxis-a-6d953c-openshift-default-5d96c8ccc7-5bjbr"
Oct 30 19:17:25.703328 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:25.700651    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/243ebea3-fb15-455c-8e02-f152b478d25a-workload-certs\") pod \"e2e-praxis-a-6d953c-openshift-default-5d96c8ccc7-5bjbr\" (UID: \"243ebea3-fb15-455c-8e02-f152b478d25a\") " pod="openshift-ingress/e2e-praxis-a-6d953c-openshift-default-5d96c8ccc7-5bjbr"
Oct 30 19:17:25.703328 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:25.700690    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hvtvd\" (UniqueName: \"kubernetes.io/projected/243ebea3-fb15-455c-8e02-f152b478d25a-kube-api-access-hvtvd\") pod \"e2e-praxis-a-6d953c-openshift-default-5d96c8ccc7-5bjbr\" (UID: \"243ebea3-fb15-455c-8e02-f152b478d25a\") " pod="openshift-ingress/e2e-praxis-a-6d953c-openshift-default-5d96c8ccc7-5bjbr"
Oct 30 19:17:25.703328 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:25.700718    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/243ebea3-fb15-455c-8e02-f152b478d25a-istiod-ca-cert\") pod \"e2e-praxis-a-6d953c-openshift-default-5d96c8ccc7-5bjbr\" (UID: \"243ebea3-fb15-455c-8e02-f152b478d25a\") " pod="openshift-ingress/e2e-praxis-a-6d953c-openshift-default-5d96c8ccc7-5bjbr"
Oct 30 19:17:25.703328 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:25.700792    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/243ebea3-fb15-455c-8e02-f152b478d25a-credential-socket\") pod \"e2e-praxis-a-6d953c-openshift-default-5d96c8ccc7-5bjbr\" (UID: \"243ebea3-fb15-455c-8e02-f152b478d25a\") " pod="openshift-ingress/e2e-praxis-a-6d953c-openshift-default-5d96c8ccc7-5bjbr"
Oct 30 19:17:25.703328 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:25.700829    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/243ebea3-fb15-455c-8e02-f152b478d25a-workload-socket\") pod \"e2e-praxis-a-6d953c-openshift-default-5d96c8ccc7-5bjbr\" (UID: \"243ebea3-fb15-455c-8e02-f152b478d25a\") " pod="openshift-ingress/e2e-praxis-a-6d953c-openshift-default-5d96c8ccc7-5bjbr"
Oct 30 19:17:25.703328 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:25.700854    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/243ebea3-fb15-455c-8e02-f152b478d25a-istio-data\") pod \"e2e-praxis-a-6d953c-openshift-default-5d96c8ccc7-5bjbr\" (UID: \"243ebea3-fb15-455c-8e02-f152b478d25a\") " pod="openshift-ingress/e2e-praxis-a-6d953c-openshift-default-5d96c8ccc7-5bjbr"
Oct 30 19:17:25.703328 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:25.700887    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/243ebea3-fb15-455c-8e02-f152b478d25a-istio-podinfo\") pod \"e2e-praxis-a-6d953c-openshift-default-5d96c8ccc7-5bjbr\" (UID: \"243ebea3-fb15-455c-8e02-f152b478d25a\") " pod="openshift-ingress/e2e-praxis-a-6d953c-openshift-default-5d96c8ccc7-5bjbr"
Oct 30 19:17:25.704190 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:25.704151    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-praxis-a-6d953c-openshift-default-dockercfg-jfkgg\""
Oct 30 19:17:25.802025 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:25.801987    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/243ebea3-fb15-455c-8e02-f152b478d25a-istio-envoy\") pod \"e2e-praxis-a-6d953c-openshift-default-5d96c8ccc7-5bjbr\" (UID: \"243ebea3-fb15-455c-8e02-f152b478d25a\") " pod="openshift-ingress/e2e-praxis-a-6d953c-openshift-default-5d96c8ccc7-5bjbr"
Oct 30 19:17:25.802911 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:25.802881    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/243ebea3-fb15-455c-8e02-f152b478d25a-istio-token\") pod \"e2e-praxis-a-6d953c-openshift-default-5d96c8ccc7-5bjbr\" (UID: \"243ebea3-fb15-455c-8e02-f152b478d25a\") " pod="openshift-ingress/e2e-praxis-a-6d953c-openshift-default-5d96c8ccc7-5bjbr"
Oct 30 19:17:25.804085 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:25.804026    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/243ebea3-fb15-455c-8e02-f152b478d25a-workload-certs\") pod \"e2e-praxis-a-6d953c-openshift-default-5d96c8ccc7-5bjbr\" (UID: \"243ebea3-fb15-455c-8e02-f152b478d25a\") " pod="openshift-ingress/e2e-praxis-a-6d953c-openshift-default-5d96c8ccc7-5bjbr"
Oct 30 19:17:25.804245 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:25.804226    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/243ebea3-fb15-455c-8e02-f152b478d25a-workload-certs\") pod \"e2e-praxis-a-6d953c-openshift-default-5d96c8ccc7-5bjbr\" (UID: \"243ebea3-fb15-455c-8e02-f152b478d25a\") " pod="openshift-ingress/e2e-praxis-a-6d953c-openshift-default-5d96c8ccc7-5bjbr"
Oct 30 19:17:25.804851 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:25.804413    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hvtvd\" (UniqueName: \"kubernetes.io/projected/243ebea3-fb15-455c-8e02-f152b478d25a-kube-api-access-hvtvd\") pod \"e2e-praxis-a-6d953c-openshift-default-5d96c8ccc7-5bjbr\" (UID: \"243ebea3-fb15-455c-8e02-f152b478d25a\") " pod="openshift-ingress/e2e-praxis-a-6d953c-openshift-default-5d96c8ccc7-5bjbr"
Oct 30 19:17:25.810857 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:25.805983    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/243ebea3-fb15-455c-8e02-f152b478d25a-istiod-ca-cert\") pod \"e2e-praxis-a-6d953c-openshift-default-5d96c8ccc7-5bjbr\" (UID: \"243ebea3-fb15-455c-8e02-f152b478d25a\") " pod="openshift-ingress/e2e-praxis-a-6d953c-openshift-default-5d96c8ccc7-5bjbr"
Oct 30 19:17:25.810857 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:25.806071    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/243ebea3-fb15-455c-8e02-f152b478d25a-credential-socket\") pod \"e2e-praxis-a-6d953c-openshift-default-5d96c8ccc7-5bjbr\" (UID: \"243ebea3-fb15-455c-8e02-f152b478d25a\") " pod="openshift-ingress/e2e-praxis-a-6d953c-openshift-default-5d96c8ccc7-5bjbr"
Oct 30 19:17:25.810857 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:25.806105    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/243ebea3-fb15-455c-8e02-f152b478d25a-workload-socket\") pod \"e2e-praxis-a-6d953c-openshift-default-5d96c8ccc7-5bjbr\" (UID: \"243ebea3-fb15-455c-8e02-f152b478d25a\") " pod="openshift-ingress/e2e-praxis-a-6d953c-openshift-default-5d96c8ccc7-5bjbr"
Oct 30 19:17:25.810857 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:25.806129    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/243ebea3-fb15-455c-8e02-f152b478d25a-istio-data\") pod \"e2e-praxis-a-6d953c-openshift-default-5d96c8ccc7-5bjbr\" (UID: \"243ebea3-fb15-455c-8e02-f152b478d25a\") " pod="openshift-ingress/e2e-praxis-a-6d953c-openshift-default-5d96c8ccc7-5bjbr"
Oct 30 19:17:25.810857 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:25.806160    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/243ebea3-fb15-455c-8e02-f152b478d25a-istio-podinfo\") pod \"e2e-praxis-a-6d953c-openshift-default-5d96c8ccc7-5bjbr\" (UID: \"243ebea3-fb15-455c-8e02-f152b478d25a\") " pod="openshift-ingress/e2e-praxis-a-6d953c-openshift-default-5d96c8ccc7-5bjbr"
Oct 30 19:17:25.810857 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:25.806865    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/243ebea3-fb15-455c-8e02-f152b478d25a-credential-socket\") pod \"e2e-praxis-a-6d953c-openshift-default-5d96c8ccc7-5bjbr\" (UID: \"243ebea3-fb15-455c-8e02-f152b478d25a\") " pod="openshift-ingress/e2e-praxis-a-6d953c-openshift-default-5d96c8ccc7-5bjbr"
Oct 30 19:17:25.810857 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:25.805510    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/243ebea3-fb15-455c-8e02-f152b478d25a-istio-envoy\") pod \"e2e-praxis-a-6d953c-openshift-default-5d96c8ccc7-5bjbr\" (UID: \"243ebea3-fb15-455c-8e02-f152b478d25a\") " pod="openshift-ingress/e2e-praxis-a-6d953c-openshift-default-5d96c8ccc7-5bjbr"
Oct 30 19:17:25.810857 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:25.807546    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/243ebea3-fb15-455c-8e02-f152b478d25a-istiod-ca-cert\") pod \"e2e-praxis-a-6d953c-openshift-default-5d96c8ccc7-5bjbr\" (UID: \"243ebea3-fb15-455c-8e02-f152b478d25a\") " pod="openshift-ingress/e2e-praxis-a-6d953c-openshift-default-5d96c8ccc7-5bjbr"
Oct 30 19:17:25.810857 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:25.807771    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/243ebea3-fb15-455c-8e02-f152b478d25a-workload-socket\") pod \"e2e-praxis-a-6d953c-openshift-default-5d96c8ccc7-5bjbr\" (UID: \"243ebea3-fb15-455c-8e02-f152b478d25a\") " pod="openshift-ingress/e2e-praxis-a-6d953c-openshift-default-5d96c8ccc7-5bjbr"
Oct 30 19:17:25.810857 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:25.807976    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/243ebea3-fb15-455c-8e02-f152b478d25a-istio-data\") pod \"e2e-praxis-a-6d953c-openshift-default-5d96c8ccc7-5bjbr\" (UID: \"243ebea3-fb15-455c-8e02-f152b478d25a\") " pod="openshift-ingress/e2e-praxis-a-6d953c-openshift-default-5d96c8ccc7-5bjbr"
Oct 30 19:17:25.810857 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:25.810720    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/243ebea3-fb15-455c-8e02-f152b478d25a-istio-podinfo\") pod \"e2e-praxis-a-6d953c-openshift-default-5d96c8ccc7-5bjbr\" (UID: \"243ebea3-fb15-455c-8e02-f152b478d25a\") " pod="openshift-ingress/e2e-praxis-a-6d953c-openshift-default-5d96c8ccc7-5bjbr"
Oct 30 19:17:25.866092 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:25.866021    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/243ebea3-fb15-455c-8e02-f152b478d25a-istio-token\") pod \"e2e-praxis-a-6d953c-openshift-default-5d96c8ccc7-5bjbr\" (UID: \"243ebea3-fb15-455c-8e02-f152b478d25a\") " pod="openshift-ingress/e2e-praxis-a-6d953c-openshift-default-5d96c8ccc7-5bjbr"
Oct 30 19:17:25.867562 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:25.867527    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hvtvd\" (UniqueName: \"kubernetes.io/projected/243ebea3-fb15-455c-8e02-f152b478d25a-kube-api-access-hvtvd\") pod \"e2e-praxis-a-6d953c-openshift-default-5d96c8ccc7-5bjbr\" (UID: \"243ebea3-fb15-455c-8e02-f152b478d25a\") " pod="openshift-ingress/e2e-praxis-a-6d953c-openshift-default-5d96c8ccc7-5bjbr"
Oct 30 19:17:25.900823 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:25.900788    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="30332320-0fa8-4c94-ba9f-42cbacfdd0d9" path="/var/lib/kubelet/pods/30332320-0fa8-4c94-ba9f-42cbacfdd0d9/volumes"
Oct 30 19:17:25.930725 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:25.929549    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-a-6d953c-openshift-default-5d96c8ccc7-5bjbr"
Oct 30 19:17:26.449629 ip-10-0-134-46 kubenswrapper[4888]: W1030 19:17:26.449601    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod243ebea3_fb15_455c_8e02_f152b478d25a.slice/crio-222066a34813dab087672f8746c842491bee38035ac710132d4501ce2fd4804a WatchSource:0}: Error finding container 222066a34813dab087672f8746c842491bee38035ac710132d4501ce2fd4804a: Status 404 returned error can't find the container with id 222066a34813dab087672f8746c842491bee38035ac710132d4501ce2fd4804a
Oct 30 19:17:26.449932 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:26.449725    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-praxis-a-6d953c-openshift-default-5d96c8ccc7-5bjbr"]
Oct 30 19:17:26.452144 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:26.452110    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:17:26.452244 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:26.452185    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:17:26.452353 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:26.452299    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:17:26.675310 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:26.674050    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-6d953c-openshift-default-5d96c8ccc7-5bjbr" event={"ID":"243ebea3-fb15-455c-8e02-f152b478d25a","Type":"ContainerStarted","Data":"ad1d864505ae3ead0652205ede92b8c31677aa81f8302d590f340688ad2e58fd"}
Oct 30 19:17:26.675310 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:26.674111    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-6d953c-openshift-default-5d96c8ccc7-5bjbr" event={"ID":"243ebea3-fb15-455c-8e02-f152b478d25a","Type":"ContainerStarted","Data":"222066a34813dab087672f8746c842491bee38035ac710132d4501ce2fd4804a"}
Oct 30 19:17:26.822360 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:26.822125    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-praxis-a-6d953c-openshift-default-5d96c8ccc7-5bjbr" podStartSLOduration=1.822104846 podStartE2EDuration="1.822104846s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:17:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:17:26.819831615 +0000 UTC m=+2611.067237640" watchObservedRunningTime="2026-10-30 19:17:26.822104846 +0000 UTC m=+2611.069510871"
Oct 30 19:17:26.937467 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:26.937387    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-praxis-a-6d953c-openshift-default-5d96c8ccc7-5bjbr"
Oct 30 19:17:27.937271 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:27.937245    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-praxis-a-6d953c-openshift-default-5d96c8ccc7-5bjbr"
Oct 30 19:17:28.691305 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:28.691260    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-praxis-a-6d953c-openshift-default-5d96c8ccc7-5bjbr"
Oct 30 19:17:28.692368 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:28.692345    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-praxis-a-6d953c-openshift-default-5d96c8ccc7-5bjbr"
Oct 30 19:17:35.618036 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:17:35.617996    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-jbsw6" containerName="extractor"
Oct 30 19:17:37.112187 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:37.112154    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-66db998887-6v8vj"]
Oct 30 19:17:37.112692 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:37.112383    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-6v8vj" podUID="5fc32c21-43ea-4089-b2c0-3d416459607d" containerName="istio-proxy" containerID="cri-o://dd7a57dd1923dc8963fb179933cccd20d3c6460def98dcf10c724f2c7f89fbb0" gracePeriod=30
Oct 30 19:17:42.393146 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:42.393091    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-6v8vj"
Oct 30 19:17:42.545627 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:42.545589    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/5fc32c21-43ea-4089-b2c0-3d416459607d-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5fc32c21-43ea-4089-b2c0-3d416459607d-istio-podinfo\") pod \"5fc32c21-43ea-4089-b2c0-3d416459607d\" (UID: \"5fc32c21-43ea-4089-b2c0-3d416459607d\") "
Oct 30 19:17:42.545824 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:42.545659    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5fc32c21-43ea-4089-b2c0-3d416459607d-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5fc32c21-43ea-4089-b2c0-3d416459607d-credential-socket\") pod \"5fc32c21-43ea-4089-b2c0-3d416459607d\" (UID: \"5fc32c21-43ea-4089-b2c0-3d416459607d\") "
Oct 30 19:17:42.545824 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:42.545678    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5fc32c21-43ea-4089-b2c0-3d416459607d-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5fc32c21-43ea-4089-b2c0-3d416459607d-workload-socket\") pod \"5fc32c21-43ea-4089-b2c0-3d416459607d\" (UID: \"5fc32c21-43ea-4089-b2c0-3d416459607d\") "
Oct 30 19:17:42.545966 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:42.545829    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5fc32c21-43ea-4089-b2c0-3d416459607d-istio-token\" (UniqueName: \"kubernetes.io/projected/5fc32c21-43ea-4089-b2c0-3d416459607d-istio-token\") pod \"5fc32c21-43ea-4089-b2c0-3d416459607d\" (UID: \"5fc32c21-43ea-4089-b2c0-3d416459607d\") "
Oct 30 19:17:42.545966 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:42.545872    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5fc32c21-43ea-4089-b2c0-3d416459607d-kube-api-access-r677k\" (UniqueName: \"kubernetes.io/projected/5fc32c21-43ea-4089-b2c0-3d416459607d-kube-api-access-r677k\") pod \"5fc32c21-43ea-4089-b2c0-3d416459607d\" (UID: \"5fc32c21-43ea-4089-b2c0-3d416459607d\") "
Oct 30 19:17:42.545966 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:42.545902    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5fc32c21-43ea-4089-b2c0-3d416459607d-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5fc32c21-43ea-4089-b2c0-3d416459607d-workload-certs\") pod \"5fc32c21-43ea-4089-b2c0-3d416459607d\" (UID: \"5fc32c21-43ea-4089-b2c0-3d416459607d\") "
Oct 30 19:17:42.545966 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:42.545941    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5fc32c21-43ea-4089-b2c0-3d416459607d-credential-socket" pod "5fc32c21-43ea-4089-b2c0-3d416459607d" (UID: "5fc32c21-43ea-4089-b2c0-3d416459607d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:17:42.545966 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:42.545954    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5fc32c21-43ea-4089-b2c0-3d416459607d-workload-socket" pod "5fc32c21-43ea-4089-b2c0-3d416459607d" (UID: "5fc32c21-43ea-4089-b2c0-3d416459607d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:17:42.546163 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:42.545958    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5fc32c21-43ea-4089-b2c0-3d416459607d-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5fc32c21-43ea-4089-b2c0-3d416459607d-istiod-ca-cert\") pod \"5fc32c21-43ea-4089-b2c0-3d416459607d\" (UID: \"5fc32c21-43ea-4089-b2c0-3d416459607d\") "
Oct 30 19:17:42.546212 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:42.546162    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5fc32c21-43ea-4089-b2c0-3d416459607d-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5fc32c21-43ea-4089-b2c0-3d416459607d-istio-data\") pod \"5fc32c21-43ea-4089-b2c0-3d416459607d\" (UID: \"5fc32c21-43ea-4089-b2c0-3d416459607d\") "
Oct 30 19:17:42.546212 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:42.546181    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5fc32c21-43ea-4089-b2c0-3d416459607d-workload-certs" pod "5fc32c21-43ea-4089-b2c0-3d416459607d" (UID: "5fc32c21-43ea-4089-b2c0-3d416459607d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:17:42.546212 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:42.546202    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5fc32c21-43ea-4089-b2c0-3d416459607d-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5fc32c21-43ea-4089-b2c0-3d416459607d-istio-envoy\") pod \"5fc32c21-43ea-4089-b2c0-3d416459607d\" (UID: \"5fc32c21-43ea-4089-b2c0-3d416459607d\") "
Oct 30 19:17:42.546414 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:42.546262    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5fc32c21-43ea-4089-b2c0-3d416459607d-istiod-ca-cert" pod "5fc32c21-43ea-4089-b2c0-3d416459607d" (UID: "5fc32c21-43ea-4089-b2c0-3d416459607d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:17:42.546486 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:42.546408    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5fc32c21-43ea-4089-b2c0-3d416459607d-istio-data" pod "5fc32c21-43ea-4089-b2c0-3d416459607d" (UID: "5fc32c21-43ea-4089-b2c0-3d416459607d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:17:42.546932 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:42.546905    4888 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5fc32c21-43ea-4089-b2c0-3d416459607d-credential-socket\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:17:42.547034 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:42.546935    4888 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5fc32c21-43ea-4089-b2c0-3d416459607d-workload-socket\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:17:42.547034 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:42.546951    4888 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5fc32c21-43ea-4089-b2c0-3d416459607d-workload-certs\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:17:42.547034 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:42.546964    4888 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5fc32c21-43ea-4089-b2c0-3d416459607d-istiod-ca-cert\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:17:42.547034 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:42.546978    4888 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5fc32c21-43ea-4089-b2c0-3d416459607d-istio-data\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:17:42.548485 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:42.548461    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5fc32c21-43ea-4089-b2c0-3d416459607d-istio-envoy" pod "5fc32c21-43ea-4089-b2c0-3d416459607d" (UID: "5fc32c21-43ea-4089-b2c0-3d416459607d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:17:42.548586 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:42.548501    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5fc32c21-43ea-4089-b2c0-3d416459607d-istio-token" pod "5fc32c21-43ea-4089-b2c0-3d416459607d" (UID: "5fc32c21-43ea-4089-b2c0-3d416459607d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:17:42.548689 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:42.548672    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5fc32c21-43ea-4089-b2c0-3d416459607d-istio-podinfo" pod "5fc32c21-43ea-4089-b2c0-3d416459607d" (UID: "5fc32c21-43ea-4089-b2c0-3d416459607d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:17:42.548747 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:42.548697    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5fc32c21-43ea-4089-b2c0-3d416459607d-kube-api-access-r677k" pod "5fc32c21-43ea-4089-b2c0-3d416459607d" (UID: "5fc32c21-43ea-4089-b2c0-3d416459607d"). InnerVolumeSpecName "kube-api-access-r677k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:17:42.648175 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:42.648094    4888 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5fc32c21-43ea-4089-b2c0-3d416459607d-istio-token\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:17:42.648175 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:42.648126    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r677k\" (UniqueName: \"kubernetes.io/projected/5fc32c21-43ea-4089-b2c0-3d416459607d-kube-api-access-r677k\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:17:42.648175 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:42.648143    4888 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5fc32c21-43ea-4089-b2c0-3d416459607d-istio-envoy\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:17:42.648175 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:42.648152    4888 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5fc32c21-43ea-4089-b2c0-3d416459607d-istio-podinfo\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:17:42.793509 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:42.793480    4888 generic.go:356] "Generic (PLEG): container finished" podID="5fc32c21-43ea-4089-b2c0-3d416459607d" containerID="dd7a57dd1923dc8963fb179933cccd20d3c6460def98dcf10c724f2c7f89fbb0" exitCode=0
Oct 30 19:17:42.793702 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:42.793518    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-6v8vj" event={"ID":"5fc32c21-43ea-4089-b2c0-3d416459607d","Type":"ContainerDied","Data":"dd7a57dd1923dc8963fb179933cccd20d3c6460def98dcf10c724f2c7f89fbb0"}
Oct 30 19:17:42.793702 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:42.793527    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-6v8vj"
Oct 30 19:17:42.793702 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:42.793562    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-6v8vj" event={"ID":"5fc32c21-43ea-4089-b2c0-3d416459607d","Type":"ContainerDied","Data":"82cdee296f8680720308aa5228fa26a6bf7d468ac56d1fa87c7a2cd18441e73c"}
Oct 30 19:17:42.793702 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:42.793587    4888 scope.go:122] "RemoveContainer" containerID="dd7a57dd1923dc8963fb179933cccd20d3c6460def98dcf10c724f2c7f89fbb0"
Oct 30 19:17:42.804758 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:42.804740    4888 scope.go:122] "RemoveContainer" containerID="dd7a57dd1923dc8963fb179933cccd20d3c6460def98dcf10c724f2c7f89fbb0"
Oct 30 19:17:42.805049 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:17:42.805027    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dd7a57dd1923dc8963fb179933cccd20d3c6460def98dcf10c724f2c7f89fbb0\": container with ID starting with dd7a57dd1923dc8963fb179933cccd20d3c6460def98dcf10c724f2c7f89fbb0 not found: ID does not exist" containerID="dd7a57dd1923dc8963fb179933cccd20d3c6460def98dcf10c724f2c7f89fbb0"
Oct 30 19:17:42.805126 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:42.805061    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dd7a57dd1923dc8963fb179933cccd20d3c6460def98dcf10c724f2c7f89fbb0"} err="failed to get container status \"dd7a57dd1923dc8963fb179933cccd20d3c6460def98dcf10c724f2c7f89fbb0\": rpc error: code = NotFound desc = could not find container \"dd7a57dd1923dc8963fb179933cccd20d3c6460def98dcf10c724f2c7f89fbb0\": container with ID starting with dd7a57dd1923dc8963fb179933cccd20d3c6460def98dcf10c724f2c7f89fbb0 not found: ID does not exist"
Oct 30 19:17:42.972470 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:42.972393    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-66db998887-6v8vj"]
Oct 30 19:17:43.025071 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:43.025034    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-66db998887-6v8vj"]
Oct 30 19:17:43.890517 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:43.890478    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5fc32c21-43ea-4089-b2c0-3d416459607d" path="/var/lib/kubelet/pods/5fc32c21-43ea-4089-b2c0-3d416459607d/volumes"
Oct 30 19:17:49.279223 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:49.279179    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-rjmdz"]
Oct 30 19:17:52.301631 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:52.301597    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-52d383-6f847499f-hmbjs"]
Oct 30 19:17:52.302064 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:52.301814    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-52d383-6f847499f-hmbjs" podUID="dd3ecf9f-3563-4640-aec5-bb83ed70f726" containerName="maas-api" containerID="cri-o://5f507b2ca96a58906a94d576f81fea6a4e09dbf32668f4a28998a476420155ab" gracePeriod=30
Oct 30 19:17:52.607848 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:52.607824    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-52d383-6f847499f-hmbjs"
Oct 30 19:17:52.695392 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:52.694972    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/dd3ecf9f-3563-4640-aec5-bb83ed70f726-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dd3ecf9f-3563-4640-aec5-bb83ed70f726-maas-api-tls\") pod \"dd3ecf9f-3563-4640-aec5-bb83ed70f726\" (UID: \"dd3ecf9f-3563-4640-aec5-bb83ed70f726\") "
Oct 30 19:17:52.695392 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:52.695084    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/dd3ecf9f-3563-4640-aec5-bb83ed70f726-metrics-certs\" (UniqueName: \"kubernetes.io/secret/dd3ecf9f-3563-4640-aec5-bb83ed70f726-metrics-certs\") pod \"dd3ecf9f-3563-4640-aec5-bb83ed70f726\" (UID: \"dd3ecf9f-3563-4640-aec5-bb83ed70f726\") "
Oct 30 19:17:52.695392 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:52.695153    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/dd3ecf9f-3563-4640-aec5-bb83ed70f726-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dd3ecf9f-3563-4640-aec5-bb83ed70f726-openshift-service-ca\") pod \"dd3ecf9f-3563-4640-aec5-bb83ed70f726\" (UID: \"dd3ecf9f-3563-4640-aec5-bb83ed70f726\") "
Oct 30 19:17:52.695392 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:52.695203    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/dd3ecf9f-3563-4640-aec5-bb83ed70f726-kube-api-access-fvgfm\" (UniqueName: \"kubernetes.io/projected/dd3ecf9f-3563-4640-aec5-bb83ed70f726-kube-api-access-fvgfm\") pod \"dd3ecf9f-3563-4640-aec5-bb83ed70f726\" (UID: \"dd3ecf9f-3563-4640-aec5-bb83ed70f726\") "
Oct 30 19:17:52.697343 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:52.697273    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dd3ecf9f-3563-4640-aec5-bb83ed70f726-openshift-service-ca" pod "dd3ecf9f-3563-4640-aec5-bb83ed70f726" (UID: "dd3ecf9f-3563-4640-aec5-bb83ed70f726"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:17:52.724306 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:52.723226    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dd3ecf9f-3563-4640-aec5-bb83ed70f726-metrics-certs" pod "dd3ecf9f-3563-4640-aec5-bb83ed70f726" (UID: "dd3ecf9f-3563-4640-aec5-bb83ed70f726"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:17:52.724306 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:52.723245    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dd3ecf9f-3563-4640-aec5-bb83ed70f726-kube-api-access-fvgfm" pod "dd3ecf9f-3563-4640-aec5-bb83ed70f726" (UID: "dd3ecf9f-3563-4640-aec5-bb83ed70f726"). InnerVolumeSpecName "kube-api-access-fvgfm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:17:52.724306 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:52.723331    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dd3ecf9f-3563-4640-aec5-bb83ed70f726-maas-api-tls" pod "dd3ecf9f-3563-4640-aec5-bb83ed70f726" (UID: "dd3ecf9f-3563-4640-aec5-bb83ed70f726"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:17:52.797718 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:52.797684    4888 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dd3ecf9f-3563-4640-aec5-bb83ed70f726-metrics-certs\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:17:52.797718 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:52.797718    4888 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dd3ecf9f-3563-4640-aec5-bb83ed70f726-openshift-service-ca\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:17:52.797947 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:52.797728    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fvgfm\" (UniqueName: \"kubernetes.io/projected/dd3ecf9f-3563-4640-aec5-bb83ed70f726-kube-api-access-fvgfm\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:17:52.797947 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:52.797740    4888 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dd3ecf9f-3563-4640-aec5-bb83ed70f726-maas-api-tls\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:17:52.812641 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:52.812609    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-52d383-5b7f46bf4c-m9fjk"]
Oct 30 19:17:52.812839 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:52.812804    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-52d383-5b7f46bf4c-m9fjk" podUID="ade6242e-b404-446e-8474-6e64b818a3f1" containerName="payload-pre-processing" containerID="cri-o://bb33afc32974c4f2bc57e31e225b7ff2088147066067827f6165c3fcebfa6b4b" gracePeriod=30
Oct 30 19:17:52.887370 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:52.887254    4888 generic.go:356] "Generic (PLEG): container finished" podID="dd3ecf9f-3563-4640-aec5-bb83ed70f726" containerID="5f507b2ca96a58906a94d576f81fea6a4e09dbf32668f4a28998a476420155ab" exitCode=0
Oct 30 19:17:52.887370 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:52.887323    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-52d383-6f847499f-hmbjs"
Oct 30 19:17:52.887588 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:52.887398    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-52d383-6f847499f-hmbjs" event={"ID":"dd3ecf9f-3563-4640-aec5-bb83ed70f726","Type":"ContainerDied","Data":"5f507b2ca96a58906a94d576f81fea6a4e09dbf32668f4a28998a476420155ab"}
Oct 30 19:17:52.887588 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:52.887459    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-52d383-6f847499f-hmbjs" event={"ID":"dd3ecf9f-3563-4640-aec5-bb83ed70f726","Type":"ContainerDied","Data":"9890b56c31b427f9d0684073d09f1782c9baa50207a157e011e2afe40762f313"}
Oct 30 19:17:52.887588 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:52.887491    4888 scope.go:122] "RemoveContainer" containerID="5f507b2ca96a58906a94d576f81fea6a4e09dbf32668f4a28998a476420155ab"
Oct 30 19:17:52.932173 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:52.932125    4888 scope.go:122] "RemoveContainer" containerID="5f507b2ca96a58906a94d576f81fea6a4e09dbf32668f4a28998a476420155ab"
Oct 30 19:17:52.934579 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:17:52.932534    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5f507b2ca96a58906a94d576f81fea6a4e09dbf32668f4a28998a476420155ab\": container with ID starting with 5f507b2ca96a58906a94d576f81fea6a4e09dbf32668f4a28998a476420155ab not found: ID does not exist" containerID="5f507b2ca96a58906a94d576f81fea6a4e09dbf32668f4a28998a476420155ab"
Oct 30 19:17:52.934579 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:52.932577    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5f507b2ca96a58906a94d576f81fea6a4e09dbf32668f4a28998a476420155ab"} err="failed to get container status \"5f507b2ca96a58906a94d576f81fea6a4e09dbf32668f4a28998a476420155ab\": rpc error: code = NotFound desc = could not find container \"5f507b2ca96a58906a94d576f81fea6a4e09dbf32668f4a28998a476420155ab\": container with ID starting with 5f507b2ca96a58906a94d576f81fea6a4e09dbf32668f4a28998a476420155ab not found: ID does not exist"
Oct 30 19:17:53.054111 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:53.054080    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-52d383-6f847499f-hmbjs"]
Oct 30 19:17:53.060958 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:53.060932    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-52d383-6f847499f-hmbjs"]
Oct 30 19:17:53.103604 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:53.103578    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-52d383-5b7f46bf4c-m9fjk"
Oct 30 19:17:53.202025 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:53.201947    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ade6242e-b404-446e-8474-6e64b818a3f1-kube-api-access-t9zwc\" (UniqueName: \"kubernetes.io/projected/ade6242e-b404-446e-8474-6e64b818a3f1-kube-api-access-t9zwc\") pod \"ade6242e-b404-446e-8474-6e64b818a3f1\" (UID: \"ade6242e-b404-446e-8474-6e64b818a3f1\") "
Oct 30 19:17:53.202210 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:53.202097    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ade6242e-b404-446e-8474-6e64b818a3f1-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ade6242e-b404-446e-8474-6e64b818a3f1-plugins-config-volume\") pod \"ade6242e-b404-446e-8474-6e64b818a3f1\" (UID: \"ade6242e-b404-446e-8474-6e64b818a3f1\") "
Oct 30 19:17:53.202556 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:53.202526    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ade6242e-b404-446e-8474-6e64b818a3f1-plugins-config-volume" pod "ade6242e-b404-446e-8474-6e64b818a3f1" (UID: "ade6242e-b404-446e-8474-6e64b818a3f1"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:17:53.207239 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:53.207216    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ade6242e-b404-446e-8474-6e64b818a3f1-kube-api-access-t9zwc" pod "ade6242e-b404-446e-8474-6e64b818a3f1" (UID: "ade6242e-b404-446e-8474-6e64b818a3f1"). InnerVolumeSpecName "kube-api-access-t9zwc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:17:53.309339 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:53.303992    4888 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ade6242e-b404-446e-8474-6e64b818a3f1-plugins-config-volume\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:17:53.309339 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:53.304028    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t9zwc\" (UniqueName: \"kubernetes.io/projected/ade6242e-b404-446e-8474-6e64b818a3f1-kube-api-access-t9zwc\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:17:53.908445 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:53.908412    4888 generic.go:356] "Generic (PLEG): container finished" podID="ade6242e-b404-446e-8474-6e64b818a3f1" containerID="bb33afc32974c4f2bc57e31e225b7ff2088147066067827f6165c3fcebfa6b4b" exitCode=0
Oct 30 19:17:53.908739 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:53.908570    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-52d383-5b7f46bf4c-m9fjk"
Oct 30 19:17:53.912301 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:53.912257    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="dd3ecf9f-3563-4640-aec5-bb83ed70f726" path="/var/lib/kubelet/pods/dd3ecf9f-3563-4640-aec5-bb83ed70f726/volumes"
Oct 30 19:17:53.913065 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:53.913037    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-52d383-5b7f46bf4c-m9fjk" event={"ID":"ade6242e-b404-446e-8474-6e64b818a3f1","Type":"ContainerDied","Data":"bb33afc32974c4f2bc57e31e225b7ff2088147066067827f6165c3fcebfa6b4b"}
Oct 30 19:17:53.913182 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:53.913095    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-52d383-5b7f46bf4c-m9fjk" event={"ID":"ade6242e-b404-446e-8474-6e64b818a3f1","Type":"ContainerDied","Data":"751fc4c52572da253a5418662e10993672f0ddf83c47d6f7b1d8b3518cbef646"}
Oct 30 19:17:53.913182 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:53.913126    4888 scope.go:122] "RemoveContainer" containerID="bb33afc32974c4f2bc57e31e225b7ff2088147066067827f6165c3fcebfa6b4b"
Oct 30 19:17:53.930892 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:53.930871    4888 scope.go:122] "RemoveContainer" containerID="bb33afc32974c4f2bc57e31e225b7ff2088147066067827f6165c3fcebfa6b4b"
Oct 30 19:17:53.931235 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:17:53.931215    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bb33afc32974c4f2bc57e31e225b7ff2088147066067827f6165c3fcebfa6b4b\": container with ID starting with bb33afc32974c4f2bc57e31e225b7ff2088147066067827f6165c3fcebfa6b4b not found: ID does not exist" containerID="bb33afc32974c4f2bc57e31e225b7ff2088147066067827f6165c3fcebfa6b4b"
Oct 30 19:17:53.931359 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:53.931244    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bb33afc32974c4f2bc57e31e225b7ff2088147066067827f6165c3fcebfa6b4b"} err="failed to get container status \"bb33afc32974c4f2bc57e31e225b7ff2088147066067827f6165c3fcebfa6b4b\": rpc error: code = NotFound desc = could not find container \"bb33afc32974c4f2bc57e31e225b7ff2088147066067827f6165c3fcebfa6b4b\": container with ID starting with bb33afc32974c4f2bc57e31e225b7ff2088147066067827f6165c3fcebfa6b4b not found: ID does not exist"
Oct 30 19:17:54.086247 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:54.085915    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-52d383-5b7f46bf4c-m9fjk"]
Oct 30 19:17:54.090507 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:54.090479    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-52d383-5b7f46bf4c-m9fjk"]
Oct 30 19:17:55.898572 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:17:55.898534    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ade6242e-b404-446e-8474-6e64b818a3f1" path="/var/lib/kubelet/pods/ade6242e-b404-446e-8474-6e64b818a3f1/volumes"
Oct 30 19:18:05.625542 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:18:05.625445    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-jbsw6" containerName="extractor"
Oct 30 19:18:29.439997 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:18:29.439960    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-rjmdz"]
Oct 30 19:18:35.625784 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:18:35.625748    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-jbsw6" containerName="extractor"
Oct 30 19:18:40.329791 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:18:40.329761    4888 generic.go:356] "Generic (PLEG): container finished" podID="243ebea3-fb15-455c-8e02-f152b478d25a" containerID="ad1d864505ae3ead0652205ede92b8c31677aa81f8302d590f340688ad2e58fd" exitCode=0
Oct 30 19:18:40.330302 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:18:40.329808    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-6d953c-openshift-default-5d96c8ccc7-5bjbr" event={"ID":"243ebea3-fb15-455c-8e02-f152b478d25a","Type":"ContainerDied","Data":"ad1d864505ae3ead0652205ede92b8c31677aa81f8302d590f340688ad2e58fd"}
Oct 30 19:18:40.330302 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:18:40.330187    4888 scope.go:122] "RemoveContainer" containerID="ad1d864505ae3ead0652205ede92b8c31677aa81f8302d590f340688ad2e58fd"
Oct 30 19:18:40.331570 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:18:40.331538    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:18:40.331673 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:18:40.331621    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:18:40.331741 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:18:40.331672    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:18:41.341185 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:18:41.341149    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-6d953c-openshift-default-5d96c8ccc7-5bjbr" event={"ID":"243ebea3-fb15-455c-8e02-f152b478d25a","Type":"ContainerStarted","Data":"16926ae7118d2ba519591d489787b6e3c04a299974b18efd7e9b50043fb7c705"}
Oct 30 19:18:41.931208 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:18:41.931177    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-praxis-a-6d953c-openshift-default-5d96c8ccc7-5bjbr"
Oct 30 19:18:41.932638 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:18:41.932617    4888 patch_prober.go:29] interesting pod/e2e-praxis-a-6d953c-openshift-default-5d96c8ccc7-5bjbr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.136:15021/healthz/ready\": dial tcp 10.132.0.136:15021: connect: connection refused" start-of-body=
Oct 30 19:18:41.932738 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:18:41.932678    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-6d953c-openshift-default-5d96c8ccc7-5bjbr" podUID="243ebea3-fb15-455c-8e02-f152b478d25a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.136:15021/healthz/ready\": dial tcp 10.132.0.136:15021: connect: connection refused"
Oct 30 19:18:42.932408 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:18:42.932374    4888 patch_prober.go:29] interesting pod/e2e-praxis-a-6d953c-openshift-default-5d96c8ccc7-5bjbr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.136:15021/healthz/ready\": dial tcp 10.132.0.136:15021: connect: connection refused" start-of-body=
Oct 30 19:18:42.932854 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:18:42.932439    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-6d953c-openshift-default-5d96c8ccc7-5bjbr" podUID="243ebea3-fb15-455c-8e02-f152b478d25a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.136:15021/healthz/ready\": dial tcp 10.132.0.136:15021: connect: connection refused"
Oct 30 19:18:43.693405 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:18:43.693373    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-praxis-a-6d953c-openshift-default-5d96c8ccc7-5bjbr"
Oct 30 19:18:43.932166 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:18:43.932136    4888 patch_prober.go:29] interesting pod/e2e-praxis-a-6d953c-openshift-default-5d96c8ccc7-5bjbr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.136:15021/healthz/ready\": dial tcp 10.132.0.136:15021: connect: connection refused" start-of-body=
Oct 30 19:18:43.932367 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:18:43.932193    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-6d953c-openshift-default-5d96c8ccc7-5bjbr" podUID="243ebea3-fb15-455c-8e02-f152b478d25a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.136:15021/healthz/ready\": dial tcp 10.132.0.136:15021: connect: connection refused"
Oct 30 19:18:44.932174 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:18:44.932134    4888 patch_prober.go:29] interesting pod/e2e-praxis-a-6d953c-openshift-default-5d96c8ccc7-5bjbr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.136:15021/healthz/ready\": dial tcp 10.132.0.136:15021: connect: connection refused" start-of-body=
Oct 30 19:18:44.932577 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:18:44.932193    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-6d953c-openshift-default-5d96c8ccc7-5bjbr" podUID="243ebea3-fb15-455c-8e02-f152b478d25a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.136:15021/healthz/ready\": dial tcp 10.132.0.136:15021: connect: connection refused"
Oct 30 19:18:45.936710 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:18:45.936671    4888 patch_prober.go:29] interesting pod/e2e-praxis-a-6d953c-openshift-default-5d96c8ccc7-5bjbr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.136:15021/healthz/ready\": dial tcp 10.132.0.136:15021: connect: connection refused" start-of-body=
Oct 30 19:18:45.937359 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:18:45.937308    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-6d953c-openshift-default-5d96c8ccc7-5bjbr" podUID="243ebea3-fb15-455c-8e02-f152b478d25a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.136:15021/healthz/ready\": dial tcp 10.132.0.136:15021: connect: connection refused"
Oct 30 19:18:46.935638 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:18:46.935607    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-praxis-a-6d953c-openshift-default-5d96c8ccc7-5bjbr"
Oct 30 19:18:46.936515 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:18:46.936494    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-praxis-a-6d953c-openshift-default-5d96c8ccc7-5bjbr"
Oct 30 19:18:49.109059 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:18:49.109019    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-praxis-a-81d66e-openshift-default-6595ddff49-nhkjb"]
Oct 30 19:18:49.109866 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:18:49.109844    4888 cpu_manager.go:414] "containerMap: removing container" podUID="dd3ecf9f-3563-4640-aec5-bb83ed70f726" containerName="maas-api"
Oct 30 19:18:49.109992 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:18:49.109868    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="dd3ecf9f-3563-4640-aec5-bb83ed70f726" containerName="maas-api"
Oct 30 19:18:49.109992 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:18:49.109908    4888 cpu_manager.go:414] "containerMap: removing container" podUID="ade6242e-b404-446e-8474-6e64b818a3f1" containerName="payload-pre-processing"
Oct 30 19:18:49.109992 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:18:49.109916    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ade6242e-b404-446e-8474-6e64b818a3f1" containerName="payload-pre-processing"
Oct 30 19:18:49.109992 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:18:49.109951    4888 cpu_manager.go:414] "containerMap: removing container" podUID="5fc32c21-43ea-4089-b2c0-3d416459607d" containerName="istio-proxy"
Oct 30 19:18:49.109992 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:18:49.109958    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5fc32c21-43ea-4089-b2c0-3d416459607d" containerName="istio-proxy"
Oct 30 19:18:49.110673 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:18:49.110140    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="ade6242e-b404-446e-8474-6e64b818a3f1" containerName="payload-pre-processing"
Oct 30 19:18:49.110673 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:18:49.110160    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="0c0bee10-4345-43a8-9008-b60054dc6850" containerName="istio-proxy"
Oct 30 19:18:49.110673 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:18:49.110170    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="5fc32c21-43ea-4089-b2c0-3d416459607d" containerName="istio-proxy"
Oct 30 19:18:49.110673 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:18:49.110183    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="dd3ecf9f-3563-4640-aec5-bb83ed70f726" containerName="maas-api"
Oct 30 19:18:49.114785 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:18:49.114759    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-a-81d66e-openshift-default-6595ddff49-nhkjb"
Oct 30 19:18:49.140492 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:18:49.140470    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-praxis-a-81d66e-openshift-default-6595ddff49-nhkjb"]
Oct 30 19:18:49.188080 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:18:49.188053    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-praxis-a-81d66e-openshift-default-dockercfg-v9fbv\""
Oct 30 19:18:49.215570 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:18:49.215537    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0da0531b-5d26-4468-9343-d071d984f66a-istio-data\") pod \"e2e-praxis-a-81d66e-openshift-default-6595ddff49-nhkjb\" (UID: \"0da0531b-5d26-4468-9343-d071d984f66a\") " pod="openshift-ingress/e2e-praxis-a-81d66e-openshift-default-6595ddff49-nhkjb"
Oct 30 19:18:49.215755 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:18:49.215586    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0da0531b-5d26-4468-9343-d071d984f66a-istio-token\") pod \"e2e-praxis-a-81d66e-openshift-default-6595ddff49-nhkjb\" (UID: \"0da0531b-5d26-4468-9343-d071d984f66a\") " pod="openshift-ingress/e2e-praxis-a-81d66e-openshift-default-6595ddff49-nhkjb"
Oct 30 19:18:49.215755 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:18:49.215648    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0da0531b-5d26-4468-9343-d071d984f66a-workload-certs\") pod \"e2e-praxis-a-81d66e-openshift-default-6595ddff49-nhkjb\" (UID: \"0da0531b-5d26-4468-9343-d071d984f66a\") " pod="openshift-ingress/e2e-praxis-a-81d66e-openshift-default-6595ddff49-nhkjb"
Oct 30 19:18:49.215755 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:18:49.215709    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0da0531b-5d26-4468-9343-d071d984f66a-istio-envoy\") pod \"e2e-praxis-a-81d66e-openshift-default-6595ddff49-nhkjb\" (UID: \"0da0531b-5d26-4468-9343-d071d984f66a\") " pod="openshift-ingress/e2e-praxis-a-81d66e-openshift-default-6595ddff49-nhkjb"
Oct 30 19:18:49.215952 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:18:49.215795    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4n7t7\" (UniqueName: \"kubernetes.io/projected/0da0531b-5d26-4468-9343-d071d984f66a-kube-api-access-4n7t7\") pod \"e2e-praxis-a-81d66e-openshift-default-6595ddff49-nhkjb\" (UID: \"0da0531b-5d26-4468-9343-d071d984f66a\") " pod="openshift-ingress/e2e-praxis-a-81d66e-openshift-default-6595ddff49-nhkjb"
Oct 30 19:18:49.215952 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:18:49.215845    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0da0531b-5d26-4468-9343-d071d984f66a-istio-podinfo\") pod \"e2e-praxis-a-81d66e-openshift-default-6595ddff49-nhkjb\" (UID: \"0da0531b-5d26-4468-9343-d071d984f66a\") " pod="openshift-ingress/e2e-praxis-a-81d66e-openshift-default-6595ddff49-nhkjb"
Oct 30 19:18:49.215952 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:18:49.215884    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0da0531b-5d26-4468-9343-d071d984f66a-workload-socket\") pod \"e2e-praxis-a-81d66e-openshift-default-6595ddff49-nhkjb\" (UID: \"0da0531b-5d26-4468-9343-d071d984f66a\") " pod="openshift-ingress/e2e-praxis-a-81d66e-openshift-default-6595ddff49-nhkjb"
Oct 30 19:18:49.215952 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:18:49.215933    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0da0531b-5d26-4468-9343-d071d984f66a-istiod-ca-cert\") pod \"e2e-praxis-a-81d66e-openshift-default-6595ddff49-nhkjb\" (UID: \"0da0531b-5d26-4468-9343-d071d984f66a\") " pod="openshift-ingress/e2e-praxis-a-81d66e-openshift-default-6595ddff49-nhkjb"
Oct 30 19:18:49.216197 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:18:49.215961    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0da0531b-5d26-4468-9343-d071d984f66a-credential-socket\") pod \"e2e-praxis-a-81d66e-openshift-default-6595ddff49-nhkjb\" (UID: \"0da0531b-5d26-4468-9343-d071d984f66a\") " pod="openshift-ingress/e2e-praxis-a-81d66e-openshift-default-6595ddff49-nhkjb"
Oct 30 19:18:49.317314 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:18:49.317263    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0da0531b-5d26-4468-9343-d071d984f66a-workload-certs\") pod \"e2e-praxis-a-81d66e-openshift-default-6595ddff49-nhkjb\" (UID: \"0da0531b-5d26-4468-9343-d071d984f66a\") " pod="openshift-ingress/e2e-praxis-a-81d66e-openshift-default-6595ddff49-nhkjb"
Oct 30 19:18:49.317507 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:18:49.317363    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0da0531b-5d26-4468-9343-d071d984f66a-istio-envoy\") pod \"e2e-praxis-a-81d66e-openshift-default-6595ddff49-nhkjb\" (UID: \"0da0531b-5d26-4468-9343-d071d984f66a\") " pod="openshift-ingress/e2e-praxis-a-81d66e-openshift-default-6595ddff49-nhkjb"
Oct 30 19:18:49.317507 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:18:49.317449    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4n7t7\" (UniqueName: \"kubernetes.io/projected/0da0531b-5d26-4468-9343-d071d984f66a-kube-api-access-4n7t7\") pod \"e2e-praxis-a-81d66e-openshift-default-6595ddff49-nhkjb\" (UID: \"0da0531b-5d26-4468-9343-d071d984f66a\") " pod="openshift-ingress/e2e-praxis-a-81d66e-openshift-default-6595ddff49-nhkjb"
Oct 30 19:18:49.317507 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:18:49.317498    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0da0531b-5d26-4468-9343-d071d984f66a-istio-podinfo\") pod \"e2e-praxis-a-81d66e-openshift-default-6595ddff49-nhkjb\" (UID: \"0da0531b-5d26-4468-9343-d071d984f66a\") " pod="openshift-ingress/e2e-praxis-a-81d66e-openshift-default-6595ddff49-nhkjb"
Oct 30 19:18:49.317676 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:18:49.317540    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0da0531b-5d26-4468-9343-d071d984f66a-workload-socket\") pod \"e2e-praxis-a-81d66e-openshift-default-6595ddff49-nhkjb\" (UID: \"0da0531b-5d26-4468-9343-d071d984f66a\") " pod="openshift-ingress/e2e-praxis-a-81d66e-openshift-default-6595ddff49-nhkjb"
Oct 30 19:18:49.317676 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:18:49.317587    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0da0531b-5d26-4468-9343-d071d984f66a-istiod-ca-cert\") pod \"e2e-praxis-a-81d66e-openshift-default-6595ddff49-nhkjb\" (UID: \"0da0531b-5d26-4468-9343-d071d984f66a\") " pod="openshift-ingress/e2e-praxis-a-81d66e-openshift-default-6595ddff49-nhkjb"
Oct 30 19:18:49.317676 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:18:49.317616    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0da0531b-5d26-4468-9343-d071d984f66a-credential-socket\") pod \"e2e-praxis-a-81d66e-openshift-default-6595ddff49-nhkjb\" (UID: \"0da0531b-5d26-4468-9343-d071d984f66a\") " pod="openshift-ingress/e2e-praxis-a-81d66e-openshift-default-6595ddff49-nhkjb"
Oct 30 19:18:49.317676 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:18:49.317652    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0da0531b-5d26-4468-9343-d071d984f66a-workload-certs\") pod \"e2e-praxis-a-81d66e-openshift-default-6595ddff49-nhkjb\" (UID: \"0da0531b-5d26-4468-9343-d071d984f66a\") " pod="openshift-ingress/e2e-praxis-a-81d66e-openshift-default-6595ddff49-nhkjb"
Oct 30 19:18:49.317893 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:18:49.317679    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0da0531b-5d26-4468-9343-d071d984f66a-istio-data\") pod \"e2e-praxis-a-81d66e-openshift-default-6595ddff49-nhkjb\" (UID: \"0da0531b-5d26-4468-9343-d071d984f66a\") " pod="openshift-ingress/e2e-praxis-a-81d66e-openshift-default-6595ddff49-nhkjb"
Oct 30 19:18:49.317893 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:18:49.317709    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0da0531b-5d26-4468-9343-d071d984f66a-istio-token\") pod \"e2e-praxis-a-81d66e-openshift-default-6595ddff49-nhkjb\" (UID: \"0da0531b-5d26-4468-9343-d071d984f66a\") " pod="openshift-ingress/e2e-praxis-a-81d66e-openshift-default-6595ddff49-nhkjb"
Oct 30 19:18:49.318004 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:18:49.317919    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0da0531b-5d26-4468-9343-d071d984f66a-credential-socket\") pod \"e2e-praxis-a-81d66e-openshift-default-6595ddff49-nhkjb\" (UID: \"0da0531b-5d26-4468-9343-d071d984f66a\") " pod="openshift-ingress/e2e-praxis-a-81d66e-openshift-default-6595ddff49-nhkjb"
Oct 30 19:18:49.318060 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:18:49.318010    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0da0531b-5d26-4468-9343-d071d984f66a-istio-data\") pod \"e2e-praxis-a-81d66e-openshift-default-6595ddff49-nhkjb\" (UID: \"0da0531b-5d26-4468-9343-d071d984f66a\") " pod="openshift-ingress/e2e-praxis-a-81d66e-openshift-default-6595ddff49-nhkjb"
Oct 30 19:18:49.318249 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:18:49.318228    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0da0531b-5d26-4468-9343-d071d984f66a-istiod-ca-cert\") pod \"e2e-praxis-a-81d66e-openshift-default-6595ddff49-nhkjb\" (UID: \"0da0531b-5d26-4468-9343-d071d984f66a\") " pod="openshift-ingress/e2e-praxis-a-81d66e-openshift-default-6595ddff49-nhkjb"
Oct 30 19:18:49.318495 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:18:49.318479    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0da0531b-5d26-4468-9343-d071d984f66a-workload-socket\") pod \"e2e-praxis-a-81d66e-openshift-default-6595ddff49-nhkjb\" (UID: \"0da0531b-5d26-4468-9343-d071d984f66a\") " pod="openshift-ingress/e2e-praxis-a-81d66e-openshift-default-6595ddff49-nhkjb"
Oct 30 19:18:49.320369 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:18:49.320346    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0da0531b-5d26-4468-9343-d071d984f66a-istio-podinfo\") pod \"e2e-praxis-a-81d66e-openshift-default-6595ddff49-nhkjb\" (UID: \"0da0531b-5d26-4468-9343-d071d984f66a\") " pod="openshift-ingress/e2e-praxis-a-81d66e-openshift-default-6595ddff49-nhkjb"
Oct 30 19:18:49.320513 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:18:49.320441    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0da0531b-5d26-4468-9343-d071d984f66a-istio-envoy\") pod \"e2e-praxis-a-81d66e-openshift-default-6595ddff49-nhkjb\" (UID: \"0da0531b-5d26-4468-9343-d071d984f66a\") " pod="openshift-ingress/e2e-praxis-a-81d66e-openshift-default-6595ddff49-nhkjb"
Oct 30 19:18:49.380140 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:18:49.380063    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0da0531b-5d26-4468-9343-d071d984f66a-istio-token\") pod \"e2e-praxis-a-81d66e-openshift-default-6595ddff49-nhkjb\" (UID: \"0da0531b-5d26-4468-9343-d071d984f66a\") " pod="openshift-ingress/e2e-praxis-a-81d66e-openshift-default-6595ddff49-nhkjb"
Oct 30 19:18:49.380140 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:18:49.380123    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4n7t7\" (UniqueName: \"kubernetes.io/projected/0da0531b-5d26-4468-9343-d071d984f66a-kube-api-access-4n7t7\") pod \"e2e-praxis-a-81d66e-openshift-default-6595ddff49-nhkjb\" (UID: \"0da0531b-5d26-4468-9343-d071d984f66a\") " pod="openshift-ingress/e2e-praxis-a-81d66e-openshift-default-6595ddff49-nhkjb"
Oct 30 19:18:49.432385 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:18:49.432352    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-a-81d66e-openshift-default-6595ddff49-nhkjb"
Oct 30 19:18:49.943171 ip-10-0-134-46 kubenswrapper[4888]: W1030 19:18:49.943141    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0da0531b_5d26_4468_9343_d071d984f66a.slice/crio-5779a13da2ac320744909ab6efd129e7302b36eb655cec2e663b44e69fc75592 WatchSource:0}: Error finding container 5779a13da2ac320744909ab6efd129e7302b36eb655cec2e663b44e69fc75592: Status 404 returned error can't find the container with id 5779a13da2ac320744909ab6efd129e7302b36eb655cec2e663b44e69fc75592
Oct 30 19:18:49.943357 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:18:49.943247    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-praxis-a-81d66e-openshift-default-6595ddff49-nhkjb"]
Oct 30 19:18:49.945718 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:18:49.945690    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:18:49.945792 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:18:49.945753    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:18:49.945792 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:18:49.945782    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:18:50.418543 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:18:50.418505    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-81d66e-openshift-default-6595ddff49-nhkjb" event={"ID":"0da0531b-5d26-4468-9343-d071d984f66a","Type":"ContainerStarted","Data":"f2cc3e4ef1a98a82945d2ae0413bc975ccbd02f3d1cad4912d65607ba3f3979e"}
Oct 30 19:18:50.418929 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:18:50.418564    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-81d66e-openshift-default-6595ddff49-nhkjb" event={"ID":"0da0531b-5d26-4468-9343-d071d984f66a","Type":"ContainerStarted","Data":"5779a13da2ac320744909ab6efd129e7302b36eb655cec2e663b44e69fc75592"}
Oct 30 19:18:50.433249 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:18:50.433227    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-praxis-a-81d66e-openshift-default-6595ddff49-nhkjb"
Oct 30 19:18:50.543894 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:18:50.543844    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-praxis-a-81d66e-openshift-default-6595ddff49-nhkjb" podStartSLOduration=1.543827244 podStartE2EDuration="1.543827244s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:18:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:18:50.540577247 +0000 UTC m=+2694.787983297" watchObservedRunningTime="2026-10-30 19:18:50.543827244 +0000 UTC m=+2694.791233266"
Oct 30 19:18:51.438886 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:18:51.438854    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-praxis-a-81d66e-openshift-default-6595ddff49-nhkjb"
Oct 30 19:18:52.431579 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:18:52.431545    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-praxis-a-81d66e-openshift-default-6595ddff49-nhkjb"
Oct 30 19:18:52.432455 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:18:52.432435    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-praxis-a-81d66e-openshift-default-6595ddff49-nhkjb"
Oct 30 19:18:56.649456 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:18:56.649431    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-7x4rb_382e8b31-a7fa-43f3-9f15-4c61627b0c04/console-operator/2.log"
Oct 30 19:18:56.654249 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:18:56.654225    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-7x4rb_382e8b31-a7fa-43f3-9f15-4c61627b0c04/console-operator/2.log"
Oct 30 19:18:56.657009 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:18:56.656988    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xfcdl_3f3fe764-2652-46c9-8e54-7882c0e6bcf1/ovn-acl-logging/0.log"
Oct 30 19:18:56.661566 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:18:56.661549    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xfcdl_3f3fe764-2652-46c9-8e54-7882c0e6bcf1/ovn-acl-logging/0.log"
Oct 30 19:18:57.785628 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:18:57.785597    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-rjmdz"]
Oct 30 19:18:58.259724 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:18:58.259688    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-praxis-b-9f939e-openshift-default-97686d89c-dhvqt"]
Oct 30 19:18:58.264017 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:18:58.263997    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-b-9f939e-openshift-default-97686d89c-dhvqt"
Oct 30 19:18:58.320930 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:18:58.320902    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-praxis-b-9f939e-openshift-default-dockercfg-bw4sm\""
Oct 30 19:18:58.359023 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:18:58.358987    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v5xgv\" (UniqueName: \"kubernetes.io/projected/d077845d-f1ed-46c2-a420-9984ccbf7902-kube-api-access-v5xgv\") pod \"e2e-praxis-b-9f939e-openshift-default-97686d89c-dhvqt\" (UID: \"d077845d-f1ed-46c2-a420-9984ccbf7902\") " pod="openshift-ingress/e2e-praxis-b-9f939e-openshift-default-97686d89c-dhvqt"
Oct 30 19:18:58.359205 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:18:58.359044    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d077845d-f1ed-46c2-a420-9984ccbf7902-workload-certs\") pod \"e2e-praxis-b-9f939e-openshift-default-97686d89c-dhvqt\" (UID: \"d077845d-f1ed-46c2-a420-9984ccbf7902\") " pod="openshift-ingress/e2e-praxis-b-9f939e-openshift-default-97686d89c-dhvqt"
Oct 30 19:18:58.359205 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:18:58.359194    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d077845d-f1ed-46c2-a420-9984ccbf7902-workload-socket\") pod \"e2e-praxis-b-9f939e-openshift-default-97686d89c-dhvqt\" (UID: \"d077845d-f1ed-46c2-a420-9984ccbf7902\") " pod="openshift-ingress/e2e-praxis-b-9f939e-openshift-default-97686d89c-dhvqt"
Oct 30 19:18:58.359326 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:18:58.359270    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d077845d-f1ed-46c2-a420-9984ccbf7902-istiod-ca-cert\") pod \"e2e-praxis-b-9f939e-openshift-default-97686d89c-dhvqt\" (UID: \"d077845d-f1ed-46c2-a420-9984ccbf7902\") " pod="openshift-ingress/e2e-praxis-b-9f939e-openshift-default-97686d89c-dhvqt"
Oct 30 19:18:58.359405 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:18:58.359374    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d077845d-f1ed-46c2-a420-9984ccbf7902-istio-podinfo\") pod \"e2e-praxis-b-9f939e-openshift-default-97686d89c-dhvqt\" (UID: \"d077845d-f1ed-46c2-a420-9984ccbf7902\") " pod="openshift-ingress/e2e-praxis-b-9f939e-openshift-default-97686d89c-dhvqt"
Oct 30 19:18:58.359469 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:18:58.359409    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d077845d-f1ed-46c2-a420-9984ccbf7902-istio-token\") pod \"e2e-praxis-b-9f939e-openshift-default-97686d89c-dhvqt\" (UID: \"d077845d-f1ed-46c2-a420-9984ccbf7902\") " pod="openshift-ingress/e2e-praxis-b-9f939e-openshift-default-97686d89c-dhvqt"
Oct 30 19:18:58.359469 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:18:58.359437    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d077845d-f1ed-46c2-a420-9984ccbf7902-credential-socket\") pod \"e2e-praxis-b-9f939e-openshift-default-97686d89c-dhvqt\" (UID: \"d077845d-f1ed-46c2-a420-9984ccbf7902\") " pod="openshift-ingress/e2e-praxis-b-9f939e-openshift-default-97686d89c-dhvqt"
Oct 30 19:18:58.359561 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:18:58.359486    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d077845d-f1ed-46c2-a420-9984ccbf7902-istio-envoy\") pod \"e2e-praxis-b-9f939e-openshift-default-97686d89c-dhvqt\" (UID: \"d077845d-f1ed-46c2-a420-9984ccbf7902\") " pod="openshift-ingress/e2e-praxis-b-9f939e-openshift-default-97686d89c-dhvqt"
Oct 30 19:18:58.359561 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:18:58.359502    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d077845d-f1ed-46c2-a420-9984ccbf7902-istio-data\") pod \"e2e-praxis-b-9f939e-openshift-default-97686d89c-dhvqt\" (UID: \"d077845d-f1ed-46c2-a420-9984ccbf7902\") " pod="openshift-ingress/e2e-praxis-b-9f939e-openshift-default-97686d89c-dhvqt"
Oct 30 19:18:58.376483 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:18:58.376453    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-praxis-b-9f939e-openshift-default-97686d89c-dhvqt"]
Oct 30 19:18:58.460880 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:18:58.460847    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d077845d-f1ed-46c2-a420-9984ccbf7902-istio-token\") pod \"e2e-praxis-b-9f939e-openshift-default-97686d89c-dhvqt\" (UID: \"d077845d-f1ed-46c2-a420-9984ccbf7902\") " pod="openshift-ingress/e2e-praxis-b-9f939e-openshift-default-97686d89c-dhvqt"
Oct 30 19:18:58.460880 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:18:58.460882    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d077845d-f1ed-46c2-a420-9984ccbf7902-credential-socket\") pod \"e2e-praxis-b-9f939e-openshift-default-97686d89c-dhvqt\" (UID: \"d077845d-f1ed-46c2-a420-9984ccbf7902\") " pod="openshift-ingress/e2e-praxis-b-9f939e-openshift-default-97686d89c-dhvqt"
Oct 30 19:18:58.461147 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:18:58.460900    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d077845d-f1ed-46c2-a420-9984ccbf7902-istio-envoy\") pod \"e2e-praxis-b-9f939e-openshift-default-97686d89c-dhvqt\" (UID: \"d077845d-f1ed-46c2-a420-9984ccbf7902\") " pod="openshift-ingress/e2e-praxis-b-9f939e-openshift-default-97686d89c-dhvqt"
Oct 30 19:18:58.461147 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:18:58.460916    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d077845d-f1ed-46c2-a420-9984ccbf7902-istio-data\") pod \"e2e-praxis-b-9f939e-openshift-default-97686d89c-dhvqt\" (UID: \"d077845d-f1ed-46c2-a420-9984ccbf7902\") " pod="openshift-ingress/e2e-praxis-b-9f939e-openshift-default-97686d89c-dhvqt"
Oct 30 19:18:58.461147 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:18:58.460980    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v5xgv\" (UniqueName: \"kubernetes.io/projected/d077845d-f1ed-46c2-a420-9984ccbf7902-kube-api-access-v5xgv\") pod \"e2e-praxis-b-9f939e-openshift-default-97686d89c-dhvqt\" (UID: \"d077845d-f1ed-46c2-a420-9984ccbf7902\") " pod="openshift-ingress/e2e-praxis-b-9f939e-openshift-default-97686d89c-dhvqt"
Oct 30 19:18:58.461147 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:18:58.461004    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d077845d-f1ed-46c2-a420-9984ccbf7902-workload-certs\") pod \"e2e-praxis-b-9f939e-openshift-default-97686d89c-dhvqt\" (UID: \"d077845d-f1ed-46c2-a420-9984ccbf7902\") " pod="openshift-ingress/e2e-praxis-b-9f939e-openshift-default-97686d89c-dhvqt"
Oct 30 19:18:58.461147 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:18:58.461079    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d077845d-f1ed-46c2-a420-9984ccbf7902-workload-socket\") pod \"e2e-praxis-b-9f939e-openshift-default-97686d89c-dhvqt\" (UID: \"d077845d-f1ed-46c2-a420-9984ccbf7902\") " pod="openshift-ingress/e2e-praxis-b-9f939e-openshift-default-97686d89c-dhvqt"
Oct 30 19:18:58.461147 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:18:58.461122    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d077845d-f1ed-46c2-a420-9984ccbf7902-istiod-ca-cert\") pod \"e2e-praxis-b-9f939e-openshift-default-97686d89c-dhvqt\" (UID: \"d077845d-f1ed-46c2-a420-9984ccbf7902\") " pod="openshift-ingress/e2e-praxis-b-9f939e-openshift-default-97686d89c-dhvqt"
Oct 30 19:18:58.461573 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:18:58.461203    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d077845d-f1ed-46c2-a420-9984ccbf7902-istio-podinfo\") pod \"e2e-praxis-b-9f939e-openshift-default-97686d89c-dhvqt\" (UID: \"d077845d-f1ed-46c2-a420-9984ccbf7902\") " pod="openshift-ingress/e2e-praxis-b-9f939e-openshift-default-97686d89c-dhvqt"
Oct 30 19:18:58.461573 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:18:58.461318    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d077845d-f1ed-46c2-a420-9984ccbf7902-credential-socket\") pod \"e2e-praxis-b-9f939e-openshift-default-97686d89c-dhvqt\" (UID: \"d077845d-f1ed-46c2-a420-9984ccbf7902\") " pod="openshift-ingress/e2e-praxis-b-9f939e-openshift-default-97686d89c-dhvqt"
Oct 30 19:18:58.461573 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:18:58.461451    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d077845d-f1ed-46c2-a420-9984ccbf7902-workload-certs\") pod \"e2e-praxis-b-9f939e-openshift-default-97686d89c-dhvqt\" (UID: \"d077845d-f1ed-46c2-a420-9984ccbf7902\") " pod="openshift-ingress/e2e-praxis-b-9f939e-openshift-default-97686d89c-dhvqt"
Oct 30 19:18:58.461573 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:18:58.461473    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d077845d-f1ed-46c2-a420-9984ccbf7902-workload-socket\") pod \"e2e-praxis-b-9f939e-openshift-default-97686d89c-dhvqt\" (UID: \"d077845d-f1ed-46c2-a420-9984ccbf7902\") " pod="openshift-ingress/e2e-praxis-b-9f939e-openshift-default-97686d89c-dhvqt"
Oct 30 19:18:58.461573 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:18:58.461492    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d077845d-f1ed-46c2-a420-9984ccbf7902-istio-data\") pod \"e2e-praxis-b-9f939e-openshift-default-97686d89c-dhvqt\" (UID: \"d077845d-f1ed-46c2-a420-9984ccbf7902\") " pod="openshift-ingress/e2e-praxis-b-9f939e-openshift-default-97686d89c-dhvqt"
Oct 30 19:18:58.461956 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:18:58.461935    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d077845d-f1ed-46c2-a420-9984ccbf7902-istiod-ca-cert\") pod \"e2e-praxis-b-9f939e-openshift-default-97686d89c-dhvqt\" (UID: \"d077845d-f1ed-46c2-a420-9984ccbf7902\") " pod="openshift-ingress/e2e-praxis-b-9f939e-openshift-default-97686d89c-dhvqt"
Oct 30 19:18:58.463643 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:18:58.463624    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d077845d-f1ed-46c2-a420-9984ccbf7902-istio-envoy\") pod \"e2e-praxis-b-9f939e-openshift-default-97686d89c-dhvqt\" (UID: \"d077845d-f1ed-46c2-a420-9984ccbf7902\") " pod="openshift-ingress/e2e-praxis-b-9f939e-openshift-default-97686d89c-dhvqt"
Oct 30 19:18:58.463931 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:18:58.463914    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d077845d-f1ed-46c2-a420-9984ccbf7902-istio-podinfo\") pod \"e2e-praxis-b-9f939e-openshift-default-97686d89c-dhvqt\" (UID: \"d077845d-f1ed-46c2-a420-9984ccbf7902\") " pod="openshift-ingress/e2e-praxis-b-9f939e-openshift-default-97686d89c-dhvqt"
Oct 30 19:18:58.523659 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:18:58.523600    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d077845d-f1ed-46c2-a420-9984ccbf7902-istio-token\") pod \"e2e-praxis-b-9f939e-openshift-default-97686d89c-dhvqt\" (UID: \"d077845d-f1ed-46c2-a420-9984ccbf7902\") " pod="openshift-ingress/e2e-praxis-b-9f939e-openshift-default-97686d89c-dhvqt"
Oct 30 19:18:58.523835 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:18:58.523814    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v5xgv\" (UniqueName: \"kubernetes.io/projected/d077845d-f1ed-46c2-a420-9984ccbf7902-kube-api-access-v5xgv\") pod \"e2e-praxis-b-9f939e-openshift-default-97686d89c-dhvqt\" (UID: \"d077845d-f1ed-46c2-a420-9984ccbf7902\") " pod="openshift-ingress/e2e-praxis-b-9f939e-openshift-default-97686d89c-dhvqt"
Oct 30 19:18:58.579433 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:18:58.579405    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-b-9f939e-openshift-default-97686d89c-dhvqt"
Oct 30 19:18:59.026547 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:18:59.026518    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-praxis-b-9f939e-openshift-default-97686d89c-dhvqt"]
Oct 30 19:18:59.027061 ip-10-0-134-46 kubenswrapper[4888]: W1030 19:18:59.027035    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd077845d_f1ed_46c2_a420_9984ccbf7902.slice/crio-6633f748f09f6d12d21a1185c964f466d3d42b26f75bbc83be57454553ddff27 WatchSource:0}: Error finding container 6633f748f09f6d12d21a1185c964f466d3d42b26f75bbc83be57454553ddff27: Status 404 returned error can't find the container with id 6633f748f09f6d12d21a1185c964f466d3d42b26f75bbc83be57454553ddff27
Oct 30 19:18:59.029519 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:18:59.029488    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:18:59.029603 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:18:59.029548    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:18:59.029603 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:18:59.029577    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:18:59.480585 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:18:59.480547    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-b-9f939e-openshift-default-97686d89c-dhvqt" event={"ID":"d077845d-f1ed-46c2-a420-9984ccbf7902","Type":"ContainerStarted","Data":"16751dc6ca4f77a859a81e67e7f88922ad68e52103486d441ab5609cef7f8936"}
Oct 30 19:18:59.480759 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:18:59.480602    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-b-9f939e-openshift-default-97686d89c-dhvqt" event={"ID":"d077845d-f1ed-46c2-a420-9984ccbf7902","Type":"ContainerStarted","Data":"6633f748f09f6d12d21a1185c964f466d3d42b26f75bbc83be57454553ddff27"}
Oct 30 19:18:59.579634 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:18:59.579598    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-praxis-b-9f939e-openshift-default-97686d89c-dhvqt"
Oct 30 19:18:59.608475 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:18:59.608423    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-praxis-b-9f939e-openshift-default-97686d89c-dhvqt" podStartSLOduration=1.608407787 podStartE2EDuration="1.608407787s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:18:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:18:59.605686904 +0000 UTC m=+2703.853092929" watchObservedRunningTime="2026-10-30 19:18:59.608407787 +0000 UTC m=+2703.855813812"
Oct 30 19:19:00.586090 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:00.586057    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-praxis-b-9f939e-openshift-default-97686d89c-dhvqt"
Oct 30 19:19:01.495122 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:01.495084    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-praxis-b-9f939e-openshift-default-97686d89c-dhvqt"
Oct 30 19:19:01.496114 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:01.496093    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-praxis-b-9f939e-openshift-default-97686d89c-dhvqt"
Oct 30 19:19:02.021955 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:02.019442    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w1-23302c-7854f56b76-vc7z7"]
Oct 30 19:19:02.029278 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:02.029253    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-23302c-7854f56b76-vc7z7"
Oct 30 19:19:02.087327 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:02.085647    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w1-23302c\""
Oct 30 19:19:02.087327 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:02.085802    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w1-23302c-dockercfg-xzt8c\""
Oct 30 19:19:02.089712 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:02.089682    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w1-23302c-7854f56b76-vc7z7"]
Oct 30 19:19:02.125619 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:02.125585    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/546e6862-d08b-48ad-bb55-3fd290c8ce37-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w1-23302c-7854f56b76-vc7z7\" (UID: \"546e6862-d08b-48ad-bb55-3fd290c8ce37\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w1-23302c-7854f56b76-vc7z7"
Oct 30 19:19:02.125840 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:02.125628    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4vx45\" (UniqueName: \"kubernetes.io/projected/546e6862-d08b-48ad-bb55-3fd290c8ce37-kube-api-access-4vx45\") pod \"payload-processing-e2e-shared-a-w1-23302c-7854f56b76-vc7z7\" (UID: \"546e6862-d08b-48ad-bb55-3fd290c8ce37\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w1-23302c-7854f56b76-vc7z7"
Oct 30 19:19:02.227197 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:02.227163    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/546e6862-d08b-48ad-bb55-3fd290c8ce37-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w1-23302c-7854f56b76-vc7z7\" (UID: \"546e6862-d08b-48ad-bb55-3fd290c8ce37\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w1-23302c-7854f56b76-vc7z7"
Oct 30 19:19:02.227395 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:02.227208    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4vx45\" (UniqueName: \"kubernetes.io/projected/546e6862-d08b-48ad-bb55-3fd290c8ce37-kube-api-access-4vx45\") pod \"payload-processing-e2e-shared-a-w1-23302c-7854f56b76-vc7z7\" (UID: \"546e6862-d08b-48ad-bb55-3fd290c8ce37\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w1-23302c-7854f56b76-vc7z7"
Oct 30 19:19:02.227866 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:02.227847    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/546e6862-d08b-48ad-bb55-3fd290c8ce37-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w1-23302c-7854f56b76-vc7z7\" (UID: \"546e6862-d08b-48ad-bb55-3fd290c8ce37\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w1-23302c-7854f56b76-vc7z7"
Oct 30 19:19:02.295560 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:02.295497    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4vx45\" (UniqueName: \"kubernetes.io/projected/546e6862-d08b-48ad-bb55-3fd290c8ce37-kube-api-access-4vx45\") pod \"payload-processing-e2e-shared-a-w1-23302c-7854f56b76-vc7z7\" (UID: \"546e6862-d08b-48ad-bb55-3fd290c8ce37\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w1-23302c-7854f56b76-vc7z7"
Oct 30 19:19:02.353504 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:02.353470    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-23302c-7854f56b76-vc7z7"
Oct 30 19:19:02.741579 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:02.741548    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w1-23302c-7854f56b76-vc7z7"]
Oct 30 19:19:02.741858 ip-10-0-134-46 kubenswrapper[4888]: W1030 19:19:02.741832    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod546e6862_d08b_48ad_bb55_3fd290c8ce37.slice/crio-888688a4129c00b1da5213002694b7354003080b536938e3a4ec6e1cdf7157c9 WatchSource:0}: Error finding container 888688a4129c00b1da5213002694b7354003080b536938e3a4ec6e1cdf7157c9: Status 404 returned error can't find the container with id 888688a4129c00b1da5213002694b7354003080b536938e3a4ec6e1cdf7157c9
Oct 30 19:19:03.512243 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:03.512207    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-23302c-7854f56b76-vc7z7" event={"ID":"546e6862-d08b-48ad-bb55-3fd290c8ce37","Type":"ContainerStarted","Data":"99a792cb2652cd87a8111bafd5dcde3fdb2cbc29b4048b1c31fc0ada6dcd34e0"}
Oct 30 19:19:03.512734 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:03.512272    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-23302c-7854f56b76-vc7z7" event={"ID":"546e6862-d08b-48ad-bb55-3fd290c8ce37","Type":"ContainerStarted","Data":"888688a4129c00b1da5213002694b7354003080b536938e3a4ec6e1cdf7157c9"}
Oct 30 19:19:03.632742 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:03.632688    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-23302c-7854f56b76-vc7z7" podStartSLOduration=2.632676033 podStartE2EDuration="2.632676033s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:19:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:19:03.63120142 +0000 UTC m=+2707.878607444" watchObservedRunningTime="2026-10-30 19:19:03.632676033 +0000 UTC m=+2707.880082094"
Oct 30 19:19:04.564337 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:04.553363    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-23302c-7854f56b76-vc7z7"
Oct 30 19:19:05.627183 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:19:05.627143    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-jbsw6" containerName="extractor"
Oct 30 19:19:14.099211 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:14.099173    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-b-9f939e-548f4979cf-7wj42"]
Oct 30 19:19:14.103413 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:14.103389    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-9f939e-548f4979cf-7wj42"
Oct 30 19:19:14.189734 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:14.189705    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/20f59c4c-20e2-4be8-a76d-d98911238024-plugins-config-volume\") pod \"payload-pre-processing-e2e-praxis-b-9f939e-548f4979cf-7wj42\" (UID: \"20f59c4c-20e2-4be8-a76d-d98911238024\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-9f939e-548f4979cf-7wj42"
Oct 30 19:19:14.189869 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:14.189821    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fqzqm\" (UniqueName: \"kubernetes.io/projected/20f59c4c-20e2-4be8-a76d-d98911238024-kube-api-access-fqzqm\") pod \"payload-pre-processing-e2e-praxis-b-9f939e-548f4979cf-7wj42\" (UID: \"20f59c4c-20e2-4be8-a76d-d98911238024\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-9f939e-548f4979cf-7wj42"
Oct 30 19:19:14.207932 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:14.207909    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-praxis-b-9f939e-dockercfg-xdfnz\""
Oct 30 19:19:14.208068 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:14.207910    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-praxis-b-9f939e\""
Oct 30 19:19:14.211422 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:14.211401    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-b-9f939e-548f4979cf-7wj42"]
Oct 30 19:19:14.299159 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:14.298834    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fqzqm\" (UniqueName: \"kubernetes.io/projected/20f59c4c-20e2-4be8-a76d-d98911238024-kube-api-access-fqzqm\") pod \"payload-pre-processing-e2e-praxis-b-9f939e-548f4979cf-7wj42\" (UID: \"20f59c4c-20e2-4be8-a76d-d98911238024\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-9f939e-548f4979cf-7wj42"
Oct 30 19:19:14.307569 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:14.299620    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/20f59c4c-20e2-4be8-a76d-d98911238024-plugins-config-volume\") pod \"payload-pre-processing-e2e-praxis-b-9f939e-548f4979cf-7wj42\" (UID: \"20f59c4c-20e2-4be8-a76d-d98911238024\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-9f939e-548f4979cf-7wj42"
Oct 30 19:19:14.309806 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:14.309780    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/20f59c4c-20e2-4be8-a76d-d98911238024-plugins-config-volume\") pod \"payload-pre-processing-e2e-praxis-b-9f939e-548f4979cf-7wj42\" (UID: \"20f59c4c-20e2-4be8-a76d-d98911238024\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-9f939e-548f4979cf-7wj42"
Oct 30 19:19:14.362528 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:14.362450    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fqzqm\" (UniqueName: \"kubernetes.io/projected/20f59c4c-20e2-4be8-a76d-d98911238024-kube-api-access-fqzqm\") pod \"payload-pre-processing-e2e-praxis-b-9f939e-548f4979cf-7wj42\" (UID: \"20f59c4c-20e2-4be8-a76d-d98911238024\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-9f939e-548f4979cf-7wj42"
Oct 30 19:19:14.417210 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:14.417182    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-9f939e-548f4979cf-7wj42"
Oct 30 19:19:14.809235 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:14.809205    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-b-9f939e-548f4979cf-7wj42"]
Oct 30 19:19:14.809721 ip-10-0-134-46 kubenswrapper[4888]: W1030 19:19:14.809685    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20f59c4c_20e2_4be8_a76d_d98911238024.slice/crio-d5fb09899d9d838f5fd3570dcb1ce833b5ba380ec5dcf83fec07d14ac6c5d2ad WatchSource:0}: Error finding container d5fb09899d9d838f5fd3570dcb1ce833b5ba380ec5dcf83fec07d14ac6c5d2ad: Status 404 returned error can't find the container with id d5fb09899d9d838f5fd3570dcb1ce833b5ba380ec5dcf83fec07d14ac6c5d2ad
Oct 30 19:19:15.563474 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:15.563449    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-23302c-7854f56b76-vc7z7"
Oct 30 19:19:15.636453 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:15.636409    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-9f939e-548f4979cf-7wj42" event={"ID":"20f59c4c-20e2-4be8-a76d-d98911238024","Type":"ContainerStarted","Data":"58a3920867d71f158759b469adc7b2aa05741d112072fc1ac9c2ba0b496be66f"}
Oct 30 19:19:15.636635 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:15.636472    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-9f939e-548f4979cf-7wj42"
Oct 30 19:19:15.636635 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:15.636492    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-9f939e-548f4979cf-7wj42" event={"ID":"20f59c4c-20e2-4be8-a76d-d98911238024","Type":"ContainerStarted","Data":"d5fb09899d9d838f5fd3570dcb1ce833b5ba380ec5dcf83fec07d14ac6c5d2ad"}
Oct 30 19:19:16.873847 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:16.873786    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-9f939e-548f4979cf-7wj42" podStartSLOduration=2.873767003 podStartE2EDuration="2.873767003s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:19:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:19:15.806575262 +0000 UTC m=+2720.053981289" watchObservedRunningTime="2026-10-30 19:19:16.873767003 +0000 UTC m=+2721.121173030"
Oct 30 19:19:16.878091 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:16.878051    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w0-19f6ea-55f66dc48d-jgnn8"]
Oct 30 19:19:16.911735 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:16.911706    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-19f6ea-55f66dc48d-jgnn8"
Oct 30 19:19:16.943949 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:16.943921    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w0-19f6ea-55f66dc48d-jgnn8"]
Oct 30 19:19:16.985997 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:16.985950    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w0-19f6ea\""
Oct 30 19:19:16.986190 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:16.986002    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w0-19f6ea-dockercfg-5kbvf\""
Oct 30 19:19:17.064496 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:17.064468    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/50a86a24-23eb-4bd4-abbd-9327c64cc99a-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w0-19f6ea-55f66dc48d-jgnn8\" (UID: \"50a86a24-23eb-4bd4-abbd-9327c64cc99a\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w0-19f6ea-55f66dc48d-jgnn8"
Oct 30 19:19:17.064696 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:17.064553    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jhhb2\" (UniqueName: \"kubernetes.io/projected/50a86a24-23eb-4bd4-abbd-9327c64cc99a-kube-api-access-jhhb2\") pod \"payload-processing-e2e-shared-a-w0-19f6ea-55f66dc48d-jgnn8\" (UID: \"50a86a24-23eb-4bd4-abbd-9327c64cc99a\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w0-19f6ea-55f66dc48d-jgnn8"
Oct 30 19:19:17.166251 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:17.166179    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jhhb2\" (UniqueName: \"kubernetes.io/projected/50a86a24-23eb-4bd4-abbd-9327c64cc99a-kube-api-access-jhhb2\") pod \"payload-processing-e2e-shared-a-w0-19f6ea-55f66dc48d-jgnn8\" (UID: \"50a86a24-23eb-4bd4-abbd-9327c64cc99a\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w0-19f6ea-55f66dc48d-jgnn8"
Oct 30 19:19:17.166417 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:17.166367    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/50a86a24-23eb-4bd4-abbd-9327c64cc99a-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w0-19f6ea-55f66dc48d-jgnn8\" (UID: \"50a86a24-23eb-4bd4-abbd-9327c64cc99a\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w0-19f6ea-55f66dc48d-jgnn8"
Oct 30 19:19:17.167087 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:17.167061    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/50a86a24-23eb-4bd4-abbd-9327c64cc99a-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w0-19f6ea-55f66dc48d-jgnn8\" (UID: \"50a86a24-23eb-4bd4-abbd-9327c64cc99a\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w0-19f6ea-55f66dc48d-jgnn8"
Oct 30 19:19:17.229170 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:17.229142    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jhhb2\" (UniqueName: \"kubernetes.io/projected/50a86a24-23eb-4bd4-abbd-9327c64cc99a-kube-api-access-jhhb2\") pod \"payload-processing-e2e-shared-a-w0-19f6ea-55f66dc48d-jgnn8\" (UID: \"50a86a24-23eb-4bd4-abbd-9327c64cc99a\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w0-19f6ea-55f66dc48d-jgnn8"
Oct 30 19:19:17.234325 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:17.234268    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-19f6ea-55f66dc48d-jgnn8"
Oct 30 19:19:17.623060 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:17.623033    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w0-19f6ea-55f66dc48d-jgnn8"]
Oct 30 19:19:17.623197 ip-10-0-134-46 kubenswrapper[4888]: W1030 19:19:17.623141    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50a86a24_23eb_4bd4_abbd_9327c64cc99a.slice/crio-38dc2e9741bad61417f6bfc0b979b4ad9c43089dc5993632a88ce3245640b932 WatchSource:0}: Error finding container 38dc2e9741bad61417f6bfc0b979b4ad9c43089dc5993632a88ce3245640b932: Status 404 returned error can't find the container with id 38dc2e9741bad61417f6bfc0b979b4ad9c43089dc5993632a88ce3245640b932
Oct 30 19:19:17.653643 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:17.653615    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-19f6ea-55f66dc48d-jgnn8" event={"ID":"50a86a24-23eb-4bd4-abbd-9327c64cc99a","Type":"ContainerStarted","Data":"38dc2e9741bad61417f6bfc0b979b4ad9c43089dc5993632a88ce3245640b932"}
Oct 30 19:19:17.664908 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:17.664882    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-wkl4b"]
Oct 30 19:19:17.670501 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:17.670423    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-wkl4b"
Oct 30 19:19:17.752269 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:17.752196    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w0-2ade09-openshift-default-dockercfg-8v89k\""
Oct 30 19:19:17.757377 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:17.757352    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-wkl4b"]
Oct 30 19:19:17.789467 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:17.789433    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f88bf4ae-259e-4a89-9c1e-70bc180d4b98-istio-envoy\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-wkl4b\" (UID: \"f88bf4ae-259e-4a89-9c1e-70bc180d4b98\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-wkl4b"
Oct 30 19:19:17.789631 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:17.789484    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f88bf4ae-259e-4a89-9c1e-70bc180d4b98-istio-data\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-wkl4b\" (UID: \"f88bf4ae-259e-4a89-9c1e-70bc180d4b98\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-wkl4b"
Oct 30 19:19:17.789631 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:17.789586    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f88bf4ae-259e-4a89-9c1e-70bc180d4b98-istio-token\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-wkl4b\" (UID: \"f88bf4ae-259e-4a89-9c1e-70bc180d4b98\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-wkl4b"
Oct 30 19:19:17.789765 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:17.789626    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9d2vc\" (UniqueName: \"kubernetes.io/projected/f88bf4ae-259e-4a89-9c1e-70bc180d4b98-kube-api-access-9d2vc\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-wkl4b\" (UID: \"f88bf4ae-259e-4a89-9c1e-70bc180d4b98\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-wkl4b"
Oct 30 19:19:17.789765 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:17.789720    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f88bf4ae-259e-4a89-9c1e-70bc180d4b98-workload-socket\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-wkl4b\" (UID: \"f88bf4ae-259e-4a89-9c1e-70bc180d4b98\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-wkl4b"
Oct 30 19:19:17.789882 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:17.789781    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f88bf4ae-259e-4a89-9c1e-70bc180d4b98-istiod-ca-cert\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-wkl4b\" (UID: \"f88bf4ae-259e-4a89-9c1e-70bc180d4b98\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-wkl4b"
Oct 30 19:19:17.789882 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:17.789873    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f88bf4ae-259e-4a89-9c1e-70bc180d4b98-workload-certs\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-wkl4b\" (UID: \"f88bf4ae-259e-4a89-9c1e-70bc180d4b98\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-wkl4b"
Oct 30 19:19:17.790001 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:17.789900    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f88bf4ae-259e-4a89-9c1e-70bc180d4b98-istio-podinfo\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-wkl4b\" (UID: \"f88bf4ae-259e-4a89-9c1e-70bc180d4b98\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-wkl4b"
Oct 30 19:19:17.790001 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:17.789927    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f88bf4ae-259e-4a89-9c1e-70bc180d4b98-credential-socket\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-wkl4b\" (UID: \"f88bf4ae-259e-4a89-9c1e-70bc180d4b98\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-wkl4b"
Oct 30 19:19:17.891523 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:17.891435    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f88bf4ae-259e-4a89-9c1e-70bc180d4b98-istio-envoy\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-wkl4b\" (UID: \"f88bf4ae-259e-4a89-9c1e-70bc180d4b98\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-wkl4b"
Oct 30 19:19:17.891523 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:17.891469    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f88bf4ae-259e-4a89-9c1e-70bc180d4b98-istio-data\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-wkl4b\" (UID: \"f88bf4ae-259e-4a89-9c1e-70bc180d4b98\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-wkl4b"
Oct 30 19:19:17.891523 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:17.891501    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f88bf4ae-259e-4a89-9c1e-70bc180d4b98-istio-token\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-wkl4b\" (UID: \"f88bf4ae-259e-4a89-9c1e-70bc180d4b98\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-wkl4b"
Oct 30 19:19:17.892156 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:17.891525    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9d2vc\" (UniqueName: \"kubernetes.io/projected/f88bf4ae-259e-4a89-9c1e-70bc180d4b98-kube-api-access-9d2vc\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-wkl4b\" (UID: \"f88bf4ae-259e-4a89-9c1e-70bc180d4b98\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-wkl4b"
Oct 30 19:19:17.892156 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:17.891570    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f88bf4ae-259e-4a89-9c1e-70bc180d4b98-workload-socket\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-wkl4b\" (UID: \"f88bf4ae-259e-4a89-9c1e-70bc180d4b98\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-wkl4b"
Oct 30 19:19:17.892156 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:17.891645    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f88bf4ae-259e-4a89-9c1e-70bc180d4b98-istiod-ca-cert\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-wkl4b\" (UID: \"f88bf4ae-259e-4a89-9c1e-70bc180d4b98\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-wkl4b"
Oct 30 19:19:17.892156 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:17.891758    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f88bf4ae-259e-4a89-9c1e-70bc180d4b98-workload-certs\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-wkl4b\" (UID: \"f88bf4ae-259e-4a89-9c1e-70bc180d4b98\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-wkl4b"
Oct 30 19:19:17.892156 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:17.891781    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f88bf4ae-259e-4a89-9c1e-70bc180d4b98-istio-podinfo\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-wkl4b\" (UID: \"f88bf4ae-259e-4a89-9c1e-70bc180d4b98\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-wkl4b"
Oct 30 19:19:17.892156 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:17.891806    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f88bf4ae-259e-4a89-9c1e-70bc180d4b98-credential-socket\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-wkl4b\" (UID: \"f88bf4ae-259e-4a89-9c1e-70bc180d4b98\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-wkl4b"
Oct 30 19:19:17.892156 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:17.891936    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f88bf4ae-259e-4a89-9c1e-70bc180d4b98-istio-data\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-wkl4b\" (UID: \"f88bf4ae-259e-4a89-9c1e-70bc180d4b98\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-wkl4b"
Oct 30 19:19:17.892557 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:17.892466    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f88bf4ae-259e-4a89-9c1e-70bc180d4b98-istiod-ca-cert\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-wkl4b\" (UID: \"f88bf4ae-259e-4a89-9c1e-70bc180d4b98\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-wkl4b"
Oct 30 19:19:17.892593 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:17.892572    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f88bf4ae-259e-4a89-9c1e-70bc180d4b98-workload-certs\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-wkl4b\" (UID: \"f88bf4ae-259e-4a89-9c1e-70bc180d4b98\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-wkl4b"
Oct 30 19:19:17.892810 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:17.892788    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f88bf4ae-259e-4a89-9c1e-70bc180d4b98-workload-socket\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-wkl4b\" (UID: \"f88bf4ae-259e-4a89-9c1e-70bc180d4b98\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-wkl4b"
Oct 30 19:19:17.893063 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:17.893004    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f88bf4ae-259e-4a89-9c1e-70bc180d4b98-credential-socket\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-wkl4b\" (UID: \"f88bf4ae-259e-4a89-9c1e-70bc180d4b98\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-wkl4b"
Oct 30 19:19:17.894726 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:17.894705    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f88bf4ae-259e-4a89-9c1e-70bc180d4b98-istio-envoy\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-wkl4b\" (UID: \"f88bf4ae-259e-4a89-9c1e-70bc180d4b98\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-wkl4b"
Oct 30 19:19:17.894813 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:17.894731    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f88bf4ae-259e-4a89-9c1e-70bc180d4b98-istio-podinfo\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-wkl4b\" (UID: \"f88bf4ae-259e-4a89-9c1e-70bc180d4b98\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-wkl4b"
Oct 30 19:19:17.955122 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:17.955085    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f88bf4ae-259e-4a89-9c1e-70bc180d4b98-istio-token\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-wkl4b\" (UID: \"f88bf4ae-259e-4a89-9c1e-70bc180d4b98\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-wkl4b"
Oct 30 19:19:17.955722 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:17.955697    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9d2vc\" (UniqueName: \"kubernetes.io/projected/f88bf4ae-259e-4a89-9c1e-70bc180d4b98-kube-api-access-9d2vc\") pod \"e2e-worker-w0-2ade09-openshift-default-785b7cf675-wkl4b\" (UID: \"f88bf4ae-259e-4a89-9c1e-70bc180d4b98\") " pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-wkl4b"
Oct 30 19:19:17.985630 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:17.985604    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-wkl4b"
Oct 30 19:19:18.423164 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:18.423137    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-wkl4b"]
Oct 30 19:19:18.423322 ip-10-0-134-46 kubenswrapper[4888]: W1030 19:19:18.423242    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf88bf4ae_259e_4a89_9c1e_70bc180d4b98.slice/crio-a263211a43b9426344b33fbc27893f499350b2c34ac88f774d6d5df61c31117d WatchSource:0}: Error finding container a263211a43b9426344b33fbc27893f499350b2c34ac88f774d6d5df61c31117d: Status 404 returned error can't find the container with id a263211a43b9426344b33fbc27893f499350b2c34ac88f774d6d5df61c31117d
Oct 30 19:19:18.426068 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:18.426032    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:19:18.426142 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:18.426111    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:19:18.426188 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:18.426156    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:19:18.662480 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:18.662450    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-19f6ea-55f66dc48d-jgnn8" event={"ID":"50a86a24-23eb-4bd4-abbd-9327c64cc99a","Type":"ContainerStarted","Data":"cce12db065b94049db00267c524af8becc40b545dca5b23f61f45de44e324684"}
Oct 30 19:19:18.662657 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:18.662555    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-19f6ea-55f66dc48d-jgnn8"
Oct 30 19:19:18.664109 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:18.664084    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-wkl4b" event={"ID":"f88bf4ae-259e-4a89-9c1e-70bc180d4b98","Type":"ContainerStarted","Data":"dd8d40d15069c966d73bbe0572e7f0d5c7d0f2288a7c5f5d7e081660db419f06"}
Oct 30 19:19:18.664215 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:18.664123    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-wkl4b" event={"ID":"f88bf4ae-259e-4a89-9c1e-70bc180d4b98","Type":"ContainerStarted","Data":"a263211a43b9426344b33fbc27893f499350b2c34ac88f774d6d5df61c31117d"}
Oct 30 19:19:18.789990 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:18.789865    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-19f6ea-55f66dc48d-jgnn8" podStartSLOduration=2.7898458010000002 podStartE2EDuration="2.789845801s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:19:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:19:18.786713787 +0000 UTC m=+2723.034119839" watchObservedRunningTime="2026-10-30 19:19:18.789845801 +0000 UTC m=+2723.037251827"
Oct 30 19:19:18.986163 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:18.986131    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-wkl4b"
Oct 30 19:19:19.989309 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:19.988264    4888 patch_prober.go:29] interesting pod/e2e-worker-w0-2ade09-openshift-default-785b7cf675-wkl4b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.142:15021/healthz/ready\": dial tcp 10.132.0.142:15021: connect: connection refused" start-of-body=
Oct 30 19:19:19.989309 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:19.988354    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-wkl4b" podUID="f88bf4ae-259e-4a89-9c1e-70bc180d4b98" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.142:15021/healthz/ready\": dial tcp 10.132.0.142:15021: connect: connection refused"
Oct 30 19:19:20.987309 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:20.986691    4888 patch_prober.go:29] interesting pod/e2e-worker-w0-2ade09-openshift-default-785b7cf675-wkl4b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.142:15021/healthz/ready\": dial tcp 10.132.0.142:15021: connect: connection refused" start-of-body=
Oct 30 19:19:20.987309 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:20.986755    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-wkl4b" podUID="f88bf4ae-259e-4a89-9c1e-70bc180d4b98" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.142:15021/healthz/ready\": dial tcp 10.132.0.142:15021: connect: connection refused"
Oct 30 19:19:21.987875 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:21.987843    4888 patch_prober.go:29] interesting pod/e2e-worker-w0-2ade09-openshift-default-785b7cf675-wkl4b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.142:15021/healthz/ready\": dial tcp 10.132.0.142:15021: connect: connection refused" start-of-body=
Oct 30 19:19:21.988495 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:21.988464    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-wkl4b" podUID="f88bf4ae-259e-4a89-9c1e-70bc180d4b98" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.142:15021/healthz/ready\": dial tcp 10.132.0.142:15021: connect: connection refused"
Oct 30 19:19:22.986839 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:22.986803    4888 patch_prober.go:29] interesting pod/e2e-worker-w0-2ade09-openshift-default-785b7cf675-wkl4b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.142:15021/healthz/ready\": dial tcp 10.132.0.142:15021: connect: connection refused" start-of-body=
Oct 30 19:19:22.987145 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:22.987114    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-wkl4b" podUID="f88bf4ae-259e-4a89-9c1e-70bc180d4b98" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.142:15021/healthz/ready\": dial tcp 10.132.0.142:15021: connect: connection refused"
Oct 30 19:19:23.986703 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:23.986665    4888 patch_prober.go:29] interesting pod/e2e-worker-w0-2ade09-openshift-default-785b7cf675-wkl4b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.142:15021/healthz/ready\": dial tcp 10.132.0.142:15021: connect: connection refused" start-of-body=
Oct 30 19:19:23.987272 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:23.987241    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-wkl4b" podUID="f88bf4ae-259e-4a89-9c1e-70bc180d4b98" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.142:15021/healthz/ready\": dial tcp 10.132.0.142:15021: connect: connection refused"
Oct 30 19:19:24.986825 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:24.986791    4888 patch_prober.go:29] interesting pod/e2e-worker-w0-2ade09-openshift-default-785b7cf675-wkl4b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.142:15021/healthz/ready\": dial tcp 10.132.0.142:15021: connect: connection refused" start-of-body=
Oct 30 19:19:24.987220 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:24.986852    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-wkl4b" podUID="f88bf4ae-259e-4a89-9c1e-70bc180d4b98" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.142:15021/healthz/ready\": dial tcp 10.132.0.142:15021: connect: connection refused"
Oct 30 19:19:26.648371 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:26.648338    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-9f939e-548f4979cf-7wj42"
Oct 30 19:19:26.773413 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:26.773355    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-wkl4b" podStartSLOduration=9.773335611 podStartE2EDuration="9.773335611s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:19:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:19:18.915645115 +0000 UTC m=+2723.163051155" watchObservedRunningTime="2026-10-30 19:19:26.773335611 +0000 UTC m=+2731.020741637"
Oct 30 19:19:27.027534 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:27.027443    4888 patch_prober.go:29] interesting pod/e2e-worker-w0-2ade09-openshift-default-785b7cf675-wkl4b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.142:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:19:27.027817 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:27.027776    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-wkl4b" podUID="f88bf4ae-259e-4a89-9c1e-70bc180d4b98" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.142:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:19:27.090492 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:27.090464    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-wkl4b"
Oct 30 19:19:27.091521 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:27.091497    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-wkl4b"
Oct 30 19:19:27.092879 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:27.092859    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-wkl4b"
Oct 30 19:19:29.692395 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:29.692365    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-19f6ea-55f66dc48d-jgnn8"
Oct 30 19:19:34.079872 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:34.079835    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-rjmdz"]
Oct 30 19:19:35.621055 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:19:35.620977    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-jbsw6" containerName="extractor"
Oct 30 19:19:38.780267 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:38.780235    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-81d66e-7bf78c4974-d779b"]
Oct 30 19:19:38.785492 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:38.785475    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-81d66e-7bf78c4974-d779b"
Oct 30 19:19:38.839144 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:38.839108    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-praxis-a-81d66e-serving-cert\""
Oct 30 19:19:38.871776 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:38.871746    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cde83ed8-4964-4ba0-93c8-d71f10bdbcba-metrics-certs\") pod \"maas-api-e2e-praxis-a-81d66e-7bf78c4974-d779b\" (UID: \"cde83ed8-4964-4ba0-93c8-d71f10bdbcba\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-81d66e-7bf78c4974-d779b"
Oct 30 19:19:38.871966 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:38.871796    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2pr6n\" (UniqueName: \"kubernetes.io/projected/cde83ed8-4964-4ba0-93c8-d71f10bdbcba-kube-api-access-2pr6n\") pod \"maas-api-e2e-praxis-a-81d66e-7bf78c4974-d779b\" (UID: \"cde83ed8-4964-4ba0-93c8-d71f10bdbcba\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-81d66e-7bf78c4974-d779b"
Oct 30 19:19:38.871966 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:38.871923    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cde83ed8-4964-4ba0-93c8-d71f10bdbcba-maas-api-tls\") pod \"maas-api-e2e-praxis-a-81d66e-7bf78c4974-d779b\" (UID: \"cde83ed8-4964-4ba0-93c8-d71f10bdbcba\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-81d66e-7bf78c4974-d779b"
Oct 30 19:19:38.872092 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:38.872023    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cde83ed8-4964-4ba0-93c8-d71f10bdbcba-openshift-service-ca\") pod \"maas-api-e2e-praxis-a-81d66e-7bf78c4974-d779b\" (UID: \"cde83ed8-4964-4ba0-93c8-d71f10bdbcba\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-81d66e-7bf78c4974-d779b"
Oct 30 19:19:38.893540 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:38.893511    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-81d66e-7bf78c4974-d779b"]
Oct 30 19:19:38.973640 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:38.973603    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cde83ed8-4964-4ba0-93c8-d71f10bdbcba-maas-api-tls\") pod \"maas-api-e2e-praxis-a-81d66e-7bf78c4974-d779b\" (UID: \"cde83ed8-4964-4ba0-93c8-d71f10bdbcba\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-81d66e-7bf78c4974-d779b"
Oct 30 19:19:38.973805 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:38.973680    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cde83ed8-4964-4ba0-93c8-d71f10bdbcba-openshift-service-ca\") pod \"maas-api-e2e-praxis-a-81d66e-7bf78c4974-d779b\" (UID: \"cde83ed8-4964-4ba0-93c8-d71f10bdbcba\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-81d66e-7bf78c4974-d779b"
Oct 30 19:19:38.973805 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:19:38.973747    4888 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-praxis-a-81d66e-serving-cert: secret "maas-api-e2e-praxis-a-81d66e-serving-cert" not found
Oct 30 19:19:38.973876 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:19:38.973820    4888 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cde83ed8-4964-4ba0-93c8-d71f10bdbcba-maas-api-tls podName:cde83ed8-4964-4ba0-93c8-d71f10bdbcba nodeName:}" failed. No retries permitted until 2026-10-30 19:19:39.473802713 +0000 UTC m=+2743.721208717 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/cde83ed8-4964-4ba0-93c8-d71f10bdbcba-maas-api-tls") pod "maas-api-e2e-praxis-a-81d66e-7bf78c4974-d779b" (UID: "cde83ed8-4964-4ba0-93c8-d71f10bdbcba") : secret "maas-api-e2e-praxis-a-81d66e-serving-cert" not found
Oct 30 19:19:38.973876 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:38.973837    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cde83ed8-4964-4ba0-93c8-d71f10bdbcba-metrics-certs\") pod \"maas-api-e2e-praxis-a-81d66e-7bf78c4974-d779b\" (UID: \"cde83ed8-4964-4ba0-93c8-d71f10bdbcba\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-81d66e-7bf78c4974-d779b"
Oct 30 19:19:38.973965 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:38.973897    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2pr6n\" (UniqueName: \"kubernetes.io/projected/cde83ed8-4964-4ba0-93c8-d71f10bdbcba-kube-api-access-2pr6n\") pod \"maas-api-e2e-praxis-a-81d66e-7bf78c4974-d779b\" (UID: \"cde83ed8-4964-4ba0-93c8-d71f10bdbcba\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-81d66e-7bf78c4974-d779b"
Oct 30 19:19:38.974533 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:38.974509    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cde83ed8-4964-4ba0-93c8-d71f10bdbcba-openshift-service-ca\") pod \"maas-api-e2e-praxis-a-81d66e-7bf78c4974-d779b\" (UID: \"cde83ed8-4964-4ba0-93c8-d71f10bdbcba\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-81d66e-7bf78c4974-d779b"
Oct 30 19:19:38.976615 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:38.976595    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cde83ed8-4964-4ba0-93c8-d71f10bdbcba-metrics-certs\") pod \"maas-api-e2e-praxis-a-81d66e-7bf78c4974-d779b\" (UID: \"cde83ed8-4964-4ba0-93c8-d71f10bdbcba\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-81d66e-7bf78c4974-d779b"
Oct 30 19:19:39.036695 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:39.036621    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2pr6n\" (UniqueName: \"kubernetes.io/projected/cde83ed8-4964-4ba0-93c8-d71f10bdbcba-kube-api-access-2pr6n\") pod \"maas-api-e2e-praxis-a-81d66e-7bf78c4974-d779b\" (UID: \"cde83ed8-4964-4ba0-93c8-d71f10bdbcba\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-81d66e-7bf78c4974-d779b"
Oct 30 19:19:39.484427 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:39.484387    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cde83ed8-4964-4ba0-93c8-d71f10bdbcba-maas-api-tls\") pod \"maas-api-e2e-praxis-a-81d66e-7bf78c4974-d779b\" (UID: \"cde83ed8-4964-4ba0-93c8-d71f10bdbcba\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-81d66e-7bf78c4974-d779b"
Oct 30 19:19:39.486981 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:39.486955    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cde83ed8-4964-4ba0-93c8-d71f10bdbcba-maas-api-tls\") pod \"maas-api-e2e-praxis-a-81d66e-7bf78c4974-d779b\" (UID: \"cde83ed8-4964-4ba0-93c8-d71f10bdbcba\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-81d66e-7bf78c4974-d779b"
Oct 30 19:19:39.699208 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:39.699174    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-81d66e-7bf78c4974-d779b"
Oct 30 19:19:40.091727 ip-10-0-134-46 kubenswrapper[4888]: W1030 19:19:40.091655    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcde83ed8_4964_4ba0_93c8_d71f10bdbcba.slice/crio-f7dc09d1faf1a43b82327c8f4cb71738d2fffe89db1e82c423f6dd5d7ad15d7c WatchSource:0}: Error finding container f7dc09d1faf1a43b82327c8f4cb71738d2fffe89db1e82c423f6dd5d7ad15d7c: Status 404 returned error can't find the container with id f7dc09d1faf1a43b82327c8f4cb71738d2fffe89db1e82c423f6dd5d7ad15d7c
Oct 30 19:19:40.092511 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:40.092485    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-81d66e-7bf78c4974-d779b"]
Oct 30 19:19:40.092619 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:40.092591    4888 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 19:19:40.853635 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:40.853602    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-81d66e-7bf78c4974-d779b" event={"ID":"cde83ed8-4964-4ba0-93c8-d71f10bdbcba","Type":"ContainerStarted","Data":"f7dc09d1faf1a43b82327c8f4cb71738d2fffe89db1e82c423f6dd5d7ad15d7c"}
Oct 30 19:19:41.862593 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:41.862562    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-81d66e-7bf78c4974-d779b" event={"ID":"cde83ed8-4964-4ba0-93c8-d71f10bdbcba","Type":"ContainerStarted","Data":"ff6080ca18d69c4442d019a04011ab885a7519cba317a21d1e2474f517b0baee"}
Oct 30 19:19:41.863015 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:41.862643    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-81d66e-7bf78c4974-d779b"
Oct 30 19:19:41.991109 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:41.991059    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-81d66e-7bf78c4974-d779b" podStartSLOduration=3.176394376 podStartE2EDuration="3.991046896s" totalImagesPullingTime="814.65252ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:19:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:19:41.990379305 +0000 UTC m=+2746.237785341" watchObservedRunningTime="2026-10-30 19:19:41.991046896 +0000 UTC m=+2746.238452921"
Oct 30 19:19:47.875738 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:47.875707    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-81d66e-7bf78c4974-d779b"
Oct 30 19:19:53.348799 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:53.348767    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-9f939e-786ddbd868-w88zc"]
Oct 30 19:19:53.353101 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:53.353080    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-9f939e-786ddbd868-w88zc"
Oct 30 19:19:53.457488 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:53.457455    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/38709aaf-ee31-44d1-a203-2de0dfa70a30-metrics-certs\") pod \"maas-api-e2e-praxis-b-9f939e-786ddbd868-w88zc\" (UID: \"38709aaf-ee31-44d1-a203-2de0dfa70a30\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-9f939e-786ddbd868-w88zc"
Oct 30 19:19:53.457643 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:53.457627    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4cg6w\" (UniqueName: \"kubernetes.io/projected/38709aaf-ee31-44d1-a203-2de0dfa70a30-kube-api-access-4cg6w\") pod \"maas-api-e2e-praxis-b-9f939e-786ddbd868-w88zc\" (UID: \"38709aaf-ee31-44d1-a203-2de0dfa70a30\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-9f939e-786ddbd868-w88zc"
Oct 30 19:19:53.457687 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:53.457661    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/38709aaf-ee31-44d1-a203-2de0dfa70a30-maas-api-tls\") pod \"maas-api-e2e-praxis-b-9f939e-786ddbd868-w88zc\" (UID: \"38709aaf-ee31-44d1-a203-2de0dfa70a30\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-9f939e-786ddbd868-w88zc"
Oct 30 19:19:53.457803 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:53.457790    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/38709aaf-ee31-44d1-a203-2de0dfa70a30-openshift-service-ca\") pod \"maas-api-e2e-praxis-b-9f939e-786ddbd868-w88zc\" (UID: \"38709aaf-ee31-44d1-a203-2de0dfa70a30\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-9f939e-786ddbd868-w88zc"
Oct 30 19:19:53.458727 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:53.458697    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-praxis-b-9f939e-serving-cert\""
Oct 30 19:19:53.462356 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:53.462333    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-9f939e-786ddbd868-w88zc"]
Oct 30 19:19:53.559502 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:53.559475    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/38709aaf-ee31-44d1-a203-2de0dfa70a30-openshift-service-ca\") pod \"maas-api-e2e-praxis-b-9f939e-786ddbd868-w88zc\" (UID: \"38709aaf-ee31-44d1-a203-2de0dfa70a30\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-9f939e-786ddbd868-w88zc"
Oct 30 19:19:53.559674 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:53.559536    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/38709aaf-ee31-44d1-a203-2de0dfa70a30-metrics-certs\") pod \"maas-api-e2e-praxis-b-9f939e-786ddbd868-w88zc\" (UID: \"38709aaf-ee31-44d1-a203-2de0dfa70a30\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-9f939e-786ddbd868-w88zc"
Oct 30 19:19:53.559674 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:53.559612    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4cg6w\" (UniqueName: \"kubernetes.io/projected/38709aaf-ee31-44d1-a203-2de0dfa70a30-kube-api-access-4cg6w\") pod \"maas-api-e2e-praxis-b-9f939e-786ddbd868-w88zc\" (UID: \"38709aaf-ee31-44d1-a203-2de0dfa70a30\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-9f939e-786ddbd868-w88zc"
Oct 30 19:19:53.559674 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:53.559638    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/38709aaf-ee31-44d1-a203-2de0dfa70a30-maas-api-tls\") pod \"maas-api-e2e-praxis-b-9f939e-786ddbd868-w88zc\" (UID: \"38709aaf-ee31-44d1-a203-2de0dfa70a30\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-9f939e-786ddbd868-w88zc"
Oct 30 19:19:53.559794 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:19:53.559775    4888 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-praxis-b-9f939e-serving-cert: secret "maas-api-e2e-praxis-b-9f939e-serving-cert" not found
Oct 30 19:19:53.559850 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:19:53.559840    4888 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/38709aaf-ee31-44d1-a203-2de0dfa70a30-maas-api-tls podName:38709aaf-ee31-44d1-a203-2de0dfa70a30 nodeName:}" failed. No retries permitted until 2026-10-30 19:19:54.059820275 +0000 UTC m=+2758.307226295 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/38709aaf-ee31-44d1-a203-2de0dfa70a30-maas-api-tls") pod "maas-api-e2e-praxis-b-9f939e-786ddbd868-w88zc" (UID: "38709aaf-ee31-44d1-a203-2de0dfa70a30") : secret "maas-api-e2e-praxis-b-9f939e-serving-cert" not found
Oct 30 19:19:53.560142 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:53.560111    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/38709aaf-ee31-44d1-a203-2de0dfa70a30-openshift-service-ca\") pod \"maas-api-e2e-praxis-b-9f939e-786ddbd868-w88zc\" (UID: \"38709aaf-ee31-44d1-a203-2de0dfa70a30\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-9f939e-786ddbd868-w88zc"
Oct 30 19:19:53.562103 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:53.562074    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/38709aaf-ee31-44d1-a203-2de0dfa70a30-metrics-certs\") pod \"maas-api-e2e-praxis-b-9f939e-786ddbd868-w88zc\" (UID: \"38709aaf-ee31-44d1-a203-2de0dfa70a30\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-9f939e-786ddbd868-w88zc"
Oct 30 19:19:53.632399 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:53.632321    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4cg6w\" (UniqueName: \"kubernetes.io/projected/38709aaf-ee31-44d1-a203-2de0dfa70a30-kube-api-access-4cg6w\") pod \"maas-api-e2e-praxis-b-9f939e-786ddbd868-w88zc\" (UID: \"38709aaf-ee31-44d1-a203-2de0dfa70a30\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-9f939e-786ddbd868-w88zc"
Oct 30 19:19:54.065826 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:54.065745    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/38709aaf-ee31-44d1-a203-2de0dfa70a30-maas-api-tls\") pod \"maas-api-e2e-praxis-b-9f939e-786ddbd868-w88zc\" (UID: \"38709aaf-ee31-44d1-a203-2de0dfa70a30\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-9f939e-786ddbd868-w88zc"
Oct 30 19:19:54.080320 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:54.080265    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/38709aaf-ee31-44d1-a203-2de0dfa70a30-maas-api-tls\") pod \"maas-api-e2e-praxis-b-9f939e-786ddbd868-w88zc\" (UID: \"38709aaf-ee31-44d1-a203-2de0dfa70a30\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-9f939e-786ddbd868-w88zc"
Oct 30 19:19:54.309233 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:54.309205    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-9f939e-786ddbd868-w88zc"
Oct 30 19:19:54.718510 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:54.718481    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-9f939e-786ddbd868-w88zc"]
Oct 30 19:19:54.960655 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:54.960623    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-9f939e-786ddbd868-w88zc" event={"ID":"38709aaf-ee31-44d1-a203-2de0dfa70a30","Type":"ContainerStarted","Data":"13cdd55b24cec0423f447e906e321d4505c9aeb6b49aec796be568199889af43"}
Oct 30 19:19:55.981316 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:55.980150    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-9f939e-786ddbd868-w88zc" event={"ID":"38709aaf-ee31-44d1-a203-2de0dfa70a30","Type":"ContainerStarted","Data":"4fd2799d6e135dc8ee85b54bde30c1d5084e8554f798e43017c0e42b1ae62c73"}
Oct 30 19:19:55.981316 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:55.981204    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-9f939e-786ddbd868-w88zc"
Oct 30 19:19:56.105437 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:19:56.105375    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-9f939e-786ddbd868-w88zc" podStartSLOduration=2.2819221499999998 podStartE2EDuration="3.105356436s" totalImagesPullingTime="823.434286ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:19:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:19:56.101910139 +0000 UTC m=+2760.349316164" watchObservedRunningTime="2026-10-30 19:19:56.105356436 +0000 UTC m=+2760.352762462"
Oct 30 19:20:02.042238 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:20:02.042158    4888 generic.go:356] "Generic (PLEG): container finished" podID="0da0531b-5d26-4468-9343-d071d984f66a" containerID="f2cc3e4ef1a98a82945d2ae0413bc975ccbd02f3d1cad4912d65607ba3f3979e" exitCode=0
Oct 30 19:20:02.042238 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:20:02.042227    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-81d66e-openshift-default-6595ddff49-nhkjb" event={"ID":"0da0531b-5d26-4468-9343-d071d984f66a","Type":"ContainerDied","Data":"f2cc3e4ef1a98a82945d2ae0413bc975ccbd02f3d1cad4912d65607ba3f3979e"}
Oct 30 19:20:02.042773 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:20:02.042751    4888 scope.go:122] "RemoveContainer" containerID="f2cc3e4ef1a98a82945d2ae0413bc975ccbd02f3d1cad4912d65607ba3f3979e"
Oct 30 19:20:02.044255 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:20:02.044229    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:20:02.044355 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:20:02.044309    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:20:02.044355 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:20:02.044343    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:20:03.017818 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:20:03.017778    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-9f939e-786ddbd868-w88zc"
Oct 30 19:20:03.063627 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:20:03.063586    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-81d66e-openshift-default-6595ddff49-nhkjb" event={"ID":"0da0531b-5d26-4468-9343-d071d984f66a","Type":"ContainerStarted","Data":"6139ff209ed15408d686eb69f0d2a7815a21ff7b5c436e487756a53552cc017d"}
Oct 30 19:20:03.433262 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:20:03.433232    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-praxis-a-81d66e-openshift-default-6595ddff49-nhkjb"
Oct 30 19:20:03.434213 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:20:03.434190    4888 patch_prober.go:29] interesting pod/e2e-praxis-a-81d66e-openshift-default-6595ddff49-nhkjb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.137:15021/healthz/ready\": dial tcp 10.132.0.137:15021: connect: connection refused" start-of-body=
Oct 30 19:20:03.434351 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:20:03.434239    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-81d66e-openshift-default-6595ddff49-nhkjb" podUID="0da0531b-5d26-4468-9343-d071d984f66a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.137:15021/healthz/ready\": dial tcp 10.132.0.137:15021: connect: connection refused"
Oct 30 19:20:04.432858 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:20:04.432822    4888 patch_prober.go:29] interesting pod/e2e-praxis-a-81d66e-openshift-default-6595ddff49-nhkjb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.137:15021/healthz/ready\": dial tcp 10.132.0.137:15021: connect: connection refused" start-of-body=
Oct 30 19:20:04.433404 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:20:04.432884    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-81d66e-openshift-default-6595ddff49-nhkjb" podUID="0da0531b-5d26-4468-9343-d071d984f66a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.137:15021/healthz/ready\": dial tcp 10.132.0.137:15021: connect: connection refused"
Oct 30 19:20:05.433526 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:20:05.433492    4888 patch_prober.go:29] interesting pod/e2e-praxis-a-81d66e-openshift-default-6595ddff49-nhkjb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.137:15021/healthz/ready\": dial tcp 10.132.0.137:15021: connect: connection refused" start-of-body=
Oct 30 19:20:05.433980 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:20:05.433549    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-81d66e-openshift-default-6595ddff49-nhkjb" podUID="0da0531b-5d26-4468-9343-d071d984f66a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.137:15021/healthz/ready\": dial tcp 10.132.0.137:15021: connect: connection refused"
Oct 30 19:20:05.626907 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:20:05.626872    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-jbsw6" containerName="extractor"
Oct 30 19:20:06.432901 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:20:06.432859    4888 patch_prober.go:29] interesting pod/e2e-praxis-a-81d66e-openshift-default-6595ddff49-nhkjb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.137:15021/healthz/ready\": dial tcp 10.132.0.137:15021: connect: connection refused" start-of-body=
Oct 30 19:20:06.433099 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:20:06.432922    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-81d66e-openshift-default-6595ddff49-nhkjb" podUID="0da0531b-5d26-4468-9343-d071d984f66a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.137:15021/healthz/ready\": dial tcp 10.132.0.137:15021: connect: connection refused"
Oct 30 19:20:07.432125 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:20:07.432094    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-praxis-a-81d66e-openshift-default-6595ddff49-nhkjb"
Oct 30 19:20:07.432720 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:20:07.432695    4888 patch_prober.go:29] interesting pod/e2e-praxis-a-81d66e-openshift-default-6595ddff49-nhkjb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.137:15021/healthz/ready\": dial tcp 10.132.0.137:15021: connect: connection refused" start-of-body=
Oct 30 19:20:07.432777 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:20:07.432751    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-a-81d66e-openshift-default-6595ddff49-nhkjb" podUID="0da0531b-5d26-4468-9343-d071d984f66a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.137:15021/healthz/ready\": dial tcp 10.132.0.137:15021: connect: connection refused"
Oct 30 19:20:08.437984 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:20:08.437954    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-praxis-a-81d66e-openshift-default-6595ddff49-nhkjb"
Oct 30 19:20:08.439104 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:20:08.439086    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-praxis-a-81d66e-openshift-default-6595ddff49-nhkjb"
Oct 30 19:20:16.064048 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:20:16.064013    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-0b653520-fpmp4"]
Oct 30 19:20:16.116868 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:20:16.116831    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-0b653520-fpmp4"]
Oct 30 19:20:17.893593 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:20:17.893550    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="91ee5bf1-aa98-44bd-b9da-2ce9631dfe49" path="/var/lib/kubelet/pods/91ee5bf1-aa98-44bd-b9da-2ce9631dfe49/volumes"
Oct 30 19:20:18.313378 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:20:18.313293    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-rjmdz"]
Oct 30 19:20:28.284907 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:20:28.284819    4888 generic.go:356] "Generic (PLEG): container finished" podID="d077845d-f1ed-46c2-a420-9984ccbf7902" containerID="16751dc6ca4f77a859a81e67e7f88922ad68e52103486d441ab5609cef7f8936" exitCode=0
Oct 30 19:20:28.285462 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:20:28.284901    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-b-9f939e-openshift-default-97686d89c-dhvqt" event={"ID":"d077845d-f1ed-46c2-a420-9984ccbf7902","Type":"ContainerDied","Data":"16751dc6ca4f77a859a81e67e7f88922ad68e52103486d441ab5609cef7f8936"}
Oct 30 19:20:28.285462 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:20:28.285449    4888 scope.go:122] "RemoveContainer" containerID="16751dc6ca4f77a859a81e67e7f88922ad68e52103486d441ab5609cef7f8936"
Oct 30 19:20:28.287062 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:20:28.287033    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:20:28.287183 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:20:28.287117    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:20:28.287183 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:20:28.287167    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:20:29.307942 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:20:29.307903    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-b-9f939e-openshift-default-97686d89c-dhvqt" event={"ID":"d077845d-f1ed-46c2-a420-9984ccbf7902","Type":"ContainerStarted","Data":"4bd56c5b0746648dab84aa26d44c02f39c21bbb6247eeb23cfa273ad2b681187"}
Oct 30 19:20:29.585703 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:20:29.581640    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-praxis-b-9f939e-openshift-default-97686d89c-dhvqt"
Oct 30 19:20:29.586033 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:20:29.586007    4888 patch_prober.go:29] interesting pod/e2e-praxis-b-9f939e-openshift-default-97686d89c-dhvqt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.138:15021/healthz/ready\": dial tcp 10.132.0.138:15021: connect: connection refused" start-of-body=
Oct 30 19:20:29.586200 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:20:29.586176    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-b-9f939e-openshift-default-97686d89c-dhvqt" podUID="d077845d-f1ed-46c2-a420-9984ccbf7902" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.138:15021/healthz/ready\": dial tcp 10.132.0.138:15021: connect: connection refused"
Oct 30 19:20:30.588310 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:20:30.587614    4888 patch_prober.go:29] interesting pod/e2e-praxis-b-9f939e-openshift-default-97686d89c-dhvqt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.138:15021/healthz/ready\": dial tcp 10.132.0.138:15021: connect: connection refused" start-of-body=
Oct 30 19:20:30.588310 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:20:30.587676    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-b-9f939e-openshift-default-97686d89c-dhvqt" podUID="d077845d-f1ed-46c2-a420-9984ccbf7902" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.138:15021/healthz/ready\": dial tcp 10.132.0.138:15021: connect: connection refused"
Oct 30 19:20:31.498454 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:20:31.498415    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-praxis-b-9f939e-openshift-default-97686d89c-dhvqt"
Oct 30 19:20:31.590305 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:20:31.589624    4888 patch_prober.go:29] interesting pod/e2e-praxis-b-9f939e-openshift-default-97686d89c-dhvqt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.138:15021/healthz/ready\": dial tcp 10.132.0.138:15021: connect: connection refused" start-of-body=
Oct 30 19:20:31.590305 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:20:31.589680    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-b-9f939e-openshift-default-97686d89c-dhvqt" podUID="d077845d-f1ed-46c2-a420-9984ccbf7902" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.138:15021/healthz/ready\": dial tcp 10.132.0.138:15021: connect: connection refused"
Oct 30 19:20:32.593309 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:20:32.590376    4888 patch_prober.go:29] interesting pod/e2e-praxis-b-9f939e-openshift-default-97686d89c-dhvqt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.138:15021/healthz/ready\": dial tcp 10.132.0.138:15021: connect: connection refused" start-of-body=
Oct 30 19:20:32.593309 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:20:32.590461    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-b-9f939e-openshift-default-97686d89c-dhvqt" podUID="d077845d-f1ed-46c2-a420-9984ccbf7902" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.138:15021/healthz/ready\": dial tcp 10.132.0.138:15021: connect: connection refused"
Oct 30 19:20:33.585684 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:20:33.585650    4888 patch_prober.go:29] interesting pod/e2e-praxis-b-9f939e-openshift-default-97686d89c-dhvqt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.138:15021/healthz/ready\": dial tcp 10.132.0.138:15021: connect: connection refused" start-of-body=
Oct 30 19:20:33.586008 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:20:33.585977    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-b-9f939e-openshift-default-97686d89c-dhvqt" podUID="d077845d-f1ed-46c2-a420-9984ccbf7902" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.138:15021/healthz/ready\": dial tcp 10.132.0.138:15021: connect: connection refused"
Oct 30 19:20:34.584935 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:20:34.584908    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-praxis-b-9f939e-openshift-default-97686d89c-dhvqt"
Oct 30 19:20:34.586083 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:20:34.586063    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-praxis-b-9f939e-openshift-default-97686d89c-dhvqt"
Oct 30 19:20:35.618887 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:20:35.618854    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-jbsw6" containerName="extractor"
Oct 30 19:20:47.978444 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:20:47.978403    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-praxis-b-f5dfb8-openshift-default-d59df9d89-28vsl"]
Oct 30 19:20:47.982591 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:20:47.982568    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-b-f5dfb8-openshift-default-d59df9d89-28vsl"
Oct 30 19:20:48.036539 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:20:48.036513    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-praxis-b-f5dfb8-openshift-default-dockercfg-jlcv5\""
Oct 30 19:20:48.090793 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:20:48.090761    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-praxis-b-f5dfb8-openshift-default-d59df9d89-28vsl"]
Oct 30 19:20:48.093556 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:20:48.093532    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b1598dab-80ac-408c-b65a-bc3cde3b01d5-istio-envoy\") pod \"e2e-praxis-b-f5dfb8-openshift-default-d59df9d89-28vsl\" (UID: \"b1598dab-80ac-408c-b65a-bc3cde3b01d5\") " pod="openshift-ingress/e2e-praxis-b-f5dfb8-openshift-default-d59df9d89-28vsl"
Oct 30 19:20:48.093718 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:20:48.093591    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1598dab-80ac-408c-b65a-bc3cde3b01d5-credential-socket\") pod \"e2e-praxis-b-f5dfb8-openshift-default-d59df9d89-28vsl\" (UID: \"b1598dab-80ac-408c-b65a-bc3cde3b01d5\") " pod="openshift-ingress/e2e-praxis-b-f5dfb8-openshift-default-d59df9d89-28vsl"
Oct 30 19:20:48.093777 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:20:48.093756    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b1598dab-80ac-408c-b65a-bc3cde3b01d5-istio-token\") pod \"e2e-praxis-b-f5dfb8-openshift-default-d59df9d89-28vsl\" (UID: \"b1598dab-80ac-408c-b65a-bc3cde3b01d5\") " pod="openshift-ingress/e2e-praxis-b-f5dfb8-openshift-default-d59df9d89-28vsl"
Oct 30 19:20:48.093822 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:20:48.093787    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1598dab-80ac-408c-b65a-bc3cde3b01d5-workload-socket\") pod \"e2e-praxis-b-f5dfb8-openshift-default-d59df9d89-28vsl\" (UID: \"b1598dab-80ac-408c-b65a-bc3cde3b01d5\") " pod="openshift-ingress/e2e-praxis-b-f5dfb8-openshift-default-d59df9d89-28vsl"
Oct 30 19:20:48.093872 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:20:48.093855    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b1598dab-80ac-408c-b65a-bc3cde3b01d5-workload-certs\") pod \"e2e-praxis-b-f5dfb8-openshift-default-d59df9d89-28vsl\" (UID: \"b1598dab-80ac-408c-b65a-bc3cde3b01d5\") " pod="openshift-ingress/e2e-praxis-b-f5dfb8-openshift-default-d59df9d89-28vsl"
Oct 30 19:20:48.093917 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:20:48.093896    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b1598dab-80ac-408c-b65a-bc3cde3b01d5-istio-data\") pod \"e2e-praxis-b-f5dfb8-openshift-default-d59df9d89-28vsl\" (UID: \"b1598dab-80ac-408c-b65a-bc3cde3b01d5\") " pod="openshift-ingress/e2e-praxis-b-f5dfb8-openshift-default-d59df9d89-28vsl"
Oct 30 19:20:48.093964 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:20:48.093920    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b1598dab-80ac-408c-b65a-bc3cde3b01d5-istiod-ca-cert\") pod \"e2e-praxis-b-f5dfb8-openshift-default-d59df9d89-28vsl\" (UID: \"b1598dab-80ac-408c-b65a-bc3cde3b01d5\") " pod="openshift-ingress/e2e-praxis-b-f5dfb8-openshift-default-d59df9d89-28vsl"
Oct 30 19:20:48.094052 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:20:48.094039    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b1598dab-80ac-408c-b65a-bc3cde3b01d5-istio-podinfo\") pod \"e2e-praxis-b-f5dfb8-openshift-default-d59df9d89-28vsl\" (UID: \"b1598dab-80ac-408c-b65a-bc3cde3b01d5\") " pod="openshift-ingress/e2e-praxis-b-f5dfb8-openshift-default-d59df9d89-28vsl"
Oct 30 19:20:48.094091 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:20:48.094062    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kspg2\" (UniqueName: \"kubernetes.io/projected/b1598dab-80ac-408c-b65a-bc3cde3b01d5-kube-api-access-kspg2\") pod \"e2e-praxis-b-f5dfb8-openshift-default-d59df9d89-28vsl\" (UID: \"b1598dab-80ac-408c-b65a-bc3cde3b01d5\") " pod="openshift-ingress/e2e-praxis-b-f5dfb8-openshift-default-d59df9d89-28vsl"
Oct 30 19:20:48.195461 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:20:48.195429    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b1598dab-80ac-408c-b65a-bc3cde3b01d5-istio-envoy\") pod \"e2e-praxis-b-f5dfb8-openshift-default-d59df9d89-28vsl\" (UID: \"b1598dab-80ac-408c-b65a-bc3cde3b01d5\") " pod="openshift-ingress/e2e-praxis-b-f5dfb8-openshift-default-d59df9d89-28vsl"
Oct 30 19:20:48.196095 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:20:48.195507    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1598dab-80ac-408c-b65a-bc3cde3b01d5-credential-socket\") pod \"e2e-praxis-b-f5dfb8-openshift-default-d59df9d89-28vsl\" (UID: \"b1598dab-80ac-408c-b65a-bc3cde3b01d5\") " pod="openshift-ingress/e2e-praxis-b-f5dfb8-openshift-default-d59df9d89-28vsl"
Oct 30 19:20:48.196095 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:20:48.195561    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b1598dab-80ac-408c-b65a-bc3cde3b01d5-istio-token\") pod \"e2e-praxis-b-f5dfb8-openshift-default-d59df9d89-28vsl\" (UID: \"b1598dab-80ac-408c-b65a-bc3cde3b01d5\") " pod="openshift-ingress/e2e-praxis-b-f5dfb8-openshift-default-d59df9d89-28vsl"
Oct 30 19:20:48.196095 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:20:48.195596    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1598dab-80ac-408c-b65a-bc3cde3b01d5-workload-socket\") pod \"e2e-praxis-b-f5dfb8-openshift-default-d59df9d89-28vsl\" (UID: \"b1598dab-80ac-408c-b65a-bc3cde3b01d5\") " pod="openshift-ingress/e2e-praxis-b-f5dfb8-openshift-default-d59df9d89-28vsl"
Oct 30 19:20:48.196095 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:20:48.195645    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b1598dab-80ac-408c-b65a-bc3cde3b01d5-workload-certs\") pod \"e2e-praxis-b-f5dfb8-openshift-default-d59df9d89-28vsl\" (UID: \"b1598dab-80ac-408c-b65a-bc3cde3b01d5\") " pod="openshift-ingress/e2e-praxis-b-f5dfb8-openshift-default-d59df9d89-28vsl"
Oct 30 19:20:48.196095 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:20:48.195984    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1598dab-80ac-408c-b65a-bc3cde3b01d5-credential-socket\") pod \"e2e-praxis-b-f5dfb8-openshift-default-d59df9d89-28vsl\" (UID: \"b1598dab-80ac-408c-b65a-bc3cde3b01d5\") " pod="openshift-ingress/e2e-praxis-b-f5dfb8-openshift-default-d59df9d89-28vsl"
Oct 30 19:20:48.196095 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:20:48.196029    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b1598dab-80ac-408c-b65a-bc3cde3b01d5-workload-certs\") pod \"e2e-praxis-b-f5dfb8-openshift-default-d59df9d89-28vsl\" (UID: \"b1598dab-80ac-408c-b65a-bc3cde3b01d5\") " pod="openshift-ingress/e2e-praxis-b-f5dfb8-openshift-default-d59df9d89-28vsl"
Oct 30 19:20:48.196717 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:20:48.196110    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1598dab-80ac-408c-b65a-bc3cde3b01d5-workload-socket\") pod \"e2e-praxis-b-f5dfb8-openshift-default-d59df9d89-28vsl\" (UID: \"b1598dab-80ac-408c-b65a-bc3cde3b01d5\") " pod="openshift-ingress/e2e-praxis-b-f5dfb8-openshift-default-d59df9d89-28vsl"
Oct 30 19:20:48.196717 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:20:48.196372    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b1598dab-80ac-408c-b65a-bc3cde3b01d5-istio-data\") pod \"e2e-praxis-b-f5dfb8-openshift-default-d59df9d89-28vsl\" (UID: \"b1598dab-80ac-408c-b65a-bc3cde3b01d5\") " pod="openshift-ingress/e2e-praxis-b-f5dfb8-openshift-default-d59df9d89-28vsl"
Oct 30 19:20:48.196717 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:20:48.196413    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b1598dab-80ac-408c-b65a-bc3cde3b01d5-istiod-ca-cert\") pod \"e2e-praxis-b-f5dfb8-openshift-default-d59df9d89-28vsl\" (UID: \"b1598dab-80ac-408c-b65a-bc3cde3b01d5\") " pod="openshift-ingress/e2e-praxis-b-f5dfb8-openshift-default-d59df9d89-28vsl"
Oct 30 19:20:48.196717 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:20:48.196475    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b1598dab-80ac-408c-b65a-bc3cde3b01d5-istio-podinfo\") pod \"e2e-praxis-b-f5dfb8-openshift-default-d59df9d89-28vsl\" (UID: \"b1598dab-80ac-408c-b65a-bc3cde3b01d5\") " pod="openshift-ingress/e2e-praxis-b-f5dfb8-openshift-default-d59df9d89-28vsl"
Oct 30 19:20:48.196717 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:20:48.196490    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kspg2\" (UniqueName: \"kubernetes.io/projected/b1598dab-80ac-408c-b65a-bc3cde3b01d5-kube-api-access-kspg2\") pod \"e2e-praxis-b-f5dfb8-openshift-default-d59df9d89-28vsl\" (UID: \"b1598dab-80ac-408c-b65a-bc3cde3b01d5\") " pod="openshift-ingress/e2e-praxis-b-f5dfb8-openshift-default-d59df9d89-28vsl"
Oct 30 19:20:48.196717 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:20:48.196710    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b1598dab-80ac-408c-b65a-bc3cde3b01d5-istio-data\") pod \"e2e-praxis-b-f5dfb8-openshift-default-d59df9d89-28vsl\" (UID: \"b1598dab-80ac-408c-b65a-bc3cde3b01d5\") " pod="openshift-ingress/e2e-praxis-b-f5dfb8-openshift-default-d59df9d89-28vsl"
Oct 30 19:20:48.197526 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:20:48.197151    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b1598dab-80ac-408c-b65a-bc3cde3b01d5-istiod-ca-cert\") pod \"e2e-praxis-b-f5dfb8-openshift-default-d59df9d89-28vsl\" (UID: \"b1598dab-80ac-408c-b65a-bc3cde3b01d5\") " pod="openshift-ingress/e2e-praxis-b-f5dfb8-openshift-default-d59df9d89-28vsl"
Oct 30 19:20:48.198054 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:20:48.198035    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b1598dab-80ac-408c-b65a-bc3cde3b01d5-istio-envoy\") pod \"e2e-praxis-b-f5dfb8-openshift-default-d59df9d89-28vsl\" (UID: \"b1598dab-80ac-408c-b65a-bc3cde3b01d5\") " pod="openshift-ingress/e2e-praxis-b-f5dfb8-openshift-default-d59df9d89-28vsl"
Oct 30 19:20:48.199005 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:20:48.198984    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b1598dab-80ac-408c-b65a-bc3cde3b01d5-istio-podinfo\") pod \"e2e-praxis-b-f5dfb8-openshift-default-d59df9d89-28vsl\" (UID: \"b1598dab-80ac-408c-b65a-bc3cde3b01d5\") " pod="openshift-ingress/e2e-praxis-b-f5dfb8-openshift-default-d59df9d89-28vsl"
Oct 30 19:20:48.258693 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:20:48.258615    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b1598dab-80ac-408c-b65a-bc3cde3b01d5-istio-token\") pod \"e2e-praxis-b-f5dfb8-openshift-default-d59df9d89-28vsl\" (UID: \"b1598dab-80ac-408c-b65a-bc3cde3b01d5\") " pod="openshift-ingress/e2e-praxis-b-f5dfb8-openshift-default-d59df9d89-28vsl"
Oct 30 19:20:48.259650 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:20:48.259626    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kspg2\" (UniqueName: \"kubernetes.io/projected/b1598dab-80ac-408c-b65a-bc3cde3b01d5-kube-api-access-kspg2\") pod \"e2e-praxis-b-f5dfb8-openshift-default-d59df9d89-28vsl\" (UID: \"b1598dab-80ac-408c-b65a-bc3cde3b01d5\") " pod="openshift-ingress/e2e-praxis-b-f5dfb8-openshift-default-d59df9d89-28vsl"
Oct 30 19:20:48.296816 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:20:48.296788    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-b-f5dfb8-openshift-default-d59df9d89-28vsl"
Oct 30 19:20:48.742098 ip-10-0-134-46 kubenswrapper[4888]: W1030 19:20:48.742066    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1598dab_80ac_408c_b65a_bc3cde3b01d5.slice/crio-f99dc4ffc737b109e772a2d6339755ee641489847ce862cfea10dab82b4e8ea8 WatchSource:0}: Error finding container f99dc4ffc737b109e772a2d6339755ee641489847ce862cfea10dab82b4e8ea8: Status 404 returned error can't find the container with id f99dc4ffc737b109e772a2d6339755ee641489847ce862cfea10dab82b4e8ea8
Oct 30 19:20:48.742531 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:20:48.742511    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-praxis-b-f5dfb8-openshift-default-d59df9d89-28vsl"]
Oct 30 19:20:48.745187 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:20:48.745158    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:20:48.745277 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:20:48.745222    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:20:48.745277 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:20:48.745267    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:20:49.457851 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:20:49.457819    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-b-f5dfb8-openshift-default-d59df9d89-28vsl" event={"ID":"b1598dab-80ac-408c-b65a-bc3cde3b01d5","Type":"ContainerStarted","Data":"fb2a49e557e558552317c215f316850d9f2b0f5845a5f4702de557d71c99469d"}
Oct 30 19:20:49.458248 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:20:49.457869    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-b-f5dfb8-openshift-default-d59df9d89-28vsl" event={"ID":"b1598dab-80ac-408c-b65a-bc3cde3b01d5","Type":"ContainerStarted","Data":"f99dc4ffc737b109e772a2d6339755ee641489847ce862cfea10dab82b4e8ea8"}
Oct 30 19:20:49.634516 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:20:49.634452    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-praxis-b-f5dfb8-openshift-default-d59df9d89-28vsl" podStartSLOduration=2.634427238 podStartE2EDuration="2.634427238s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:20:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:20:49.63355091 +0000 UTC m=+2813.880956936" watchObservedRunningTime="2026-10-30 19:20:49.634427238 +0000 UTC m=+2813.881833264"
Oct 30 19:20:50.298071 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:20:50.298027    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-praxis-b-f5dfb8-openshift-default-d59df9d89-28vsl"
Oct 30 19:20:50.320563 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:20:50.320530    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-praxis-b-f5dfb8-openshift-default-d59df9d89-28vsl"
Oct 30 19:20:50.466503 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:20:50.466472    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-praxis-b-f5dfb8-openshift-default-d59df9d89-28vsl"
Oct 30 19:20:50.467476 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:20:50.467453    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-praxis-b-f5dfb8-openshift-default-d59df9d89-28vsl"
Oct 30 19:21:05.619231 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:21:05.619148    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-jbsw6" containerName="extractor"
Oct 30 19:21:11.730115 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:21:11.730083    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w0-a56c7c-openshift-default-5db77d74cf-jjcdj"]
Oct 30 19:21:11.735561 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:21:11.735532    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w0-a56c7c-openshift-default-5db77d74cf-jjcdj"
Oct 30 19:21:11.853306 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:21:11.830923    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/36712903-e111-4fb5-a600-9b666b1d1c9f-istio-envoy\") pod \"e2e-shared-b-w0-a56c7c-openshift-default-5db77d74cf-jjcdj\" (UID: \"36712903-e111-4fb5-a600-9b666b1d1c9f\") " pod="openshift-ingress/e2e-shared-b-w0-a56c7c-openshift-default-5db77d74cf-jjcdj"
Oct 30 19:21:11.854422 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:21:11.854388    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/36712903-e111-4fb5-a600-9b666b1d1c9f-credential-socket\") pod \"e2e-shared-b-w0-a56c7c-openshift-default-5db77d74cf-jjcdj\" (UID: \"36712903-e111-4fb5-a600-9b666b1d1c9f\") " pod="openshift-ingress/e2e-shared-b-w0-a56c7c-openshift-default-5db77d74cf-jjcdj"
Oct 30 19:21:11.854601 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:21:11.854461    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/36712903-e111-4fb5-a600-9b666b1d1c9f-workload-certs\") pod \"e2e-shared-b-w0-a56c7c-openshift-default-5db77d74cf-jjcdj\" (UID: \"36712903-e111-4fb5-a600-9b666b1d1c9f\") " pod="openshift-ingress/e2e-shared-b-w0-a56c7c-openshift-default-5db77d74cf-jjcdj"
Oct 30 19:21:11.854601 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:21:11.854485    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/36712903-e111-4fb5-a600-9b666b1d1c9f-istiod-ca-cert\") pod \"e2e-shared-b-w0-a56c7c-openshift-default-5db77d74cf-jjcdj\" (UID: \"36712903-e111-4fb5-a600-9b666b1d1c9f\") " pod="openshift-ingress/e2e-shared-b-w0-a56c7c-openshift-default-5db77d74cf-jjcdj"
Oct 30 19:21:11.854748 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:21:11.854627    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/36712903-e111-4fb5-a600-9b666b1d1c9f-istio-token\") pod \"e2e-shared-b-w0-a56c7c-openshift-default-5db77d74cf-jjcdj\" (UID: \"36712903-e111-4fb5-a600-9b666b1d1c9f\") " pod="openshift-ingress/e2e-shared-b-w0-a56c7c-openshift-default-5db77d74cf-jjcdj"
Oct 30 19:21:11.854748 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:21:11.854694    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/36712903-e111-4fb5-a600-9b666b1d1c9f-istio-data\") pod \"e2e-shared-b-w0-a56c7c-openshift-default-5db77d74cf-jjcdj\" (UID: \"36712903-e111-4fb5-a600-9b666b1d1c9f\") " pod="openshift-ingress/e2e-shared-b-w0-a56c7c-openshift-default-5db77d74cf-jjcdj"
Oct 30 19:21:11.854748 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:21:11.854718    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/36712903-e111-4fb5-a600-9b666b1d1c9f-workload-socket\") pod \"e2e-shared-b-w0-a56c7c-openshift-default-5db77d74cf-jjcdj\" (UID: \"36712903-e111-4fb5-a600-9b666b1d1c9f\") " pod="openshift-ingress/e2e-shared-b-w0-a56c7c-openshift-default-5db77d74cf-jjcdj"
Oct 30 19:21:11.854917 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:21:11.854791    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-psgqh\" (UniqueName: \"kubernetes.io/projected/36712903-e111-4fb5-a600-9b666b1d1c9f-kube-api-access-psgqh\") pod \"e2e-shared-b-w0-a56c7c-openshift-default-5db77d74cf-jjcdj\" (UID: \"36712903-e111-4fb5-a600-9b666b1d1c9f\") " pod="openshift-ingress/e2e-shared-b-w0-a56c7c-openshift-default-5db77d74cf-jjcdj"
Oct 30 19:21:11.854917 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:21:11.854833    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/36712903-e111-4fb5-a600-9b666b1d1c9f-istio-podinfo\") pod \"e2e-shared-b-w0-a56c7c-openshift-default-5db77d74cf-jjcdj\" (UID: \"36712903-e111-4fb5-a600-9b666b1d1c9f\") " pod="openshift-ingress/e2e-shared-b-w0-a56c7c-openshift-default-5db77d74cf-jjcdj"
Oct 30 19:21:11.855033 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:21:11.842709    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w0-a56c7c-openshift-default-dockercfg-d8mnz\""
Oct 30 19:21:11.877725 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:21:11.877685    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w0-a56c7c-openshift-default-5db77d74cf-jjcdj"]
Oct 30 19:21:11.956308 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:21:11.956259    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-psgqh\" (UniqueName: \"kubernetes.io/projected/36712903-e111-4fb5-a600-9b666b1d1c9f-kube-api-access-psgqh\") pod \"e2e-shared-b-w0-a56c7c-openshift-default-5db77d74cf-jjcdj\" (UID: \"36712903-e111-4fb5-a600-9b666b1d1c9f\") " pod="openshift-ingress/e2e-shared-b-w0-a56c7c-openshift-default-5db77d74cf-jjcdj"
Oct 30 19:21:11.956515 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:21:11.956351    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/36712903-e111-4fb5-a600-9b666b1d1c9f-istio-podinfo\") pod \"e2e-shared-b-w0-a56c7c-openshift-default-5db77d74cf-jjcdj\" (UID: \"36712903-e111-4fb5-a600-9b666b1d1c9f\") " pod="openshift-ingress/e2e-shared-b-w0-a56c7c-openshift-default-5db77d74cf-jjcdj"
Oct 30 19:21:11.956515 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:21:11.956476    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/36712903-e111-4fb5-a600-9b666b1d1c9f-istio-envoy\") pod \"e2e-shared-b-w0-a56c7c-openshift-default-5db77d74cf-jjcdj\" (UID: \"36712903-e111-4fb5-a600-9b666b1d1c9f\") " pod="openshift-ingress/e2e-shared-b-w0-a56c7c-openshift-default-5db77d74cf-jjcdj"
Oct 30 19:21:11.956515 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:21:11.956503    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/36712903-e111-4fb5-a600-9b666b1d1c9f-credential-socket\") pod \"e2e-shared-b-w0-a56c7c-openshift-default-5db77d74cf-jjcdj\" (UID: \"36712903-e111-4fb5-a600-9b666b1d1c9f\") " pod="openshift-ingress/e2e-shared-b-w0-a56c7c-openshift-default-5db77d74cf-jjcdj"
Oct 30 19:21:11.956688 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:21:11.956543    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/36712903-e111-4fb5-a600-9b666b1d1c9f-workload-certs\") pod \"e2e-shared-b-w0-a56c7c-openshift-default-5db77d74cf-jjcdj\" (UID: \"36712903-e111-4fb5-a600-9b666b1d1c9f\") " pod="openshift-ingress/e2e-shared-b-w0-a56c7c-openshift-default-5db77d74cf-jjcdj"
Oct 30 19:21:11.956688 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:21:11.956568    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/36712903-e111-4fb5-a600-9b666b1d1c9f-istiod-ca-cert\") pod \"e2e-shared-b-w0-a56c7c-openshift-default-5db77d74cf-jjcdj\" (UID: \"36712903-e111-4fb5-a600-9b666b1d1c9f\") " pod="openshift-ingress/e2e-shared-b-w0-a56c7c-openshift-default-5db77d74cf-jjcdj"
Oct 30 19:21:11.956822 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:21:11.956714    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/36712903-e111-4fb5-a600-9b666b1d1c9f-istio-token\") pod \"e2e-shared-b-w0-a56c7c-openshift-default-5db77d74cf-jjcdj\" (UID: \"36712903-e111-4fb5-a600-9b666b1d1c9f\") " pod="openshift-ingress/e2e-shared-b-w0-a56c7c-openshift-default-5db77d74cf-jjcdj"
Oct 30 19:21:11.956822 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:21:11.956807    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/36712903-e111-4fb5-a600-9b666b1d1c9f-istio-data\") pod \"e2e-shared-b-w0-a56c7c-openshift-default-5db77d74cf-jjcdj\" (UID: \"36712903-e111-4fb5-a600-9b666b1d1c9f\") " pod="openshift-ingress/e2e-shared-b-w0-a56c7c-openshift-default-5db77d74cf-jjcdj"
Oct 30 19:21:11.956928 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:21:11.956838    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/36712903-e111-4fb5-a600-9b666b1d1c9f-workload-socket\") pod \"e2e-shared-b-w0-a56c7c-openshift-default-5db77d74cf-jjcdj\" (UID: \"36712903-e111-4fb5-a600-9b666b1d1c9f\") " pod="openshift-ingress/e2e-shared-b-w0-a56c7c-openshift-default-5db77d74cf-jjcdj"
Oct 30 19:21:11.957268 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:21:11.957240    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/36712903-e111-4fb5-a600-9b666b1d1c9f-credential-socket\") pod \"e2e-shared-b-w0-a56c7c-openshift-default-5db77d74cf-jjcdj\" (UID: \"36712903-e111-4fb5-a600-9b666b1d1c9f\") " pod="openshift-ingress/e2e-shared-b-w0-a56c7c-openshift-default-5db77d74cf-jjcdj"
Oct 30 19:21:11.957440 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:21:11.957269    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/36712903-e111-4fb5-a600-9b666b1d1c9f-workload-socket\") pod \"e2e-shared-b-w0-a56c7c-openshift-default-5db77d74cf-jjcdj\" (UID: \"36712903-e111-4fb5-a600-9b666b1d1c9f\") " pod="openshift-ingress/e2e-shared-b-w0-a56c7c-openshift-default-5db77d74cf-jjcdj"
Oct 30 19:21:11.957515 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:21:11.957482    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/36712903-e111-4fb5-a600-9b666b1d1c9f-workload-certs\") pod \"e2e-shared-b-w0-a56c7c-openshift-default-5db77d74cf-jjcdj\" (UID: \"36712903-e111-4fb5-a600-9b666b1d1c9f\") " pod="openshift-ingress/e2e-shared-b-w0-a56c7c-openshift-default-5db77d74cf-jjcdj"
Oct 30 19:21:11.957576 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:21:11.957537    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/36712903-e111-4fb5-a600-9b666b1d1c9f-istio-data\") pod \"e2e-shared-b-w0-a56c7c-openshift-default-5db77d74cf-jjcdj\" (UID: \"36712903-e111-4fb5-a600-9b666b1d1c9f\") " pod="openshift-ingress/e2e-shared-b-w0-a56c7c-openshift-default-5db77d74cf-jjcdj"
Oct 30 19:21:11.958162 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:21:11.958140    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/36712903-e111-4fb5-a600-9b666b1d1c9f-istiod-ca-cert\") pod \"e2e-shared-b-w0-a56c7c-openshift-default-5db77d74cf-jjcdj\" (UID: \"36712903-e111-4fb5-a600-9b666b1d1c9f\") " pod="openshift-ingress/e2e-shared-b-w0-a56c7c-openshift-default-5db77d74cf-jjcdj"
Oct 30 19:21:11.959412 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:21:11.959383    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/36712903-e111-4fb5-a600-9b666b1d1c9f-istio-envoy\") pod \"e2e-shared-b-w0-a56c7c-openshift-default-5db77d74cf-jjcdj\" (UID: \"36712903-e111-4fb5-a600-9b666b1d1c9f\") " pod="openshift-ingress/e2e-shared-b-w0-a56c7c-openshift-default-5db77d74cf-jjcdj"
Oct 30 19:21:11.959644 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:21:11.959621    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/36712903-e111-4fb5-a600-9b666b1d1c9f-istio-podinfo\") pod \"e2e-shared-b-w0-a56c7c-openshift-default-5db77d74cf-jjcdj\" (UID: \"36712903-e111-4fb5-a600-9b666b1d1c9f\") " pod="openshift-ingress/e2e-shared-b-w0-a56c7c-openshift-default-5db77d74cf-jjcdj"
Oct 30 19:21:12.019645 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:21:12.019584    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/36712903-e111-4fb5-a600-9b666b1d1c9f-istio-token\") pod \"e2e-shared-b-w0-a56c7c-openshift-default-5db77d74cf-jjcdj\" (UID: \"36712903-e111-4fb5-a600-9b666b1d1c9f\") " pod="openshift-ingress/e2e-shared-b-w0-a56c7c-openshift-default-5db77d74cf-jjcdj"
Oct 30 19:21:12.019767 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:21:12.019669    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-psgqh\" (UniqueName: \"kubernetes.io/projected/36712903-e111-4fb5-a600-9b666b1d1c9f-kube-api-access-psgqh\") pod \"e2e-shared-b-w0-a56c7c-openshift-default-5db77d74cf-jjcdj\" (UID: \"36712903-e111-4fb5-a600-9b666b1d1c9f\") " pod="openshift-ingress/e2e-shared-b-w0-a56c7c-openshift-default-5db77d74cf-jjcdj"
Oct 30 19:21:12.060924 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:21:12.060890    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w0-a56c7c-openshift-default-5db77d74cf-jjcdj"
Oct 30 19:21:12.502009 ip-10-0-134-46 kubenswrapper[4888]: W1030 19:21:12.501977    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36712903_e111_4fb5_a600_9b666b1d1c9f.slice/crio-ccf737881f458817b3dc5efb3c8992192cd2b78aaac0bb5389d0110199f08bf0 WatchSource:0}: Error finding container ccf737881f458817b3dc5efb3c8992192cd2b78aaac0bb5389d0110199f08bf0: Status 404 returned error can't find the container with id ccf737881f458817b3dc5efb3c8992192cd2b78aaac0bb5389d0110199f08bf0
Oct 30 19:21:12.502736 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:21:12.502712    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w0-a56c7c-openshift-default-5db77d74cf-jjcdj"]
Oct 30 19:21:12.504980 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:21:12.504949    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:21:12.505070 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:21:12.505011    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:21:12.505070 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:21:12.505038    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:21:12.653779 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:21:12.653742    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w0-a56c7c-openshift-default-5db77d74cf-jjcdj" event={"ID":"36712903-e111-4fb5-a600-9b666b1d1c9f","Type":"ContainerStarted","Data":"3b56a8d77360e89020665adaeadd010c41bb43e32029d47ffb874b68bb7feb18"}
Oct 30 19:21:12.653923 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:21:12.653811    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w0-a56c7c-openshift-default-5db77d74cf-jjcdj" event={"ID":"36712903-e111-4fb5-a600-9b666b1d1c9f","Type":"ContainerStarted","Data":"ccf737881f458817b3dc5efb3c8992192cd2b78aaac0bb5389d0110199f08bf0"}
Oct 30 19:21:12.779490 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:21:12.779397    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w0-a56c7c-openshift-default-5db77d74cf-jjcdj" podStartSLOduration=1.779382874 podStartE2EDuration="1.779382874s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:21:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:21:12.777537633 +0000 UTC m=+2837.024943671" watchObservedRunningTime="2026-10-30 19:21:12.779382874 +0000 UTC m=+2837.026788898"
Oct 30 19:21:13.061121 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:21:13.061049    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w0-a56c7c-openshift-default-5db77d74cf-jjcdj"
Oct 30 19:21:13.067583 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:21:13.067560    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w0-a56c7c-openshift-default-5db77d74cf-jjcdj"
Oct 30 19:21:13.662517 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:21:13.662484    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w0-a56c7c-openshift-default-5db77d74cf-jjcdj"
Oct 30 19:21:13.663625 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:21:13.663610    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w0-a56c7c-openshift-default-5db77d74cf-jjcdj"
Oct 30 19:21:35.380846 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:21:35.380812    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-praxis-a-6d953c-openshift-default-5d96c8ccc7-5bjbr"]
Oct 30 19:21:35.381304 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:21:35.381101    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-praxis-a-6d953c-openshift-default-5d96c8ccc7-5bjbr" podUID="243ebea3-fb15-455c-8e02-f152b478d25a" containerName="istio-proxy" containerID="cri-o://16926ae7118d2ba519591d489787b6e3c04a299974b18efd7e9b50043fb7c705" gracePeriod=30
Oct 30 19:21:35.633085 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:21:35.633005    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-jbsw6" containerName="extractor"
Oct 30 19:21:40.667700 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:21:40.667674    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-a-6d953c-openshift-default-5d96c8ccc7-5bjbr"
Oct 30 19:21:40.737626 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:21:40.737601    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/243ebea3-fb15-455c-8e02-f152b478d25a-kube-api-access-hvtvd\" (UniqueName: \"kubernetes.io/projected/243ebea3-fb15-455c-8e02-f152b478d25a-kube-api-access-hvtvd\") pod \"243ebea3-fb15-455c-8e02-f152b478d25a\" (UID: \"243ebea3-fb15-455c-8e02-f152b478d25a\") "
Oct 30 19:21:40.737810 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:21:40.737638    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/243ebea3-fb15-455c-8e02-f152b478d25a-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/243ebea3-fb15-455c-8e02-f152b478d25a-istio-podinfo\") pod \"243ebea3-fb15-455c-8e02-f152b478d25a\" (UID: \"243ebea3-fb15-455c-8e02-f152b478d25a\") "
Oct 30 19:21:40.737810 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:21:40.737689    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/243ebea3-fb15-455c-8e02-f152b478d25a-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/243ebea3-fb15-455c-8e02-f152b478d25a-workload-certs\") pod \"243ebea3-fb15-455c-8e02-f152b478d25a\" (UID: \"243ebea3-fb15-455c-8e02-f152b478d25a\") "
Oct 30 19:21:40.737810 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:21:40.737708    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/243ebea3-fb15-455c-8e02-f152b478d25a-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/243ebea3-fb15-455c-8e02-f152b478d25a-istiod-ca-cert\") pod \"243ebea3-fb15-455c-8e02-f152b478d25a\" (UID: \"243ebea3-fb15-455c-8e02-f152b478d25a\") "
Oct 30 19:21:40.737810 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:21:40.737739    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/243ebea3-fb15-455c-8e02-f152b478d25a-istio-token\" (UniqueName: \"kubernetes.io/projected/243ebea3-fb15-455c-8e02-f152b478d25a-istio-token\") pod \"243ebea3-fb15-455c-8e02-f152b478d25a\" (UID: \"243ebea3-fb15-455c-8e02-f152b478d25a\") "
Oct 30 19:21:40.737810 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:21:40.737798    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/243ebea3-fb15-455c-8e02-f152b478d25a-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/243ebea3-fb15-455c-8e02-f152b478d25a-istio-envoy\") pod \"243ebea3-fb15-455c-8e02-f152b478d25a\" (UID: \"243ebea3-fb15-455c-8e02-f152b478d25a\") "
Oct 30 19:21:40.738099 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:21:40.737830    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/243ebea3-fb15-455c-8e02-f152b478d25a-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/243ebea3-fb15-455c-8e02-f152b478d25a-istio-data\") pod \"243ebea3-fb15-455c-8e02-f152b478d25a\" (UID: \"243ebea3-fb15-455c-8e02-f152b478d25a\") "
Oct 30 19:21:40.738099 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:21:40.737898    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/243ebea3-fb15-455c-8e02-f152b478d25a-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/243ebea3-fb15-455c-8e02-f152b478d25a-credential-socket\") pod \"243ebea3-fb15-455c-8e02-f152b478d25a\" (UID: \"243ebea3-fb15-455c-8e02-f152b478d25a\") "
Oct 30 19:21:40.738099 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:21:40.738037    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/243ebea3-fb15-455c-8e02-f152b478d25a-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/243ebea3-fb15-455c-8e02-f152b478d25a-workload-socket\") pod \"243ebea3-fb15-455c-8e02-f152b478d25a\" (UID: \"243ebea3-fb15-455c-8e02-f152b478d25a\") "
Oct 30 19:21:40.738272 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:21:40.738166    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/243ebea3-fb15-455c-8e02-f152b478d25a-istio-data" pod "243ebea3-fb15-455c-8e02-f152b478d25a" (UID: "243ebea3-fb15-455c-8e02-f152b478d25a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:21:40.738272 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:21:40.738186    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/243ebea3-fb15-455c-8e02-f152b478d25a-istiod-ca-cert" pod "243ebea3-fb15-455c-8e02-f152b478d25a" (UID: "243ebea3-fb15-455c-8e02-f152b478d25a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:21:40.738407 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:21:40.738374    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/243ebea3-fb15-455c-8e02-f152b478d25a-workload-certs" pod "243ebea3-fb15-455c-8e02-f152b478d25a" (UID: "243ebea3-fb15-455c-8e02-f152b478d25a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:21:40.738464 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:21:40.738404    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/243ebea3-fb15-455c-8e02-f152b478d25a-workload-socket" pod "243ebea3-fb15-455c-8e02-f152b478d25a" (UID: "243ebea3-fb15-455c-8e02-f152b478d25a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:21:40.738501 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:21:40.738456    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/243ebea3-fb15-455c-8e02-f152b478d25a-credential-socket" pod "243ebea3-fb15-455c-8e02-f152b478d25a" (UID: "243ebea3-fb15-455c-8e02-f152b478d25a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:21:40.738896 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:21:40.738875    4888 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/243ebea3-fb15-455c-8e02-f152b478d25a-workload-socket\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:21:40.738989 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:21:40.738899    4888 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/243ebea3-fb15-455c-8e02-f152b478d25a-workload-certs\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:21:40.738989 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:21:40.738912    4888 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/243ebea3-fb15-455c-8e02-f152b478d25a-istiod-ca-cert\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:21:40.738989 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:21:40.738924    4888 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/243ebea3-fb15-455c-8e02-f152b478d25a-istio-data\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:21:40.738989 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:21:40.738937    4888 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/243ebea3-fb15-455c-8e02-f152b478d25a-credential-socket\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:21:40.740111 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:21:40.740083    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/243ebea3-fb15-455c-8e02-f152b478d25a-kube-api-access-hvtvd" pod "243ebea3-fb15-455c-8e02-f152b478d25a" (UID: "243ebea3-fb15-455c-8e02-f152b478d25a"). InnerVolumeSpecName "kube-api-access-hvtvd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:21:40.740358 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:21:40.740336    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/243ebea3-fb15-455c-8e02-f152b478d25a-istio-envoy" pod "243ebea3-fb15-455c-8e02-f152b478d25a" (UID: "243ebea3-fb15-455c-8e02-f152b478d25a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:21:40.740640 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:21:40.740612    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/243ebea3-fb15-455c-8e02-f152b478d25a-istio-token" pod "243ebea3-fb15-455c-8e02-f152b478d25a" (UID: "243ebea3-fb15-455c-8e02-f152b478d25a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:21:40.740725 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:21:40.740640    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/243ebea3-fb15-455c-8e02-f152b478d25a-istio-podinfo" pod "243ebea3-fb15-455c-8e02-f152b478d25a" (UID: "243ebea3-fb15-455c-8e02-f152b478d25a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:21:40.840796 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:21:40.840736    4888 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/243ebea3-fb15-455c-8e02-f152b478d25a-istio-token\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:21:40.840796 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:21:40.840773    4888 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/243ebea3-fb15-455c-8e02-f152b478d25a-istio-envoy\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:21:40.840796 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:21:40.840789    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hvtvd\" (UniqueName: \"kubernetes.io/projected/243ebea3-fb15-455c-8e02-f152b478d25a-kube-api-access-hvtvd\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:21:40.840796 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:21:40.840803    4888 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/243ebea3-fb15-455c-8e02-f152b478d25a-istio-podinfo\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:21:40.868622 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:21:40.868588    4888 generic.go:356] "Generic (PLEG): container finished" podID="243ebea3-fb15-455c-8e02-f152b478d25a" containerID="16926ae7118d2ba519591d489787b6e3c04a299974b18efd7e9b50043fb7c705" exitCode=0
Oct 30 19:21:40.868819 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:21:40.868658    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-a-6d953c-openshift-default-5d96c8ccc7-5bjbr"
Oct 30 19:21:40.868819 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:21:40.868676    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-6d953c-openshift-default-5d96c8ccc7-5bjbr" event={"ID":"243ebea3-fb15-455c-8e02-f152b478d25a","Type":"ContainerDied","Data":"16926ae7118d2ba519591d489787b6e3c04a299974b18efd7e9b50043fb7c705"}
Oct 30 19:21:40.868819 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:21:40.868746    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-6d953c-openshift-default-5d96c8ccc7-5bjbr" event={"ID":"243ebea3-fb15-455c-8e02-f152b478d25a","Type":"ContainerDied","Data":"222066a34813dab087672f8746c842491bee38035ac710132d4501ce2fd4804a"}
Oct 30 19:21:40.868819 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:21:40.868779    4888 scope.go:122] "RemoveContainer" containerID="16926ae7118d2ba519591d489787b6e3c04a299974b18efd7e9b50043fb7c705"
Oct 30 19:21:40.882811 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:21:40.882786    4888 scope.go:122] "RemoveContainer" containerID="ad1d864505ae3ead0652205ede92b8c31677aa81f8302d590f340688ad2e58fd"
Oct 30 19:21:40.893609 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:21:40.893590    4888 scope.go:122] "RemoveContainer" containerID="16926ae7118d2ba519591d489787b6e3c04a299974b18efd7e9b50043fb7c705"
Oct 30 19:21:40.893985 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:21:40.893949    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"16926ae7118d2ba519591d489787b6e3c04a299974b18efd7e9b50043fb7c705\": container with ID starting with 16926ae7118d2ba519591d489787b6e3c04a299974b18efd7e9b50043fb7c705 not found: ID does not exist" containerID="16926ae7118d2ba519591d489787b6e3c04a299974b18efd7e9b50043fb7c705"
Oct 30 19:21:40.894097 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:21:40.893995    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"16926ae7118d2ba519591d489787b6e3c04a299974b18efd7e9b50043fb7c705"} err="failed to get container status \"16926ae7118d2ba519591d489787b6e3c04a299974b18efd7e9b50043fb7c705\": rpc error: code = NotFound desc = could not find container \"16926ae7118d2ba519591d489787b6e3c04a299974b18efd7e9b50043fb7c705\": container with ID starting with 16926ae7118d2ba519591d489787b6e3c04a299974b18efd7e9b50043fb7c705 not found: ID does not exist"
Oct 30 19:21:40.894097 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:21:40.894023    4888 scope.go:122] "RemoveContainer" containerID="ad1d864505ae3ead0652205ede92b8c31677aa81f8302d590f340688ad2e58fd"
Oct 30 19:21:40.894379 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:21:40.894356    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ad1d864505ae3ead0652205ede92b8c31677aa81f8302d590f340688ad2e58fd\": container with ID starting with ad1d864505ae3ead0652205ede92b8c31677aa81f8302d590f340688ad2e58fd not found: ID does not exist" containerID="ad1d864505ae3ead0652205ede92b8c31677aa81f8302d590f340688ad2e58fd"
Oct 30 19:21:40.894482 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:21:40.894386    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ad1d864505ae3ead0652205ede92b8c31677aa81f8302d590f340688ad2e58fd"} err="failed to get container status \"ad1d864505ae3ead0652205ede92b8c31677aa81f8302d590f340688ad2e58fd\": rpc error: code = NotFound desc = could not find container \"ad1d864505ae3ead0652205ede92b8c31677aa81f8302d590f340688ad2e58fd\": container with ID starting with ad1d864505ae3ead0652205ede92b8c31677aa81f8302d590f340688ad2e58fd not found: ID does not exist"
Oct 30 19:21:41.039570 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:21:41.039487    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-praxis-a-6d953c-openshift-default-5d96c8ccc7-5bjbr"]
Oct 30 19:21:41.043677 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:21:41.043656    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-praxis-a-6d953c-openshift-default-5d96c8ccc7-5bjbr"]
Oct 30 19:21:41.891079 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:21:41.891045    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="243ebea3-fb15-455c-8e02-f152b478d25a" path="/var/lib/kubelet/pods/243ebea3-fb15-455c-8e02-f152b478d25a/volumes"
Oct 30 19:21:49.631247 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:21:49.631218    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-rjmdz"]
Oct 30 19:21:50.216352 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:21:50.216321    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-b-9f939e-548f4979cf-7wj42"]
Oct 30 19:21:55.216664 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:21:55.216624    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-9f939e-548f4979cf-7wj42" podUID="20f59c4c-20e2-4be8-a76d-d98911238024" containerName="payload-pre-processing" containerID="cri-o://58a3920867d71f158759b469adc7b2aa05741d112072fc1ac9c2ba0b496be66f" gracePeriod=25
Oct 30 19:21:55.483475 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:21:55.483423    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-9f939e-548f4979cf-7wj42"
Oct 30 19:21:55.662467 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:21:55.662426    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/20f59c4c-20e2-4be8-a76d-d98911238024-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/20f59c4c-20e2-4be8-a76d-d98911238024-plugins-config-volume\") pod \"20f59c4c-20e2-4be8-a76d-d98911238024\" (UID: \"20f59c4c-20e2-4be8-a76d-d98911238024\") "
Oct 30 19:21:55.662640 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:21:55.662606    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/20f59c4c-20e2-4be8-a76d-d98911238024-kube-api-access-fqzqm\" (UniqueName: \"kubernetes.io/projected/20f59c4c-20e2-4be8-a76d-d98911238024-kube-api-access-fqzqm\") pod \"20f59c4c-20e2-4be8-a76d-d98911238024\" (UID: \"20f59c4c-20e2-4be8-a76d-d98911238024\") "
Oct 30 19:21:55.662904 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:21:55.662875    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/20f59c4c-20e2-4be8-a76d-d98911238024-plugins-config-volume" pod "20f59c4c-20e2-4be8-a76d-d98911238024" (UID: "20f59c4c-20e2-4be8-a76d-d98911238024"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:21:55.663175 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:21:55.663148    4888 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/20f59c4c-20e2-4be8-a76d-d98911238024-plugins-config-volume\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:21:55.665409 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:21:55.665381    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/20f59c4c-20e2-4be8-a76d-d98911238024-kube-api-access-fqzqm" pod "20f59c4c-20e2-4be8-a76d-d98911238024" (UID: "20f59c4c-20e2-4be8-a76d-d98911238024"). InnerVolumeSpecName "kube-api-access-fqzqm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:21:55.764037 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:21:55.763955    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fqzqm\" (UniqueName: \"kubernetes.io/projected/20f59c4c-20e2-4be8-a76d-d98911238024-kube-api-access-fqzqm\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:21:55.992709 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:21:55.992673    4888 generic.go:356] "Generic (PLEG): container finished" podID="20f59c4c-20e2-4be8-a76d-d98911238024" containerID="58a3920867d71f158759b469adc7b2aa05741d112072fc1ac9c2ba0b496be66f" exitCode=0
Oct 30 19:21:55.992897 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:21:55.992717    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-9f939e-548f4979cf-7wj42"
Oct 30 19:21:55.992897 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:21:55.992763    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-9f939e-548f4979cf-7wj42" event={"ID":"20f59c4c-20e2-4be8-a76d-d98911238024","Type":"ContainerDied","Data":"58a3920867d71f158759b469adc7b2aa05741d112072fc1ac9c2ba0b496be66f"}
Oct 30 19:21:55.992897 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:21:55.992823    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-9f939e-548f4979cf-7wj42" event={"ID":"20f59c4c-20e2-4be8-a76d-d98911238024","Type":"ContainerDied","Data":"d5fb09899d9d838f5fd3570dcb1ce833b5ba380ec5dcf83fec07d14ac6c5d2ad"}
Oct 30 19:21:55.992897 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:21:55.992850    4888 scope.go:122] "RemoveContainer" containerID="58a3920867d71f158759b469adc7b2aa05741d112072fc1ac9c2ba0b496be66f"
Oct 30 19:21:56.003109 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:21:56.003092    4888 scope.go:122] "RemoveContainer" containerID="58a3920867d71f158759b469adc7b2aa05741d112072fc1ac9c2ba0b496be66f"
Oct 30 19:21:56.003400 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:21:56.003381    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"58a3920867d71f158759b469adc7b2aa05741d112072fc1ac9c2ba0b496be66f\": container with ID starting with 58a3920867d71f158759b469adc7b2aa05741d112072fc1ac9c2ba0b496be66f not found: ID does not exist" containerID="58a3920867d71f158759b469adc7b2aa05741d112072fc1ac9c2ba0b496be66f"
Oct 30 19:21:56.003474 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:21:56.003412    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"58a3920867d71f158759b469adc7b2aa05741d112072fc1ac9c2ba0b496be66f"} err="failed to get container status \"58a3920867d71f158759b469adc7b2aa05741d112072fc1ac9c2ba0b496be66f\": rpc error: code = NotFound desc = could not find container \"58a3920867d71f158759b469adc7b2aa05741d112072fc1ac9c2ba0b496be66f\": container with ID starting with 58a3920867d71f158759b469adc7b2aa05741d112072fc1ac9c2ba0b496be66f not found: ID does not exist"
Oct 30 19:21:56.170225 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:21:56.170189    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-b-9f939e-548f4979cf-7wj42"]
Oct 30 19:21:56.219314 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:21:56.219266    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-b-9f939e-548f4979cf-7wj42"]
Oct 30 19:21:57.023760 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:21:57.023725    4888 scope.go:122] "RemoveContainer" containerID="1ca565f82145a722b9ea8c57d1c00fc342746c79052c6653e8f5389d566f7531"
Oct 30 19:21:57.891270 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:21:57.891234    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="20f59c4c-20e2-4be8-a76d-d98911238024" path="/var/lib/kubelet/pods/20f59c4c-20e2-4be8-a76d-d98911238024/volumes"
Oct 30 19:22:05.628663 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:22:05.628625    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-jbsw6" containerName="extractor"
Oct 30 19:22:06.682495 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:06.682459    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-9f939e-786ddbd868-w88zc"]
Oct 30 19:22:06.682938 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:06.682700    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-9f939e-786ddbd868-w88zc" podUID="38709aaf-ee31-44d1-a203-2de0dfa70a30" containerName="maas-api" containerID="cri-o://4fd2799d6e135dc8ee85b54bde30c1d5084e8554f798e43017c0e42b1ae62c73" gracePeriod=30
Oct 30 19:22:06.982786 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:06.982757    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-9f939e-786ddbd868-w88zc"
Oct 30 19:22:07.093249 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:07.093214    4888 generic.go:356] "Generic (PLEG): container finished" podID="38709aaf-ee31-44d1-a203-2de0dfa70a30" containerID="4fd2799d6e135dc8ee85b54bde30c1d5084e8554f798e43017c0e42b1ae62c73" exitCode=0
Oct 30 19:22:07.093634 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:07.093614    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-9f939e-786ddbd868-w88zc" event={"ID":"38709aaf-ee31-44d1-a203-2de0dfa70a30","Type":"ContainerDied","Data":"4fd2799d6e135dc8ee85b54bde30c1d5084e8554f798e43017c0e42b1ae62c73"}
Oct 30 19:22:07.093917 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:07.093901    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-9f939e-786ddbd868-w88zc" event={"ID":"38709aaf-ee31-44d1-a203-2de0dfa70a30","Type":"ContainerDied","Data":"13cdd55b24cec0423f447e906e321d4505c9aeb6b49aec796be568199889af43"}
Oct 30 19:22:07.094046 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:07.094034    4888 scope.go:122] "RemoveContainer" containerID="4fd2799d6e135dc8ee85b54bde30c1d5084e8554f798e43017c0e42b1ae62c73"
Oct 30 19:22:07.094226 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:07.093848    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-9f939e-786ddbd868-w88zc"
Oct 30 19:22:07.124931 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:07.124461    4888 scope.go:122] "RemoveContainer" containerID="4fd2799d6e135dc8ee85b54bde30c1d5084e8554f798e43017c0e42b1ae62c73"
Oct 30 19:22:07.128038 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:22:07.128008    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4fd2799d6e135dc8ee85b54bde30c1d5084e8554f798e43017c0e42b1ae62c73\": container with ID starting with 4fd2799d6e135dc8ee85b54bde30c1d5084e8554f798e43017c0e42b1ae62c73 not found: ID does not exist" containerID="4fd2799d6e135dc8ee85b54bde30c1d5084e8554f798e43017c0e42b1ae62c73"
Oct 30 19:22:07.128264 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:07.128233    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4fd2799d6e135dc8ee85b54bde30c1d5084e8554f798e43017c0e42b1ae62c73"} err="failed to get container status \"4fd2799d6e135dc8ee85b54bde30c1d5084e8554f798e43017c0e42b1ae62c73\": rpc error: code = NotFound desc = could not find container \"4fd2799d6e135dc8ee85b54bde30c1d5084e8554f798e43017c0e42b1ae62c73\": container with ID starting with 4fd2799d6e135dc8ee85b54bde30c1d5084e8554f798e43017c0e42b1ae62c73 not found: ID does not exist"
Oct 30 19:22:07.128668 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:07.128647    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/38709aaf-ee31-44d1-a203-2de0dfa70a30-metrics-certs\" (UniqueName: \"kubernetes.io/secret/38709aaf-ee31-44d1-a203-2de0dfa70a30-metrics-certs\") pod \"38709aaf-ee31-44d1-a203-2de0dfa70a30\" (UID: \"38709aaf-ee31-44d1-a203-2de0dfa70a30\") "
Oct 30 19:22:07.128842 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:07.128822    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/38709aaf-ee31-44d1-a203-2de0dfa70a30-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/38709aaf-ee31-44d1-a203-2de0dfa70a30-openshift-service-ca\") pod \"38709aaf-ee31-44d1-a203-2de0dfa70a30\" (UID: \"38709aaf-ee31-44d1-a203-2de0dfa70a30\") "
Oct 30 19:22:07.129227 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:07.129195    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/38709aaf-ee31-44d1-a203-2de0dfa70a30-openshift-service-ca" pod "38709aaf-ee31-44d1-a203-2de0dfa70a30" (UID: "38709aaf-ee31-44d1-a203-2de0dfa70a30"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:22:07.129374 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:07.129306    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/38709aaf-ee31-44d1-a203-2de0dfa70a30-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/38709aaf-ee31-44d1-a203-2de0dfa70a30-maas-api-tls\") pod \"38709aaf-ee31-44d1-a203-2de0dfa70a30\" (UID: \"38709aaf-ee31-44d1-a203-2de0dfa70a30\") "
Oct 30 19:22:07.129501 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:07.129483    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/38709aaf-ee31-44d1-a203-2de0dfa70a30-kube-api-access-4cg6w\" (UniqueName: \"kubernetes.io/projected/38709aaf-ee31-44d1-a203-2de0dfa70a30-kube-api-access-4cg6w\") pod \"38709aaf-ee31-44d1-a203-2de0dfa70a30\" (UID: \"38709aaf-ee31-44d1-a203-2de0dfa70a30\") "
Oct 30 19:22:07.130458 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:07.130430    4888 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/38709aaf-ee31-44d1-a203-2de0dfa70a30-openshift-service-ca\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:22:07.131802 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:07.131775    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/38709aaf-ee31-44d1-a203-2de0dfa70a30-metrics-certs" pod "38709aaf-ee31-44d1-a203-2de0dfa70a30" (UID: "38709aaf-ee31-44d1-a203-2de0dfa70a30"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:22:07.133084 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:07.133061    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/38709aaf-ee31-44d1-a203-2de0dfa70a30-kube-api-access-4cg6w" pod "38709aaf-ee31-44d1-a203-2de0dfa70a30" (UID: "38709aaf-ee31-44d1-a203-2de0dfa70a30"). InnerVolumeSpecName "kube-api-access-4cg6w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:22:07.133360 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:07.133344    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/38709aaf-ee31-44d1-a203-2de0dfa70a30-maas-api-tls" pod "38709aaf-ee31-44d1-a203-2de0dfa70a30" (UID: "38709aaf-ee31-44d1-a203-2de0dfa70a30"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:22:07.231550 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:07.231472    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4cg6w\" (UniqueName: \"kubernetes.io/projected/38709aaf-ee31-44d1-a203-2de0dfa70a30-kube-api-access-4cg6w\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:22:07.231550 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:07.231509    4888 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/38709aaf-ee31-44d1-a203-2de0dfa70a30-metrics-certs\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:22:07.231550 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:07.231548    4888 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/38709aaf-ee31-44d1-a203-2de0dfa70a30-maas-api-tls\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:22:07.524741 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:07.524660    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-9f939e-786ddbd868-w88zc"]
Oct 30 19:22:07.527686 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:07.527666    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-9f939e-786ddbd868-w88zc"]
Oct 30 19:22:07.904461 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:07.904408    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="38709aaf-ee31-44d1-a203-2de0dfa70a30" path="/var/lib/kubelet/pods/38709aaf-ee31-44d1-a203-2de0dfa70a30/volumes"
Oct 30 19:22:17.100327 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:17.100267    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-praxis-b-9f939e-openshift-default-97686d89c-dhvqt"]
Oct 30 19:22:17.100974 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:17.100548    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-praxis-b-9f939e-openshift-default-97686d89c-dhvqt" podUID="d077845d-f1ed-46c2-a420-9984ccbf7902" containerName="istio-proxy" containerID="cri-o://4bd56c5b0746648dab84aa26d44c02f39c21bbb6247eeb23cfa273ad2b681187" gracePeriod=30
Oct 30 19:22:19.587918 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:19.587877    4888 patch_prober.go:29] interesting pod/e2e-praxis-b-9f939e-openshift-default-97686d89c-dhvqt container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 19:22:19.588410 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:19.587979    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-praxis-b-9f939e-openshift-default-97686d89c-dhvqt" podUID="d077845d-f1ed-46c2-a420-9984ccbf7902" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 19:22:22.260272 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:22.260241    4888 generic.go:356] "Generic (PLEG): container finished" podID="d077845d-f1ed-46c2-a420-9984ccbf7902" containerID="4bd56c5b0746648dab84aa26d44c02f39c21bbb6247eeb23cfa273ad2b681187" exitCode=0
Oct 30 19:22:22.260671 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:22.260321    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-b-9f939e-openshift-default-97686d89c-dhvqt" event={"ID":"d077845d-f1ed-46c2-a420-9984ccbf7902","Type":"ContainerDied","Data":"4bd56c5b0746648dab84aa26d44c02f39c21bbb6247eeb23cfa273ad2b681187"}
Oct 30 19:22:22.260671 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:22.260398    4888 scope.go:122] "RemoveContainer" containerID="16751dc6ca4f77a859a81e67e7f88922ad68e52103486d441ab5609cef7f8936"
Oct 30 19:22:22.395942 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:22.395917    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-b-9f939e-openshift-default-97686d89c-dhvqt"
Oct 30 19:22:22.434406 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:22.434378    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d077845d-f1ed-46c2-a420-9984ccbf7902-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d077845d-f1ed-46c2-a420-9984ccbf7902-istio-envoy\") pod \"d077845d-f1ed-46c2-a420-9984ccbf7902\" (UID: \"d077845d-f1ed-46c2-a420-9984ccbf7902\") "
Oct 30 19:22:22.434572 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:22.434556    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d077845d-f1ed-46c2-a420-9984ccbf7902-istio-token\" (UniqueName: \"kubernetes.io/projected/d077845d-f1ed-46c2-a420-9984ccbf7902-istio-token\") pod \"d077845d-f1ed-46c2-a420-9984ccbf7902\" (UID: \"d077845d-f1ed-46c2-a420-9984ccbf7902\") "
Oct 30 19:22:22.434634 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:22.434599    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d077845d-f1ed-46c2-a420-9984ccbf7902-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d077845d-f1ed-46c2-a420-9984ccbf7902-istio-data\") pod \"d077845d-f1ed-46c2-a420-9984ccbf7902\" (UID: \"d077845d-f1ed-46c2-a420-9984ccbf7902\") "
Oct 30 19:22:22.434699 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:22.434635    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d077845d-f1ed-46c2-a420-9984ccbf7902-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d077845d-f1ed-46c2-a420-9984ccbf7902-istiod-ca-cert\") pod \"d077845d-f1ed-46c2-a420-9984ccbf7902\" (UID: \"d077845d-f1ed-46c2-a420-9984ccbf7902\") "
Oct 30 19:22:22.434699 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:22.434686    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d077845d-f1ed-46c2-a420-9984ccbf7902-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d077845d-f1ed-46c2-a420-9984ccbf7902-credential-socket\") pod \"d077845d-f1ed-46c2-a420-9984ccbf7902\" (UID: \"d077845d-f1ed-46c2-a420-9984ccbf7902\") "
Oct 30 19:22:22.434808 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:22.434724    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d077845d-f1ed-46c2-a420-9984ccbf7902-kube-api-access-v5xgv\" (UniqueName: \"kubernetes.io/projected/d077845d-f1ed-46c2-a420-9984ccbf7902-kube-api-access-v5xgv\") pod \"d077845d-f1ed-46c2-a420-9984ccbf7902\" (UID: \"d077845d-f1ed-46c2-a420-9984ccbf7902\") "
Oct 30 19:22:22.434808 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:22.434770    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d077845d-f1ed-46c2-a420-9984ccbf7902-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d077845d-f1ed-46c2-a420-9984ccbf7902-workload-certs\") pod \"d077845d-f1ed-46c2-a420-9984ccbf7902\" (UID: \"d077845d-f1ed-46c2-a420-9984ccbf7902\") "
Oct 30 19:22:22.434911 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:22.434805    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/d077845d-f1ed-46c2-a420-9984ccbf7902-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d077845d-f1ed-46c2-a420-9984ccbf7902-istio-podinfo\") pod \"d077845d-f1ed-46c2-a420-9984ccbf7902\" (UID: \"d077845d-f1ed-46c2-a420-9984ccbf7902\") "
Oct 30 19:22:22.434911 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:22.434873    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d077845d-f1ed-46c2-a420-9984ccbf7902-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d077845d-f1ed-46c2-a420-9984ccbf7902-workload-socket\") pod \"d077845d-f1ed-46c2-a420-9984ccbf7902\" (UID: \"d077845d-f1ed-46c2-a420-9984ccbf7902\") "
Oct 30 19:22:22.435033 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:22.434900    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d077845d-f1ed-46c2-a420-9984ccbf7902-istio-data" pod "d077845d-f1ed-46c2-a420-9984ccbf7902" (UID: "d077845d-f1ed-46c2-a420-9984ccbf7902"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:22:22.435213 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:22.435010    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d077845d-f1ed-46c2-a420-9984ccbf7902-credential-socket" pod "d077845d-f1ed-46c2-a420-9984ccbf7902" (UID: "d077845d-f1ed-46c2-a420-9984ccbf7902"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:22:22.435609 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:22.435472    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d077845d-f1ed-46c2-a420-9984ccbf7902-workload-certs" pod "d077845d-f1ed-46c2-a420-9984ccbf7902" (UID: "d077845d-f1ed-46c2-a420-9984ccbf7902"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:22:22.435609 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:22.435486    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d077845d-f1ed-46c2-a420-9984ccbf7902-workload-socket" pod "d077845d-f1ed-46c2-a420-9984ccbf7902" (UID: "d077845d-f1ed-46c2-a420-9984ccbf7902"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:22:22.436101 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:22.436066    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d077845d-f1ed-46c2-a420-9984ccbf7902-istiod-ca-cert" pod "d077845d-f1ed-46c2-a420-9984ccbf7902" (UID: "d077845d-f1ed-46c2-a420-9984ccbf7902"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:22:22.436375 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:22.436323    4888 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d077845d-f1ed-46c2-a420-9984ccbf7902-credential-socket\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:22:22.436375 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:22.436347    4888 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d077845d-f1ed-46c2-a420-9984ccbf7902-istio-data\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:22:22.437147 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:22.437021    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d077845d-f1ed-46c2-a420-9984ccbf7902-istio-envoy" pod "d077845d-f1ed-46c2-a420-9984ccbf7902" (UID: "d077845d-f1ed-46c2-a420-9984ccbf7902"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:22:22.437536 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:22.437512    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d077845d-f1ed-46c2-a420-9984ccbf7902-istio-token" pod "d077845d-f1ed-46c2-a420-9984ccbf7902" (UID: "d077845d-f1ed-46c2-a420-9984ccbf7902"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:22:22.437799 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:22.437779    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d077845d-f1ed-46c2-a420-9984ccbf7902-istio-podinfo" pod "d077845d-f1ed-46c2-a420-9984ccbf7902" (UID: "d077845d-f1ed-46c2-a420-9984ccbf7902"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:22:22.437883 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:22.437810    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d077845d-f1ed-46c2-a420-9984ccbf7902-kube-api-access-v5xgv" pod "d077845d-f1ed-46c2-a420-9984ccbf7902" (UID: "d077845d-f1ed-46c2-a420-9984ccbf7902"). InnerVolumeSpecName "kube-api-access-v5xgv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:22:22.537521 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:22.537496    4888 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d077845d-f1ed-46c2-a420-9984ccbf7902-workload-socket\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:22:22.537521 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:22.537517    4888 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d077845d-f1ed-46c2-a420-9984ccbf7902-istio-envoy\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:22:22.537679 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:22.537529    4888 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d077845d-f1ed-46c2-a420-9984ccbf7902-istio-token\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:22:22.537679 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:22.537538    4888 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d077845d-f1ed-46c2-a420-9984ccbf7902-istiod-ca-cert\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:22:22.537679 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:22.537546    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v5xgv\" (UniqueName: \"kubernetes.io/projected/d077845d-f1ed-46c2-a420-9984ccbf7902-kube-api-access-v5xgv\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:22:22.537679 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:22.537556    4888 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d077845d-f1ed-46c2-a420-9984ccbf7902-workload-certs\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:22:22.537679 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:22.537565    4888 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d077845d-f1ed-46c2-a420-9984ccbf7902-istio-podinfo\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:22:23.153929 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:23.153888    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-a56c7c-696c9f7cd-qm7sl"]
Oct 30 19:22:23.154575 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:23.154555    4888 cpu_manager.go:414] "containerMap: removing container" podUID="20f59c4c-20e2-4be8-a76d-d98911238024" containerName="payload-pre-processing"
Oct 30 19:22:23.154700 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:23.154575    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="20f59c4c-20e2-4be8-a76d-d98911238024" containerName="payload-pre-processing"
Oct 30 19:22:23.154700 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:23.154599    4888 cpu_manager.go:414] "containerMap: removing container" podUID="243ebea3-fb15-455c-8e02-f152b478d25a" containerName="istio-proxy"
Oct 30 19:22:23.154700 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:23.154607    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="243ebea3-fb15-455c-8e02-f152b478d25a" containerName="istio-proxy"
Oct 30 19:22:23.154700 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:23.154628    4888 cpu_manager.go:414] "containerMap: removing container" podUID="d077845d-f1ed-46c2-a420-9984ccbf7902" containerName="istio-proxy"
Oct 30 19:22:23.154700 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:23.154635    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d077845d-f1ed-46c2-a420-9984ccbf7902" containerName="istio-proxy"
Oct 30 19:22:23.154700 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:23.154649    4888 cpu_manager.go:414] "containerMap: removing container" podUID="d077845d-f1ed-46c2-a420-9984ccbf7902" containerName="istio-proxy"
Oct 30 19:22:23.154700 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:23.154656    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d077845d-f1ed-46c2-a420-9984ccbf7902" containerName="istio-proxy"
Oct 30 19:22:23.154700 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:23.154695    4888 cpu_manager.go:414] "containerMap: removing container" podUID="38709aaf-ee31-44d1-a203-2de0dfa70a30" containerName="maas-api"
Oct 30 19:22:23.154700 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:23.154702    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="38709aaf-ee31-44d1-a203-2de0dfa70a30" containerName="maas-api"
Oct 30 19:22:23.155130 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:23.154858    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="243ebea3-fb15-455c-8e02-f152b478d25a" containerName="istio-proxy"
Oct 30 19:22:23.155130 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:23.154876    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="38709aaf-ee31-44d1-a203-2de0dfa70a30" containerName="maas-api"
Oct 30 19:22:23.155130 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:23.154890    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="20f59c4c-20e2-4be8-a76d-d98911238024" containerName="payload-pre-processing"
Oct 30 19:22:23.155130 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:23.154900    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="243ebea3-fb15-455c-8e02-f152b478d25a" containerName="istio-proxy"
Oct 30 19:22:23.155130 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:23.154910    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="d077845d-f1ed-46c2-a420-9984ccbf7902" containerName="istio-proxy"
Oct 30 19:22:23.155130 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:23.154920    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="d077845d-f1ed-46c2-a420-9984ccbf7902" containerName="istio-proxy"
Oct 30 19:22:23.158857 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:23.158837    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-a56c7c-696c9f7cd-qm7sl"
Oct 30 19:22:23.216200 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:23.216178    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w0-a56c7c-serving-cert\""
Oct 30 19:22:23.218263 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:23.218243    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-a56c7c-696c9f7cd-qm7sl"]
Oct 30 19:22:23.243981 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:23.243956    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/05eefdb3-fc35-4d7a-b3be-20819a7b7aaf-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w0-a56c7c-696c9f7cd-qm7sl\" (UID: \"05eefdb3-fc35-4d7a-b3be-20819a7b7aaf\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-a56c7c-696c9f7cd-qm7sl"
Oct 30 19:22:23.244081 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:23.244002    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/05eefdb3-fc35-4d7a-b3be-20819a7b7aaf-maas-api-tls\") pod \"maas-api-e2e-shared-b-w0-a56c7c-696c9f7cd-qm7sl\" (UID: \"05eefdb3-fc35-4d7a-b3be-20819a7b7aaf\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-a56c7c-696c9f7cd-qm7sl"
Oct 30 19:22:23.244124 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:23.244090    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kjrfz\" (UniqueName: \"kubernetes.io/projected/05eefdb3-fc35-4d7a-b3be-20819a7b7aaf-kube-api-access-kjrfz\") pod \"maas-api-e2e-shared-b-w0-a56c7c-696c9f7cd-qm7sl\" (UID: \"05eefdb3-fc35-4d7a-b3be-20819a7b7aaf\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-a56c7c-696c9f7cd-qm7sl"
Oct 30 19:22:23.244239 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:23.244215    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/05eefdb3-fc35-4d7a-b3be-20819a7b7aaf-metrics-certs\") pod \"maas-api-e2e-shared-b-w0-a56c7c-696c9f7cd-qm7sl\" (UID: \"05eefdb3-fc35-4d7a-b3be-20819a7b7aaf\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-a56c7c-696c9f7cd-qm7sl"
Oct 30 19:22:23.269108 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:23.269085    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-b-9f939e-openshift-default-97686d89c-dhvqt"
Oct 30 19:22:23.269477 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:23.269105    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-b-9f939e-openshift-default-97686d89c-dhvqt" event={"ID":"d077845d-f1ed-46c2-a420-9984ccbf7902","Type":"ContainerDied","Data":"6633f748f09f6d12d21a1185c964f466d3d42b26f75bbc83be57454553ddff27"}
Oct 30 19:22:23.269477 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:23.269151    4888 scope.go:122] "RemoveContainer" containerID="4bd56c5b0746648dab84aa26d44c02f39c21bbb6247eeb23cfa273ad2b681187"
Oct 30 19:22:23.345924 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:23.345900    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/05eefdb3-fc35-4d7a-b3be-20819a7b7aaf-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w0-a56c7c-696c9f7cd-qm7sl\" (UID: \"05eefdb3-fc35-4d7a-b3be-20819a7b7aaf\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-a56c7c-696c9f7cd-qm7sl"
Oct 30 19:22:23.346077 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:23.345954    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/05eefdb3-fc35-4d7a-b3be-20819a7b7aaf-maas-api-tls\") pod \"maas-api-e2e-shared-b-w0-a56c7c-696c9f7cd-qm7sl\" (UID: \"05eefdb3-fc35-4d7a-b3be-20819a7b7aaf\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-a56c7c-696c9f7cd-qm7sl"
Oct 30 19:22:23.346077 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:23.346032    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kjrfz\" (UniqueName: \"kubernetes.io/projected/05eefdb3-fc35-4d7a-b3be-20819a7b7aaf-kube-api-access-kjrfz\") pod \"maas-api-e2e-shared-b-w0-a56c7c-696c9f7cd-qm7sl\" (UID: \"05eefdb3-fc35-4d7a-b3be-20819a7b7aaf\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-a56c7c-696c9f7cd-qm7sl"
Oct 30 19:22:23.346203 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:23.346125    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/05eefdb3-fc35-4d7a-b3be-20819a7b7aaf-metrics-certs\") pod \"maas-api-e2e-shared-b-w0-a56c7c-696c9f7cd-qm7sl\" (UID: \"05eefdb3-fc35-4d7a-b3be-20819a7b7aaf\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-a56c7c-696c9f7cd-qm7sl"
Oct 30 19:22:23.346522 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:22:23.346500    4888 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-a56c7c-serving-cert: secret "maas-api-e2e-shared-b-w0-a56c7c-serving-cert" not found
Oct 30 19:22:23.346622 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:22:23.346585    4888 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/05eefdb3-fc35-4d7a-b3be-20819a7b7aaf-maas-api-tls podName:05eefdb3-fc35-4d7a-b3be-20819a7b7aaf nodeName:}" failed. No retries permitted until 2026-10-30 19:22:23.846569647 +0000 UTC m=+2908.093975650 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/05eefdb3-fc35-4d7a-b3be-20819a7b7aaf-maas-api-tls") pod "maas-api-e2e-shared-b-w0-a56c7c-696c9f7cd-qm7sl" (UID: "05eefdb3-fc35-4d7a-b3be-20819a7b7aaf") : secret "maas-api-e2e-shared-b-w0-a56c7c-serving-cert" not found
Oct 30 19:22:23.346700 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:23.346632    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/05eefdb3-fc35-4d7a-b3be-20819a7b7aaf-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w0-a56c7c-696c9f7cd-qm7sl\" (UID: \"05eefdb3-fc35-4d7a-b3be-20819a7b7aaf\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-a56c7c-696c9f7cd-qm7sl"
Oct 30 19:22:23.349012 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:23.348990    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/05eefdb3-fc35-4d7a-b3be-20819a7b7aaf-metrics-certs\") pod \"maas-api-e2e-shared-b-w0-a56c7c-696c9f7cd-qm7sl\" (UID: \"05eefdb3-fc35-4d7a-b3be-20819a7b7aaf\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-a56c7c-696c9f7cd-qm7sl"
Oct 30 19:22:23.411217 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:23.411116    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kjrfz\" (UniqueName: \"kubernetes.io/projected/05eefdb3-fc35-4d7a-b3be-20819a7b7aaf-kube-api-access-kjrfz\") pod \"maas-api-e2e-shared-b-w0-a56c7c-696c9f7cd-qm7sl\" (UID: \"05eefdb3-fc35-4d7a-b3be-20819a7b7aaf\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-a56c7c-696c9f7cd-qm7sl"
Oct 30 19:22:23.450073 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:23.450044    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-praxis-b-9f939e-openshift-default-97686d89c-dhvqt"]
Oct 30 19:22:23.458369 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:23.458343    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-praxis-b-9f939e-openshift-default-97686d89c-dhvqt"]
Oct 30 19:22:23.853993 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:23.853947    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/05eefdb3-fc35-4d7a-b3be-20819a7b7aaf-maas-api-tls\") pod \"maas-api-e2e-shared-b-w0-a56c7c-696c9f7cd-qm7sl\" (UID: \"05eefdb3-fc35-4d7a-b3be-20819a7b7aaf\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-a56c7c-696c9f7cd-qm7sl"
Oct 30 19:22:23.856550 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:23.856528    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/05eefdb3-fc35-4d7a-b3be-20819a7b7aaf-maas-api-tls\") pod \"maas-api-e2e-shared-b-w0-a56c7c-696c9f7cd-qm7sl\" (UID: \"05eefdb3-fc35-4d7a-b3be-20819a7b7aaf\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-a56c7c-696c9f7cd-qm7sl"
Oct 30 19:22:23.891859 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:23.891830    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d077845d-f1ed-46c2-a420-9984ccbf7902" path="/var/lib/kubelet/pods/d077845d-f1ed-46c2-a420-9984ccbf7902/volumes"
Oct 30 19:22:24.072460 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:24.072425    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-a56c7c-696c9f7cd-qm7sl"
Oct 30 19:22:24.475682 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:24.475646    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-a56c7c-696c9f7cd-qm7sl"]
Oct 30 19:22:24.476100 ip-10-0-134-46 kubenswrapper[4888]: W1030 19:22:24.475777    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05eefdb3_fc35_4d7a_b3be_20819a7b7aaf.slice/crio-93d10e6e28957c9a409fdd56a1fdbdaca846eadadbc4ab4e6a1ec2ad286cddaa WatchSource:0}: Error finding container 93d10e6e28957c9a409fdd56a1fdbdaca846eadadbc4ab4e6a1ec2ad286cddaa: Status 404 returned error can't find the container with id 93d10e6e28957c9a409fdd56a1fdbdaca846eadadbc4ab4e6a1ec2ad286cddaa
Oct 30 19:22:25.285346 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:25.285313    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-a56c7c-696c9f7cd-qm7sl" event={"ID":"05eefdb3-fc35-4d7a-b3be-20819a7b7aaf","Type":"ContainerStarted","Data":"93d10e6e28957c9a409fdd56a1fdbdaca846eadadbc4ab4e6a1ec2ad286cddaa"}
Oct 30 19:22:26.305426 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:26.305386    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-a56c7c-696c9f7cd-qm7sl" event={"ID":"05eefdb3-fc35-4d7a-b3be-20819a7b7aaf","Type":"ContainerStarted","Data":"aece5e329f9346f51fe375eecf4511e8931e00f5723e2524317d23fa9185cc19"}
Oct 30 19:22:26.306032 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:26.305620    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-a56c7c-696c9f7cd-qm7sl"
Oct 30 19:22:26.429421 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:26.429343    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-a56c7c-696c9f7cd-qm7sl" podStartSLOduration=2.659207277 podStartE2EDuration="3.429326135s" totalImagesPullingTime="770.118858ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:22:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:22:26.426821133 +0000 UTC m=+2910.674227158" watchObservedRunningTime="2026-10-30 19:22:26.429326135 +0000 UTC m=+2910.676732162"
Oct 30 19:22:32.317452 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:32.317424    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-a56c7c-696c9f7cd-qm7sl"
Oct 30 19:22:34.877171 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:34.877138    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-autores-c60f5c-openshift-default-9c4d97949-vk946"]
Oct 30 19:22:34.877744 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:34.877730    4888 cpu_manager.go:414] "containerMap: removing container" podUID="243ebea3-fb15-455c-8e02-f152b478d25a" containerName="istio-proxy"
Oct 30 19:22:34.877795 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:34.877745    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="243ebea3-fb15-455c-8e02-f152b478d25a" containerName="istio-proxy"
Oct 30 19:22:34.881936 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:34.881916    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-c60f5c-openshift-default-9c4d97949-vk946"
Oct 30 19:22:34.943765 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:34.943730    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-autores-c60f5c-openshift-default-dockercfg-ksnp5\""
Oct 30 19:22:34.993025 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:34.992974    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2b71ddaa-a90a-4764-a703-91f6f81db363-istio-token\") pod \"e2e-autores-c60f5c-openshift-default-9c4d97949-vk946\" (UID: \"2b71ddaa-a90a-4764-a703-91f6f81db363\") " pod="openshift-ingress/e2e-autores-c60f5c-openshift-default-9c4d97949-vk946"
Oct 30 19:22:34.993208 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:34.993068    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2b71ddaa-a90a-4764-a703-91f6f81db363-credential-socket\") pod \"e2e-autores-c60f5c-openshift-default-9c4d97949-vk946\" (UID: \"2b71ddaa-a90a-4764-a703-91f6f81db363\") " pod="openshift-ingress/e2e-autores-c60f5c-openshift-default-9c4d97949-vk946"
Oct 30 19:22:34.993208 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:34.993106    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2b71ddaa-a90a-4764-a703-91f6f81db363-workload-certs\") pod \"e2e-autores-c60f5c-openshift-default-9c4d97949-vk946\" (UID: \"2b71ddaa-a90a-4764-a703-91f6f81db363\") " pod="openshift-ingress/e2e-autores-c60f5c-openshift-default-9c4d97949-vk946"
Oct 30 19:22:34.993208 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:34.993129    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2b71ddaa-a90a-4764-a703-91f6f81db363-istio-podinfo\") pod \"e2e-autores-c60f5c-openshift-default-9c4d97949-vk946\" (UID: \"2b71ddaa-a90a-4764-a703-91f6f81db363\") " pod="openshift-ingress/e2e-autores-c60f5c-openshift-default-9c4d97949-vk946"
Oct 30 19:22:34.993208 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:34.993174    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2b71ddaa-a90a-4764-a703-91f6f81db363-istio-data\") pod \"e2e-autores-c60f5c-openshift-default-9c4d97949-vk946\" (UID: \"2b71ddaa-a90a-4764-a703-91f6f81db363\") " pod="openshift-ingress/e2e-autores-c60f5c-openshift-default-9c4d97949-vk946"
Oct 30 19:22:34.993208 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:34.993204    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bqkw4\" (UniqueName: \"kubernetes.io/projected/2b71ddaa-a90a-4764-a703-91f6f81db363-kube-api-access-bqkw4\") pod \"e2e-autores-c60f5c-openshift-default-9c4d97949-vk946\" (UID: \"2b71ddaa-a90a-4764-a703-91f6f81db363\") " pod="openshift-ingress/e2e-autores-c60f5c-openshift-default-9c4d97949-vk946"
Oct 30 19:22:34.993536 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:34.993244    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2b71ddaa-a90a-4764-a703-91f6f81db363-workload-socket\") pod \"e2e-autores-c60f5c-openshift-default-9c4d97949-vk946\" (UID: \"2b71ddaa-a90a-4764-a703-91f6f81db363\") " pod="openshift-ingress/e2e-autores-c60f5c-openshift-default-9c4d97949-vk946"
Oct 30 19:22:34.993536 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:34.993324    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2b71ddaa-a90a-4764-a703-91f6f81db363-istio-envoy\") pod \"e2e-autores-c60f5c-openshift-default-9c4d97949-vk946\" (UID: \"2b71ddaa-a90a-4764-a703-91f6f81db363\") " pod="openshift-ingress/e2e-autores-c60f5c-openshift-default-9c4d97949-vk946"
Oct 30 19:22:34.993536 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:34.993388    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2b71ddaa-a90a-4764-a703-91f6f81db363-istiod-ca-cert\") pod \"e2e-autores-c60f5c-openshift-default-9c4d97949-vk946\" (UID: \"2b71ddaa-a90a-4764-a703-91f6f81db363\") " pod="openshift-ingress/e2e-autores-c60f5c-openshift-default-9c4d97949-vk946"
Oct 30 19:22:35.000510 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:35.000482    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-autores-c60f5c-openshift-default-9c4d97949-vk946"]
Oct 30 19:22:35.094351 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:35.094319    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2b71ddaa-a90a-4764-a703-91f6f81db363-istio-envoy\") pod \"e2e-autores-c60f5c-openshift-default-9c4d97949-vk946\" (UID: \"2b71ddaa-a90a-4764-a703-91f6f81db363\") " pod="openshift-ingress/e2e-autores-c60f5c-openshift-default-9c4d97949-vk946"
Oct 30 19:22:35.094545 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:35.094407    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2b71ddaa-a90a-4764-a703-91f6f81db363-istiod-ca-cert\") pod \"e2e-autores-c60f5c-openshift-default-9c4d97949-vk946\" (UID: \"2b71ddaa-a90a-4764-a703-91f6f81db363\") " pod="openshift-ingress/e2e-autores-c60f5c-openshift-default-9c4d97949-vk946"
Oct 30 19:22:35.094545 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:35.094521    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2b71ddaa-a90a-4764-a703-91f6f81db363-istio-token\") pod \"e2e-autores-c60f5c-openshift-default-9c4d97949-vk946\" (UID: \"2b71ddaa-a90a-4764-a703-91f6f81db363\") " pod="openshift-ingress/e2e-autores-c60f5c-openshift-default-9c4d97949-vk946"
Oct 30 19:22:35.094694 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:35.094606    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2b71ddaa-a90a-4764-a703-91f6f81db363-credential-socket\") pod \"e2e-autores-c60f5c-openshift-default-9c4d97949-vk946\" (UID: \"2b71ddaa-a90a-4764-a703-91f6f81db363\") " pod="openshift-ingress/e2e-autores-c60f5c-openshift-default-9c4d97949-vk946"
Oct 30 19:22:35.094694 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:35.094647    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2b71ddaa-a90a-4764-a703-91f6f81db363-workload-certs\") pod \"e2e-autores-c60f5c-openshift-default-9c4d97949-vk946\" (UID: \"2b71ddaa-a90a-4764-a703-91f6f81db363\") " pod="openshift-ingress/e2e-autores-c60f5c-openshift-default-9c4d97949-vk946"
Oct 30 19:22:35.094694 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:35.094672    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2b71ddaa-a90a-4764-a703-91f6f81db363-istio-podinfo\") pod \"e2e-autores-c60f5c-openshift-default-9c4d97949-vk946\" (UID: \"2b71ddaa-a90a-4764-a703-91f6f81db363\") " pod="openshift-ingress/e2e-autores-c60f5c-openshift-default-9c4d97949-vk946"
Oct 30 19:22:35.094852 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:35.094725    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2b71ddaa-a90a-4764-a703-91f6f81db363-istio-data\") pod \"e2e-autores-c60f5c-openshift-default-9c4d97949-vk946\" (UID: \"2b71ddaa-a90a-4764-a703-91f6f81db363\") " pod="openshift-ingress/e2e-autores-c60f5c-openshift-default-9c4d97949-vk946"
Oct 30 19:22:35.094852 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:35.094760    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bqkw4\" (UniqueName: \"kubernetes.io/projected/2b71ddaa-a90a-4764-a703-91f6f81db363-kube-api-access-bqkw4\") pod \"e2e-autores-c60f5c-openshift-default-9c4d97949-vk946\" (UID: \"2b71ddaa-a90a-4764-a703-91f6f81db363\") " pod="openshift-ingress/e2e-autores-c60f5c-openshift-default-9c4d97949-vk946"
Oct 30 19:22:35.094852 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:35.094810    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2b71ddaa-a90a-4764-a703-91f6f81db363-workload-socket\") pod \"e2e-autores-c60f5c-openshift-default-9c4d97949-vk946\" (UID: \"2b71ddaa-a90a-4764-a703-91f6f81db363\") " pod="openshift-ingress/e2e-autores-c60f5c-openshift-default-9c4d97949-vk946"
Oct 30 19:22:35.095016 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:35.094880    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2b71ddaa-a90a-4764-a703-91f6f81db363-credential-socket\") pod \"e2e-autores-c60f5c-openshift-default-9c4d97949-vk946\" (UID: \"2b71ddaa-a90a-4764-a703-91f6f81db363\") " pod="openshift-ingress/e2e-autores-c60f5c-openshift-default-9c4d97949-vk946"
Oct 30 19:22:35.095131 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:35.095105    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2b71ddaa-a90a-4764-a703-91f6f81db363-workload-certs\") pod \"e2e-autores-c60f5c-openshift-default-9c4d97949-vk946\" (UID: \"2b71ddaa-a90a-4764-a703-91f6f81db363\") " pod="openshift-ingress/e2e-autores-c60f5c-openshift-default-9c4d97949-vk946"
Oct 30 19:22:35.095197 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:35.095172    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2b71ddaa-a90a-4764-a703-91f6f81db363-workload-socket\") pod \"e2e-autores-c60f5c-openshift-default-9c4d97949-vk946\" (UID: \"2b71ddaa-a90a-4764-a703-91f6f81db363\") " pod="openshift-ingress/e2e-autores-c60f5c-openshift-default-9c4d97949-vk946"
Oct 30 19:22:35.095245 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:35.095210    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2b71ddaa-a90a-4764-a703-91f6f81db363-istio-data\") pod \"e2e-autores-c60f5c-openshift-default-9c4d97949-vk946\" (UID: \"2b71ddaa-a90a-4764-a703-91f6f81db363\") " pod="openshift-ingress/e2e-autores-c60f5c-openshift-default-9c4d97949-vk946"
Oct 30 19:22:35.095620 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:35.095600    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2b71ddaa-a90a-4764-a703-91f6f81db363-istiod-ca-cert\") pod \"e2e-autores-c60f5c-openshift-default-9c4d97949-vk946\" (UID: \"2b71ddaa-a90a-4764-a703-91f6f81db363\") " pod="openshift-ingress/e2e-autores-c60f5c-openshift-default-9c4d97949-vk946"
Oct 30 19:22:35.097248 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:35.097221    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2b71ddaa-a90a-4764-a703-91f6f81db363-istio-envoy\") pod \"e2e-autores-c60f5c-openshift-default-9c4d97949-vk946\" (UID: \"2b71ddaa-a90a-4764-a703-91f6f81db363\") " pod="openshift-ingress/e2e-autores-c60f5c-openshift-default-9c4d97949-vk946"
Oct 30 19:22:35.097843 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:35.097819    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2b71ddaa-a90a-4764-a703-91f6f81db363-istio-podinfo\") pod \"e2e-autores-c60f5c-openshift-default-9c4d97949-vk946\" (UID: \"2b71ddaa-a90a-4764-a703-91f6f81db363\") " pod="openshift-ingress/e2e-autores-c60f5c-openshift-default-9c4d97949-vk946"
Oct 30 19:22:35.159981 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:35.159907    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2b71ddaa-a90a-4764-a703-91f6f81db363-istio-token\") pod \"e2e-autores-c60f5c-openshift-default-9c4d97949-vk946\" (UID: \"2b71ddaa-a90a-4764-a703-91f6f81db363\") " pod="openshift-ingress/e2e-autores-c60f5c-openshift-default-9c4d97949-vk946"
Oct 30 19:22:35.163118 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:35.163096    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bqkw4\" (UniqueName: \"kubernetes.io/projected/2b71ddaa-a90a-4764-a703-91f6f81db363-kube-api-access-bqkw4\") pod \"e2e-autores-c60f5c-openshift-default-9c4d97949-vk946\" (UID: \"2b71ddaa-a90a-4764-a703-91f6f81db363\") " pod="openshift-ingress/e2e-autores-c60f5c-openshift-default-9c4d97949-vk946"
Oct 30 19:22:35.199442 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:35.199419    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-c60f5c-openshift-default-9c4d97949-vk946"
Oct 30 19:22:35.619704 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:22:35.619664    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-jbsw6" containerName="extractor"
Oct 30 19:22:35.654516 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:35.654487    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-autores-c60f5c-openshift-default-9c4d97949-vk946"]
Oct 30 19:22:35.655212 ip-10-0-134-46 kubenswrapper[4888]: W1030 19:22:35.655174    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b71ddaa_a90a_4764_a703_91f6f81db363.slice/crio-c0d72e9035a002a82f2234a40d147cb0afffea57f1e19006e6707ede8fe81d51 WatchSource:0}: Error finding container c0d72e9035a002a82f2234a40d147cb0afffea57f1e19006e6707ede8fe81d51: Status 404 returned error can't find the container with id c0d72e9035a002a82f2234a40d147cb0afffea57f1e19006e6707ede8fe81d51
Oct 30 19:22:35.658357 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:35.658273    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:22:35.658449 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:35.658393    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:22:35.658496 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:35.658450    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:22:36.384112 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:36.384063    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-c60f5c-openshift-default-9c4d97949-vk946" event={"ID":"2b71ddaa-a90a-4764-a703-91f6f81db363","Type":"ContainerStarted","Data":"a078a02fc2a846bde79c2a47d37550014aa112ebe408d23f4b0a85e6f74f1aa3"}
Oct 30 19:22:36.384584 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:36.384128    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-c60f5c-openshift-default-9c4d97949-vk946" event={"ID":"2b71ddaa-a90a-4764-a703-91f6f81db363","Type":"ContainerStarted","Data":"c0d72e9035a002a82f2234a40d147cb0afffea57f1e19006e6707ede8fe81d51"}
Oct 30 19:22:36.610549 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:36.610490    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-autores-c60f5c-openshift-default-9c4d97949-vk946" podStartSLOduration=2.610472282 podStartE2EDuration="2.610472282s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:22:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:22:36.55844378 +0000 UTC m=+2920.805849802" watchObservedRunningTime="2026-10-30 19:22:36.610472282 +0000 UTC m=+2920.857878308"
Oct 30 19:22:37.200550 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:37.200517    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-autores-c60f5c-openshift-default-9c4d97949-vk946"
Oct 30 19:22:37.206934 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:37.206909    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-autores-c60f5c-openshift-default-9c4d97949-vk946"
Oct 30 19:22:37.393194 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:37.393161    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-autores-c60f5c-openshift-default-9c4d97949-vk946"
Oct 30 19:22:37.394386 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:37.394359    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-autores-c60f5c-openshift-default-9c4d97949-vk946"
Oct 30 19:22:40.526536 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:40.526504    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-rjmdz"]
Oct 30 19:22:43.443303 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:43.443006    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-c60f5c-9cc67ddcf-hr6vs"]
Oct 30 19:22:43.449939 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:43.449912    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-c60f5c-9cc67ddcf-hr6vs"
Oct 30 19:22:43.533309 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:43.512697    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-c60f5c-9cc67ddcf-hr6vs"]
Oct 30 19:22:43.569797 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:43.537576    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-autores-c60f5c\""
Oct 30 19:22:43.570005 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:43.537687    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-autores-c60f5c-dockercfg-v5tz5\""
Oct 30 19:22:43.570088 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:43.566257    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dnjhx\" (UniqueName: \"kubernetes.io/projected/6d598741-f60f-4c9b-b937-4035ec6943a2-kube-api-access-dnjhx\") pod \"payload-pre-processing-e2e-autores-c60f5c-9cc67ddcf-hr6vs\" (UID: \"6d598741-f60f-4c9b-b937-4035ec6943a2\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-c60f5c-9cc67ddcf-hr6vs"
Oct 30 19:22:43.570492 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:43.570467    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6d598741-f60f-4c9b-b937-4035ec6943a2-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-c60f5c-9cc67ddcf-hr6vs\" (UID: \"6d598741-f60f-4c9b-b937-4035ec6943a2\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-c60f5c-9cc67ddcf-hr6vs"
Oct 30 19:22:43.699373 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:43.698075    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-c60f5c-7cf756c6b-sql7j"]
Oct 30 19:22:43.743701 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:43.743661    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-c60f5c-7cf756c6b-sql7j"
Oct 30 19:22:43.758038 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:43.758003    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6d598741-f60f-4c9b-b937-4035ec6943a2-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-c60f5c-9cc67ddcf-hr6vs\" (UID: \"6d598741-f60f-4c9b-b937-4035ec6943a2\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-c60f5c-9cc67ddcf-hr6vs"
Oct 30 19:22:43.758195 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:43.758083    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dnjhx\" (UniqueName: \"kubernetes.io/projected/6d598741-f60f-4c9b-b937-4035ec6943a2-kube-api-access-dnjhx\") pod \"payload-pre-processing-e2e-autores-c60f5c-9cc67ddcf-hr6vs\" (UID: \"6d598741-f60f-4c9b-b937-4035ec6943a2\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-c60f5c-9cc67ddcf-hr6vs"
Oct 30 19:22:43.763728 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:43.759174    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6d598741-f60f-4c9b-b937-4035ec6943a2-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-c60f5c-9cc67ddcf-hr6vs\" (UID: \"6d598741-f60f-4c9b-b937-4035ec6943a2\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-c60f5c-9cc67ddcf-hr6vs"
Oct 30 19:22:43.809318 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:43.801078    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-c60f5c-7cf756c6b-sql7j"]
Oct 30 19:22:43.848034 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:43.848001    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dnjhx\" (UniqueName: \"kubernetes.io/projected/6d598741-f60f-4c9b-b937-4035ec6943a2-kube-api-access-dnjhx\") pod \"payload-pre-processing-e2e-autores-c60f5c-9cc67ddcf-hr6vs\" (UID: \"6d598741-f60f-4c9b-b937-4035ec6943a2\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-c60f5c-9cc67ddcf-hr6vs"
Oct 30 19:22:43.859098 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:43.859063    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4ktlx\" (UniqueName: \"kubernetes.io/projected/3a30428d-1c18-4962-b665-beb8bf3fad1d-kube-api-access-4ktlx\") pod \"payload-processing-e2e-autores-c60f5c-7cf756c6b-sql7j\" (UID: \"3a30428d-1c18-4962-b665-beb8bf3fad1d\") " pod="openshift-ingress/payload-processing-e2e-autores-c60f5c-7cf756c6b-sql7j"
Oct 30 19:22:43.859255 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:43.859146    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3a30428d-1c18-4962-b665-beb8bf3fad1d-plugins-config-volume\") pod \"payload-processing-e2e-autores-c60f5c-7cf756c6b-sql7j\" (UID: \"3a30428d-1c18-4962-b665-beb8bf3fad1d\") " pod="openshift-ingress/payload-processing-e2e-autores-c60f5c-7cf756c6b-sql7j"
Oct 30 19:22:43.960150 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:43.960063    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4ktlx\" (UniqueName: \"kubernetes.io/projected/3a30428d-1c18-4962-b665-beb8bf3fad1d-kube-api-access-4ktlx\") pod \"payload-processing-e2e-autores-c60f5c-7cf756c6b-sql7j\" (UID: \"3a30428d-1c18-4962-b665-beb8bf3fad1d\") " pod="openshift-ingress/payload-processing-e2e-autores-c60f5c-7cf756c6b-sql7j"
Oct 30 19:22:43.960150 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:43.960114    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3a30428d-1c18-4962-b665-beb8bf3fad1d-plugins-config-volume\") pod \"payload-processing-e2e-autores-c60f5c-7cf756c6b-sql7j\" (UID: \"3a30428d-1c18-4962-b665-beb8bf3fad1d\") " pod="openshift-ingress/payload-processing-e2e-autores-c60f5c-7cf756c6b-sql7j"
Oct 30 19:22:43.961190 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:43.961167    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3a30428d-1c18-4962-b665-beb8bf3fad1d-plugins-config-volume\") pod \"payload-processing-e2e-autores-c60f5c-7cf756c6b-sql7j\" (UID: \"3a30428d-1c18-4962-b665-beb8bf3fad1d\") " pod="openshift-ingress/payload-processing-e2e-autores-c60f5c-7cf756c6b-sql7j"
Oct 30 19:22:44.024272 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:44.024222    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4ktlx\" (UniqueName: \"kubernetes.io/projected/3a30428d-1c18-4962-b665-beb8bf3fad1d-kube-api-access-4ktlx\") pod \"payload-processing-e2e-autores-c60f5c-7cf756c6b-sql7j\" (UID: \"3a30428d-1c18-4962-b665-beb8bf3fad1d\") " pod="openshift-ingress/payload-processing-e2e-autores-c60f5c-7cf756c6b-sql7j"
Oct 30 19:22:44.066729 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:44.066698    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-c60f5c-9cc67ddcf-hr6vs"
Oct 30 19:22:44.124385 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:44.121298    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-c60f5c-7cf756c6b-sql7j"
Oct 30 19:22:44.565250 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:44.565219    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-c60f5c-9cc67ddcf-hr6vs"]
Oct 30 19:22:44.565636 ip-10-0-134-46 kubenswrapper[4888]: W1030 19:22:44.565411    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d598741_f60f_4c9b_b937_4035ec6943a2.slice/crio-0d9dac4bd5dc7a550f3c1592302c5a3327f8550421c1d253b8ed8e2729cc7afb WatchSource:0}: Error finding container 0d9dac4bd5dc7a550f3c1592302c5a3327f8550421c1d253b8ed8e2729cc7afb: Status 404 returned error can't find the container with id 0d9dac4bd5dc7a550f3c1592302c5a3327f8550421c1d253b8ed8e2729cc7afb
Oct 30 19:22:44.771307 ip-10-0-134-46 kubenswrapper[4888]: W1030 19:22:44.767482    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a30428d_1c18_4962_b665_beb8bf3fad1d.slice/crio-3156d8a672a8ca397907286feb95c17db8956ef086409509981702f017fb8643 WatchSource:0}: Error finding container 3156d8a672a8ca397907286feb95c17db8956ef086409509981702f017fb8643: Status 404 returned error can't find the container with id 3156d8a672a8ca397907286feb95c17db8956ef086409509981702f017fb8643
Oct 30 19:22:44.795714 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:44.795683    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-c60f5c-7cf756c6b-sql7j"]
Oct 30 19:22:45.504223 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:45.500539    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-c60f5c-7cf756c6b-sql7j" event={"ID":"3a30428d-1c18-4962-b665-beb8bf3fad1d","Type":"ContainerStarted","Data":"a2cd073e753fdd46fa9d2b83e63240a0201c946d83cffa05abcb2ddc1d462952"}
Oct 30 19:22:45.504223 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:45.500602    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-c60f5c-7cf756c6b-sql7j" event={"ID":"3a30428d-1c18-4962-b665-beb8bf3fad1d","Type":"ContainerStarted","Data":"3156d8a672a8ca397907286feb95c17db8956ef086409509981702f017fb8643"}
Oct 30 19:22:45.504223 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:45.500662    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-autores-c60f5c-7cf756c6b-sql7j"
Oct 30 19:22:45.504223 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:45.503683    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-c60f5c-9cc67ddcf-hr6vs" event={"ID":"6d598741-f60f-4c9b-b937-4035ec6943a2","Type":"ContainerStarted","Data":"a3091d910aa05185394b6cf534c9e0142e4c81a3c341c3e9af498a4f79b1ab50"}
Oct 30 19:22:45.504223 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:45.503731    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-c60f5c-9cc67ddcf-hr6vs" event={"ID":"6d598741-f60f-4c9b-b937-4035ec6943a2","Type":"ContainerStarted","Data":"0d9dac4bd5dc7a550f3c1592302c5a3327f8550421c1d253b8ed8e2729cc7afb"}
Oct 30 19:22:45.504675 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:45.504387    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-autores-c60f5c-9cc67ddcf-hr6vs"
Oct 30 19:22:45.634113 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:45.634055    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-autores-c60f5c-7cf756c6b-sql7j" podStartSLOduration=2.634037397 podStartE2EDuration="2.634037397s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:22:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:22:45.626590092 +0000 UTC m=+2929.873996115" watchObservedRunningTime="2026-10-30 19:22:45.634037397 +0000 UTC m=+2929.881443427"
Oct 30 19:22:45.756346 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:45.756229    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-autores-c60f5c-9cc67ddcf-hr6vs" podStartSLOduration=2.756215648 podStartE2EDuration="2.756215648s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:22:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:22:45.753864896 +0000 UTC m=+2930.001270920" watchObservedRunningTime="2026-10-30 19:22:45.756215648 +0000 UTC m=+2930.003621710"
Oct 30 19:22:51.507507 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:51.507428    4888 patch_prober.go:29] interesting pod/e2e-praxis-b-f5dfb8-openshift-default-d59df9d89-28vsl container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.145:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:22:51.517134 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:51.507498    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-praxis-b-f5dfb8-openshift-default-d59df9d89-28vsl" podUID="b1598dab-80ac-408c-b65a-bc3cde3b01d5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.145:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:22:52.587130 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:52.587101    4888 generic.go:356] "Generic (PLEG): container finished" podID="b1598dab-80ac-408c-b65a-bc3cde3b01d5" containerID="fb2a49e557e558552317c215f316850d9f2b0f5845a5f4702de557d71c99469d" exitCode=0
Oct 30 19:22:52.587646 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:52.587174    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-b-f5dfb8-openshift-default-d59df9d89-28vsl" event={"ID":"b1598dab-80ac-408c-b65a-bc3cde3b01d5","Type":"ContainerDied","Data":"fb2a49e557e558552317c215f316850d9f2b0f5845a5f4702de557d71c99469d"}
Oct 30 19:22:52.587719 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:52.587682    4888 scope.go:122] "RemoveContainer" containerID="fb2a49e557e558552317c215f316850d9f2b0f5845a5f4702de557d71c99469d"
Oct 30 19:22:52.589402 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:52.589382    4888 generic.go:356] "Generic (PLEG): container finished" podID="36712903-e111-4fb5-a600-9b666b1d1c9f" containerID="3b56a8d77360e89020665adaeadd010c41bb43e32029d47ffb874b68bb7feb18" exitCode=0
Oct 30 19:22:52.589521 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:52.589431    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w0-a56c7c-openshift-default-5db77d74cf-jjcdj" event={"ID":"36712903-e111-4fb5-a600-9b666b1d1c9f","Type":"ContainerDied","Data":"3b56a8d77360e89020665adaeadd010c41bb43e32029d47ffb874b68bb7feb18"}
Oct 30 19:22:52.589586 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:52.589556    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:22:52.589648 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:52.589630    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:22:52.589689 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:52.589675    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:22:52.589828 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:52.589812    4888 scope.go:122] "RemoveContainer" containerID="3b56a8d77360e89020665adaeadd010c41bb43e32029d47ffb874b68bb7feb18"
Oct 30 19:22:52.591345 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:52.591316    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:22:52.591505 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:52.591395    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:22:52.591505 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:52.591445    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:22:53.629673 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:53.629633    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-b-f5dfb8-openshift-default-d59df9d89-28vsl" event={"ID":"b1598dab-80ac-408c-b65a-bc3cde3b01d5","Type":"ContainerStarted","Data":"9c0edb68221716f42b960b076aa36b86f73e97f779f65368ee89418000a387f3"}
Oct 30 19:22:53.632505 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:53.632469    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w0-a56c7c-openshift-default-5db77d74cf-jjcdj" event={"ID":"36712903-e111-4fb5-a600-9b666b1d1c9f","Type":"ContainerStarted","Data":"625bdab96a1a7504f8d4687c94b34e97ea397a73432468ad06cdfcfcc69fddfe"}
Oct 30 19:22:54.062136 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:54.062062    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w0-a56c7c-openshift-default-5db77d74cf-jjcdj"
Oct 30 19:22:54.063249 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:54.063225    4888 patch_prober.go:29] interesting pod/e2e-shared-b-w0-a56c7c-openshift-default-5db77d74cf-jjcdj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.146:15021/healthz/ready\": dial tcp 10.132.0.146:15021: connect: connection refused" start-of-body=
Oct 30 19:22:54.063356 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:54.063295    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w0-a56c7c-openshift-default-5db77d74cf-jjcdj" podUID="36712903-e111-4fb5-a600-9b666b1d1c9f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.146:15021/healthz/ready\": dial tcp 10.132.0.146:15021: connect: connection refused"
Oct 30 19:22:54.299311 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:54.297461    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-praxis-b-f5dfb8-openshift-default-d59df9d89-28vsl"
Oct 30 19:22:54.299311 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:54.297582    4888 patch_prober.go:29] interesting pod/e2e-praxis-b-f5dfb8-openshift-default-d59df9d89-28vsl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.145:15021/healthz/ready\": dial tcp 10.132.0.145:15021: connect: connection refused" start-of-body=
Oct 30 19:22:54.299311 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:54.297636    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-b-f5dfb8-openshift-default-d59df9d89-28vsl" podUID="b1598dab-80ac-408c-b65a-bc3cde3b01d5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.145:15021/healthz/ready\": dial tcp 10.132.0.145:15021: connect: connection refused"
Oct 30 19:22:55.061773 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:55.061739    4888 patch_prober.go:29] interesting pod/e2e-shared-b-w0-a56c7c-openshift-default-5db77d74cf-jjcdj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.146:15021/healthz/ready\": dial tcp 10.132.0.146:15021: connect: connection refused" start-of-body=
Oct 30 19:22:55.062176 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:55.061797    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w0-a56c7c-openshift-default-5db77d74cf-jjcdj" podUID="36712903-e111-4fb5-a600-9b666b1d1c9f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.146:15021/healthz/ready\": dial tcp 10.132.0.146:15021: connect: connection refused"
Oct 30 19:22:55.298310 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:55.297673    4888 patch_prober.go:29] interesting pod/e2e-praxis-b-f5dfb8-openshift-default-d59df9d89-28vsl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.145:15021/healthz/ready\": dial tcp 10.132.0.145:15021: connect: connection refused" start-of-body=
Oct 30 19:22:55.298310 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:55.297735    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-b-f5dfb8-openshift-default-d59df9d89-28vsl" podUID="b1598dab-80ac-408c-b65a-bc3cde3b01d5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.145:15021/healthz/ready\": dial tcp 10.132.0.145:15021: connect: connection refused"
Oct 30 19:22:56.061998 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:56.061963    4888 patch_prober.go:29] interesting pod/e2e-shared-b-w0-a56c7c-openshift-default-5db77d74cf-jjcdj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.146:15021/healthz/ready\": dial tcp 10.132.0.146:15021: connect: connection refused" start-of-body=
Oct 30 19:22:56.062441 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:56.062014    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w0-a56c7c-openshift-default-5db77d74cf-jjcdj" podUID="36712903-e111-4fb5-a600-9b666b1d1c9f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.146:15021/healthz/ready\": dial tcp 10.132.0.146:15021: connect: connection refused"
Oct 30 19:22:56.297705 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:56.297667    4888 patch_prober.go:29] interesting pod/e2e-praxis-b-f5dfb8-openshift-default-d59df9d89-28vsl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.145:15021/healthz/ready\": dial tcp 10.132.0.145:15021: connect: connection refused" start-of-body=
Oct 30 19:22:56.298038 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:56.297999    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-b-f5dfb8-openshift-default-d59df9d89-28vsl" podUID="b1598dab-80ac-408c-b65a-bc3cde3b01d5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.145:15021/healthz/ready\": dial tcp 10.132.0.145:15021: connect: connection refused"
Oct 30 19:22:56.537353 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:56.537325    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-autores-c60f5c-7cf756c6b-sql7j"
Oct 30 19:22:57.061563 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:57.061532    4888 patch_prober.go:29] interesting pod/e2e-shared-b-w0-a56c7c-openshift-default-5db77d74cf-jjcdj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.146:15021/healthz/ready\": dial tcp 10.132.0.146:15021: connect: connection refused" start-of-body=
Oct 30 19:22:57.061835 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:57.061810    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w0-a56c7c-openshift-default-5db77d74cf-jjcdj" podUID="36712903-e111-4fb5-a600-9b666b1d1c9f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.146:15021/healthz/ready\": dial tcp 10.132.0.146:15021: connect: connection refused"
Oct 30 19:22:57.306309 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:57.306186    4888 patch_prober.go:29] interesting pod/e2e-praxis-b-f5dfb8-openshift-default-d59df9d89-28vsl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.145:15021/healthz/ready\": dial tcp 10.132.0.145:15021: connect: connection refused" start-of-body=
Oct 30 19:22:57.306309 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:57.306245    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-b-f5dfb8-openshift-default-d59df9d89-28vsl" podUID="b1598dab-80ac-408c-b65a-bc3cde3b01d5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.145:15021/healthz/ready\": dial tcp 10.132.0.145:15021: connect: connection refused"
Oct 30 19:22:57.545645 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:57.545619    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-autores-c60f5c-9cc67ddcf-hr6vs"
Oct 30 19:22:58.062209 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:58.062170    4888 patch_prober.go:29] interesting pod/e2e-shared-b-w0-a56c7c-openshift-default-5db77d74cf-jjcdj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.146:15021/healthz/ready\": dial tcp 10.132.0.146:15021: connect: connection refused" start-of-body=
Oct 30 19:22:58.062408 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:58.062237    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w0-a56c7c-openshift-default-5db77d74cf-jjcdj" podUID="36712903-e111-4fb5-a600-9b666b1d1c9f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.146:15021/healthz/ready\": dial tcp 10.132.0.146:15021: connect: connection refused"
Oct 30 19:22:58.299821 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:58.299787    4888 patch_prober.go:29] interesting pod/e2e-praxis-b-f5dfb8-openshift-default-d59df9d89-28vsl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.145:15021/healthz/ready\": dial tcp 10.132.0.145:15021: connect: connection refused" start-of-body=
Oct 30 19:22:58.300134 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:58.300092    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-b-f5dfb8-openshift-default-d59df9d89-28vsl" podUID="b1598dab-80ac-408c-b65a-bc3cde3b01d5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.145:15021/healthz/ready\": dial tcp 10.132.0.145:15021: connect: connection refused"
Oct 30 19:22:58.662594 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:58.662556    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w0-a56c7c-openshift-default-5db77d74cf-jjcdj"
Oct 30 19:22:59.136743 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:59.136708    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w0-a56c7c-openshift-default-5db77d74cf-jjcdj"
Oct 30 19:22:59.302132 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:59.302102    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-praxis-b-f5dfb8-openshift-default-d59df9d89-28vsl"
Oct 30 19:22:59.302527 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:59.302498    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-praxis-b-f5dfb8-openshift-default-d59df9d89-28vsl"
Oct 30 19:22:59.303326 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:59.303273    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-praxis-b-f5dfb8-openshift-default-d59df9d89-28vsl"
Oct 30 19:22:59.694886 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:22:59.694853    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w0-a56c7c-openshift-default-5db77d74cf-jjcdj"
Oct 30 19:23:05.619778 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:23:05.619740    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-jbsw6" containerName="extractor"
Oct 30 19:23:27.701774 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:23:27.701740    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-rjmdz"]
Oct 30 19:23:35.620506 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:23:35.620464    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-jbsw6" containerName="extractor"
Oct 30 19:23:56.704730 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:23:56.704702    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-7x4rb_382e8b31-a7fa-43f3-9f15-4c61627b0c04/console-operator/2.log"
Oct 30 19:23:56.711847 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:23:56.711825    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-7x4rb_382e8b31-a7fa-43f3-9f15-4c61627b0c04/console-operator/2.log"
Oct 30 19:23:56.714792 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:23:56.714766    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xfcdl_3f3fe764-2652-46c9-8e54-7882c0e6bcf1/ovn-acl-logging/0.log"
Oct 30 19:23:56.720870 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:23:56.720852    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xfcdl_3f3fe764-2652-46c9-8e54-7882c0e6bcf1/ovn-acl-logging/0.log"
Oct 30 19:24:05.619914 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:24:05.619824    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-jbsw6" containerName="extractor"
Oct 30 19:24:16.405028 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:16.404986    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-rjmdz"]
Oct 30 19:24:25.417976 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:25.417943    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-66db998887-mhzc5"]
Oct 30 19:24:25.418450 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:25.418203    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-mhzc5" podUID="bf0f21cf-8198-4fe4-8983-db083c82b34a" containerName="istio-proxy" containerID="cri-o://4a9b2f452f9145f7aa7b2379b1ae2e9d191d8db4e55563588af3e448788acfbb" gracePeriod=30
Oct 30 19:24:26.365384 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:26.365316    4888 generic.go:356] "Generic (PLEG): container finished" podID="2b71ddaa-a90a-4764-a703-91f6f81db363" containerID="a078a02fc2a846bde79c2a47d37550014aa112ebe408d23f4b0a85e6f74f1aa3" exitCode=0
Oct 30 19:24:26.365568 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:26.365392    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-c60f5c-openshift-default-9c4d97949-vk946" event={"ID":"2b71ddaa-a90a-4764-a703-91f6f81db363","Type":"ContainerDied","Data":"a078a02fc2a846bde79c2a47d37550014aa112ebe408d23f4b0a85e6f74f1aa3"}
Oct 30 19:24:26.365829 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:26.365812    4888 scope.go:122] "RemoveContainer" containerID="a078a02fc2a846bde79c2a47d37550014aa112ebe408d23f4b0a85e6f74f1aa3"
Oct 30 19:24:26.367378 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:26.367349    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:24:26.367498 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:26.367418    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:24:26.367498 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:26.367450    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:24:27.399596 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:27.399565    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-c60f5c-openshift-default-9c4d97949-vk946" event={"ID":"2b71ddaa-a90a-4764-a703-91f6f81db363","Type":"ContainerStarted","Data":"783cdf54ed93425253fcd9b1bddfba051f2d0510a5f7456338e0f5e6004b631d"}
Oct 30 19:24:27.507722 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:27.507685    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-jhlsk"]
Oct 30 19:24:27.507971 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:27.507945    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-jhlsk" podUID="b43a09c5-9f21-4ac7-99cd-25300651d3a8" containerName="istio-proxy" containerID="cri-o://e3cbb8fa9ae750d9f607caadb0d4953bd778a6e860f96e68cbba4d4279b69d95" gracePeriod=30
Oct 30 19:24:28.200150 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:28.200113    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-autores-c60f5c-openshift-default-9c4d97949-vk946"
Oct 30 19:24:28.200342 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:28.200313    4888 patch_prober.go:29] interesting pod/e2e-autores-c60f5c-openshift-default-9c4d97949-vk946 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.148:15021/healthz/ready\": dial tcp 10.132.0.148:15021: connect: connection refused" start-of-body=
Oct 30 19:24:28.200405 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:28.200360    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-c60f5c-openshift-default-9c4d97949-vk946" podUID="2b71ddaa-a90a-4764-a703-91f6f81db363" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.148:15021/healthz/ready\": dial tcp 10.132.0.148:15021: connect: connection refused"
Oct 30 19:24:29.199700 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:29.199659    4888 patch_prober.go:29] interesting pod/e2e-autores-c60f5c-openshift-default-9c4d97949-vk946 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.148:15021/healthz/ready\": dial tcp 10.132.0.148:15021: connect: connection refused" start-of-body=
Oct 30 19:24:29.200216 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:29.199715    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-c60f5c-openshift-default-9c4d97949-vk946" podUID="2b71ddaa-a90a-4764-a703-91f6f81db363" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.148:15021/healthz/ready\": dial tcp 10.132.0.148:15021: connect: connection refused"
Oct 30 19:24:30.200743 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:30.200706    4888 patch_prober.go:29] interesting pod/e2e-autores-c60f5c-openshift-default-9c4d97949-vk946 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.148:15021/healthz/ready\": dial tcp 10.132.0.148:15021: connect: connection refused" start-of-body=
Oct 30 19:24:30.201438 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:30.201375    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-c60f5c-openshift-default-9c4d97949-vk946" podUID="2b71ddaa-a90a-4764-a703-91f6f81db363" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.148:15021/healthz/ready\": dial tcp 10.132.0.148:15021: connect: connection refused"
Oct 30 19:24:30.750997 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:30.750975    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-mhzc5"
Oct 30 19:24:30.898495 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:30.898431    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bf0f21cf-8198-4fe4-8983-db083c82b34a-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bf0f21cf-8198-4fe4-8983-db083c82b34a-istio-data\") pod \"bf0f21cf-8198-4fe4-8983-db083c82b34a\" (UID: \"bf0f21cf-8198-4fe4-8983-db083c82b34a\") "
Oct 30 19:24:30.898851 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:30.898825    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bf0f21cf-8198-4fe4-8983-db083c82b34a-istio-data" pod "bf0f21cf-8198-4fe4-8983-db083c82b34a" (UID: "bf0f21cf-8198-4fe4-8983-db083c82b34a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:24:30.898979 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:30.898951    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/bf0f21cf-8198-4fe4-8983-db083c82b34a-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bf0f21cf-8198-4fe4-8983-db083c82b34a-istio-podinfo\") pod \"bf0f21cf-8198-4fe4-8983-db083c82b34a\" (UID: \"bf0f21cf-8198-4fe4-8983-db083c82b34a\") "
Oct 30 19:24:30.899061 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:30.899001    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bf0f21cf-8198-4fe4-8983-db083c82b34a-kube-api-access-jszwr\" (UniqueName: \"kubernetes.io/projected/bf0f21cf-8198-4fe4-8983-db083c82b34a-kube-api-access-jszwr\") pod \"bf0f21cf-8198-4fe4-8983-db083c82b34a\" (UID: \"bf0f21cf-8198-4fe4-8983-db083c82b34a\") "
Oct 30 19:24:30.899119 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:30.899056    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bf0f21cf-8198-4fe4-8983-db083c82b34a-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bf0f21cf-8198-4fe4-8983-db083c82b34a-credential-socket\") pod \"bf0f21cf-8198-4fe4-8983-db083c82b34a\" (UID: \"bf0f21cf-8198-4fe4-8983-db083c82b34a\") "
Oct 30 19:24:30.899555 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:30.899532    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/bf0f21cf-8198-4fe4-8983-db083c82b34a-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bf0f21cf-8198-4fe4-8983-db083c82b34a-istiod-ca-cert\") pod \"bf0f21cf-8198-4fe4-8983-db083c82b34a\" (UID: \"bf0f21cf-8198-4fe4-8983-db083c82b34a\") "
Oct 30 19:24:30.899664 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:30.899611    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bf0f21cf-8198-4fe4-8983-db083c82b34a-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bf0f21cf-8198-4fe4-8983-db083c82b34a-workload-certs\") pod \"bf0f21cf-8198-4fe4-8983-db083c82b34a\" (UID: \"bf0f21cf-8198-4fe4-8983-db083c82b34a\") "
Oct 30 19:24:30.899723 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:30.899660    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bf0f21cf-8198-4fe4-8983-db083c82b34a-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bf0f21cf-8198-4fe4-8983-db083c82b34a-workload-socket\") pod \"bf0f21cf-8198-4fe4-8983-db083c82b34a\" (UID: \"bf0f21cf-8198-4fe4-8983-db083c82b34a\") "
Oct 30 19:24:30.899837 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:30.899819    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bf0f21cf-8198-4fe4-8983-db083c82b34a-istio-token\" (UniqueName: \"kubernetes.io/projected/bf0f21cf-8198-4fe4-8983-db083c82b34a-istio-token\") pod \"bf0f21cf-8198-4fe4-8983-db083c82b34a\" (UID: \"bf0f21cf-8198-4fe4-8983-db083c82b34a\") "
Oct 30 19:24:30.899901 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:30.899888    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bf0f21cf-8198-4fe4-8983-db083c82b34a-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bf0f21cf-8198-4fe4-8983-db083c82b34a-istio-envoy\") pod \"bf0f21cf-8198-4fe4-8983-db083c82b34a\" (UID: \"bf0f21cf-8198-4fe4-8983-db083c82b34a\") "
Oct 30 19:24:30.900711 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:30.900667    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bf0f21cf-8198-4fe4-8983-db083c82b34a-istiod-ca-cert" pod "bf0f21cf-8198-4fe4-8983-db083c82b34a" (UID: "bf0f21cf-8198-4fe4-8983-db083c82b34a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:24:30.900853 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:30.900763    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bf0f21cf-8198-4fe4-8983-db083c82b34a-workload-socket" pod "bf0f21cf-8198-4fe4-8983-db083c82b34a" (UID: "bf0f21cf-8198-4fe4-8983-db083c82b34a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:24:30.900923 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:30.900907    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bf0f21cf-8198-4fe4-8983-db083c82b34a-workload-certs" pod "bf0f21cf-8198-4fe4-8983-db083c82b34a" (UID: "bf0f21cf-8198-4fe4-8983-db083c82b34a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:24:30.901499 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:30.901473    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bf0f21cf-8198-4fe4-8983-db083c82b34a-credential-socket" pod "bf0f21cf-8198-4fe4-8983-db083c82b34a" (UID: "bf0f21cf-8198-4fe4-8983-db083c82b34a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:24:30.903270 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:30.902601    4888 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bf0f21cf-8198-4fe4-8983-db083c82b34a-istio-data\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:24:30.903270 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:30.902642    4888 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bf0f21cf-8198-4fe4-8983-db083c82b34a-credential-socket\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:24:30.903270 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:30.902656    4888 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bf0f21cf-8198-4fe4-8983-db083c82b34a-istiod-ca-cert\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:24:30.903270 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:30.902673    4888 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bf0f21cf-8198-4fe4-8983-db083c82b34a-workload-certs\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:24:30.903270 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:30.902703    4888 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bf0f21cf-8198-4fe4-8983-db083c82b34a-workload-socket\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:24:30.904126 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:30.904090    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/bf0f21cf-8198-4fe4-8983-db083c82b34a-istio-podinfo" pod "bf0f21cf-8198-4fe4-8983-db083c82b34a" (UID: "bf0f21cf-8198-4fe4-8983-db083c82b34a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:24:30.904506 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:30.904483    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bf0f21cf-8198-4fe4-8983-db083c82b34a-istio-envoy" pod "bf0f21cf-8198-4fe4-8983-db083c82b34a" (UID: "bf0f21cf-8198-4fe4-8983-db083c82b34a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:24:30.904672 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:30.904654    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bf0f21cf-8198-4fe4-8983-db083c82b34a-kube-api-access-jszwr" pod "bf0f21cf-8198-4fe4-8983-db083c82b34a" (UID: "bf0f21cf-8198-4fe4-8983-db083c82b34a"). InnerVolumeSpecName "kube-api-access-jszwr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:24:30.905429 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:30.905400    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bf0f21cf-8198-4fe4-8983-db083c82b34a-istio-token" pod "bf0f21cf-8198-4fe4-8983-db083c82b34a" (UID: "bf0f21cf-8198-4fe4-8983-db083c82b34a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:24:31.005536 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:31.005501    4888 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bf0f21cf-8198-4fe4-8983-db083c82b34a-istio-podinfo\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:24:31.005536 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:31.005534    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jszwr\" (UniqueName: \"kubernetes.io/projected/bf0f21cf-8198-4fe4-8983-db083c82b34a-kube-api-access-jszwr\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:24:31.005795 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:31.005550    4888 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bf0f21cf-8198-4fe4-8983-db083c82b34a-istio-token\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:24:31.005795 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:31.005566    4888 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bf0f21cf-8198-4fe4-8983-db083c82b34a-istio-envoy\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:24:31.200671 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:31.200561    4888 patch_prober.go:29] interesting pod/e2e-autores-c60f5c-openshift-default-9c4d97949-vk946 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.148:15021/healthz/ready\": dial tcp 10.132.0.148:15021: connect: connection refused" start-of-body=
Oct 30 19:24:31.200671 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:31.200629    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-c60f5c-openshift-default-9c4d97949-vk946" podUID="2b71ddaa-a90a-4764-a703-91f6f81db363" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.148:15021/healthz/ready\": dial tcp 10.132.0.148:15021: connect: connection refused"
Oct 30 19:24:31.429440 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:31.429405    4888 generic.go:356] "Generic (PLEG): container finished" podID="bf0f21cf-8198-4fe4-8983-db083c82b34a" containerID="4a9b2f452f9145f7aa7b2379b1ae2e9d191d8db4e55563588af3e448788acfbb" exitCode=0
Oct 30 19:24:31.429622 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:31.429470    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-mhzc5"
Oct 30 19:24:31.429622 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:31.429536    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-mhzc5" event={"ID":"bf0f21cf-8198-4fe4-8983-db083c82b34a","Type":"ContainerDied","Data":"4a9b2f452f9145f7aa7b2379b1ae2e9d191d8db4e55563588af3e448788acfbb"}
Oct 30 19:24:31.429622 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:31.429580    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-66db998887-mhzc5" event={"ID":"bf0f21cf-8198-4fe4-8983-db083c82b34a","Type":"ContainerDied","Data":"bfc080a2db45bcab96ef48a9e600c9e467611cce07e8128b8ab5ea7619fd2b43"}
Oct 30 19:24:31.429622 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:31.429609    4888 scope.go:122] "RemoveContainer" containerID="4a9b2f452f9145f7aa7b2379b1ae2e9d191d8db4e55563588af3e448788acfbb"
Oct 30 19:24:31.442451 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:31.442423    4888 scope.go:122] "RemoveContainer" containerID="bf28cd527b037444b211a9ce7cbb22d6e612330c035e394c25bba819000865be"
Oct 30 19:24:31.451824 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:31.451805    4888 scope.go:122] "RemoveContainer" containerID="4a9b2f452f9145f7aa7b2379b1ae2e9d191d8db4e55563588af3e448788acfbb"
Oct 30 19:24:31.452103 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:24:31.452072    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4a9b2f452f9145f7aa7b2379b1ae2e9d191d8db4e55563588af3e448788acfbb\": container with ID starting with 4a9b2f452f9145f7aa7b2379b1ae2e9d191d8db4e55563588af3e448788acfbb not found: ID does not exist" containerID="4a9b2f452f9145f7aa7b2379b1ae2e9d191d8db4e55563588af3e448788acfbb"
Oct 30 19:24:31.452160 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:31.452111    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4a9b2f452f9145f7aa7b2379b1ae2e9d191d8db4e55563588af3e448788acfbb"} err="failed to get container status \"4a9b2f452f9145f7aa7b2379b1ae2e9d191d8db4e55563588af3e448788acfbb\": rpc error: code = NotFound desc = could not find container \"4a9b2f452f9145f7aa7b2379b1ae2e9d191d8db4e55563588af3e448788acfbb\": container with ID starting with 4a9b2f452f9145f7aa7b2379b1ae2e9d191d8db4e55563588af3e448788acfbb not found: ID does not exist"
Oct 30 19:24:31.452160 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:31.452128    4888 scope.go:122] "RemoveContainer" containerID="bf28cd527b037444b211a9ce7cbb22d6e612330c035e394c25bba819000865be"
Oct 30 19:24:31.452468 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:24:31.452432    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bf28cd527b037444b211a9ce7cbb22d6e612330c035e394c25bba819000865be\": container with ID starting with bf28cd527b037444b211a9ce7cbb22d6e612330c035e394c25bba819000865be not found: ID does not exist" containerID="bf28cd527b037444b211a9ce7cbb22d6e612330c035e394c25bba819000865be"
Oct 30 19:24:31.452545 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:31.452476    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bf28cd527b037444b211a9ce7cbb22d6e612330c035e394c25bba819000865be"} err="failed to get container status \"bf28cd527b037444b211a9ce7cbb22d6e612330c035e394c25bba819000865be\": rpc error: code = NotFound desc = could not find container \"bf28cd527b037444b211a9ce7cbb22d6e612330c035e394c25bba819000865be\": container with ID starting with bf28cd527b037444b211a9ce7cbb22d6e612330c035e394c25bba819000865be not found: ID does not exist"
Oct 30 19:24:31.605873 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:31.605817    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-66db998887-mhzc5"]
Oct 30 19:24:31.608045 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:31.608015    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-66db998887-mhzc5"]
Oct 30 19:24:31.908359 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:31.908321    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="bf0f21cf-8198-4fe4-8983-db083c82b34a" path="/var/lib/kubelet/pods/bf0f21cf-8198-4fe4-8983-db083c82b34a/volumes"
Oct 30 19:24:32.200798 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:32.200701    4888 patch_prober.go:29] interesting pod/e2e-autores-c60f5c-openshift-default-9c4d97949-vk946 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.148:15021/healthz/ready\": dial tcp 10.132.0.148:15021: connect: connection refused" start-of-body=
Oct 30 19:24:32.200798 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:32.200755    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-c60f5c-openshift-default-9c4d97949-vk946" podUID="2b71ddaa-a90a-4764-a703-91f6f81db363" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.148:15021/healthz/ready\": dial tcp 10.132.0.148:15021: connect: connection refused"
Oct 30 19:24:32.323321 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:32.323193    4888 patch_prober.go:29] interesting pod/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-jhlsk container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 19:24:32.323321 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:32.323252    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-jhlsk" podUID="b43a09c5-9f21-4ac7-99cd-25300651d3a8" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 19:24:32.797105 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:32.797079    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-jhlsk"
Oct 30 19:24:32.931420 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:32.931383    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b43a09c5-9f21-4ac7-99cd-25300651d3a8-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b43a09c5-9f21-4ac7-99cd-25300651d3a8-istio-envoy\") pod \"b43a09c5-9f21-4ac7-99cd-25300651d3a8\" (UID: \"b43a09c5-9f21-4ac7-99cd-25300651d3a8\") "
Oct 30 19:24:32.931599 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:32.931483    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/b43a09c5-9f21-4ac7-99cd-25300651d3a8-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b43a09c5-9f21-4ac7-99cd-25300651d3a8-istio-podinfo\") pod \"b43a09c5-9f21-4ac7-99cd-25300651d3a8\" (UID: \"b43a09c5-9f21-4ac7-99cd-25300651d3a8\") "
Oct 30 19:24:32.931599 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:32.931577    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b43a09c5-9f21-4ac7-99cd-25300651d3a8-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b43a09c5-9f21-4ac7-99cd-25300651d3a8-istiod-ca-cert\") pod \"b43a09c5-9f21-4ac7-99cd-25300651d3a8\" (UID: \"b43a09c5-9f21-4ac7-99cd-25300651d3a8\") "
Oct 30 19:24:32.931680 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:32.931609    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b43a09c5-9f21-4ac7-99cd-25300651d3a8-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b43a09c5-9f21-4ac7-99cd-25300651d3a8-workload-certs\") pod \"b43a09c5-9f21-4ac7-99cd-25300651d3a8\" (UID: \"b43a09c5-9f21-4ac7-99cd-25300651d3a8\") "
Oct 30 19:24:32.931724 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:32.931679    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b43a09c5-9f21-4ac7-99cd-25300651d3a8-kube-api-access-nrbcf\" (UniqueName: \"kubernetes.io/projected/b43a09c5-9f21-4ac7-99cd-25300651d3a8-kube-api-access-nrbcf\") pod \"b43a09c5-9f21-4ac7-99cd-25300651d3a8\" (UID: \"b43a09c5-9f21-4ac7-99cd-25300651d3a8\") "
Oct 30 19:24:32.931786 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:32.931720    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b43a09c5-9f21-4ac7-99cd-25300651d3a8-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b43a09c5-9f21-4ac7-99cd-25300651d3a8-credential-socket\") pod \"b43a09c5-9f21-4ac7-99cd-25300651d3a8\" (UID: \"b43a09c5-9f21-4ac7-99cd-25300651d3a8\") "
Oct 30 19:24:32.931786 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:32.931781    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b43a09c5-9f21-4ac7-99cd-25300651d3a8-istio-token\" (UniqueName: \"kubernetes.io/projected/b43a09c5-9f21-4ac7-99cd-25300651d3a8-istio-token\") pod \"b43a09c5-9f21-4ac7-99cd-25300651d3a8\" (UID: \"b43a09c5-9f21-4ac7-99cd-25300651d3a8\") "
Oct 30 19:24:32.931894 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:32.931843    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b43a09c5-9f21-4ac7-99cd-25300651d3a8-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b43a09c5-9f21-4ac7-99cd-25300651d3a8-istio-data\") pod \"b43a09c5-9f21-4ac7-99cd-25300651d3a8\" (UID: \"b43a09c5-9f21-4ac7-99cd-25300651d3a8\") "
Oct 30 19:24:32.931960 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:32.931898    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b43a09c5-9f21-4ac7-99cd-25300651d3a8-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b43a09c5-9f21-4ac7-99cd-25300651d3a8-workload-socket\") pod \"b43a09c5-9f21-4ac7-99cd-25300651d3a8\" (UID: \"b43a09c5-9f21-4ac7-99cd-25300651d3a8\") "
Oct 30 19:24:32.931960 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:32.931936    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b43a09c5-9f21-4ac7-99cd-25300651d3a8-istiod-ca-cert" pod "b43a09c5-9f21-4ac7-99cd-25300651d3a8" (UID: "b43a09c5-9f21-4ac7-99cd-25300651d3a8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:24:32.932068 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:32.931978    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b43a09c5-9f21-4ac7-99cd-25300651d3a8-workload-certs" pod "b43a09c5-9f21-4ac7-99cd-25300651d3a8" (UID: "b43a09c5-9f21-4ac7-99cd-25300651d3a8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:24:32.932068 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:32.932040    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b43a09c5-9f21-4ac7-99cd-25300651d3a8-credential-socket" pod "b43a09c5-9f21-4ac7-99cd-25300651d3a8" (UID: "b43a09c5-9f21-4ac7-99cd-25300651d3a8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:24:32.932245 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:32.932219    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b43a09c5-9f21-4ac7-99cd-25300651d3a8-istio-data" pod "b43a09c5-9f21-4ac7-99cd-25300651d3a8" (UID: "b43a09c5-9f21-4ac7-99cd-25300651d3a8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:24:32.932428 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:32.932241    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b43a09c5-9f21-4ac7-99cd-25300651d3a8-workload-socket" pod "b43a09c5-9f21-4ac7-99cd-25300651d3a8" (UID: "b43a09c5-9f21-4ac7-99cd-25300651d3a8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:24:32.932712 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:32.932692    4888 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b43a09c5-9f21-4ac7-99cd-25300651d3a8-istiod-ca-cert\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:24:32.932782 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:32.932720    4888 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b43a09c5-9f21-4ac7-99cd-25300651d3a8-workload-certs\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:24:32.932782 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:32.932737    4888 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b43a09c5-9f21-4ac7-99cd-25300651d3a8-credential-socket\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:24:32.933919 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:32.933895    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b43a09c5-9f21-4ac7-99cd-25300651d3a8-istio-podinfo" pod "b43a09c5-9f21-4ac7-99cd-25300651d3a8" (UID: "b43a09c5-9f21-4ac7-99cd-25300651d3a8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:24:32.934129 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:32.934109    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b43a09c5-9f21-4ac7-99cd-25300651d3a8-istio-token" pod "b43a09c5-9f21-4ac7-99cd-25300651d3a8" (UID: "b43a09c5-9f21-4ac7-99cd-25300651d3a8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:24:32.934405 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:32.934378    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b43a09c5-9f21-4ac7-99cd-25300651d3a8-kube-api-access-nrbcf" pod "b43a09c5-9f21-4ac7-99cd-25300651d3a8" (UID: "b43a09c5-9f21-4ac7-99cd-25300651d3a8"). InnerVolumeSpecName "kube-api-access-nrbcf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:24:32.934504 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:32.934389    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b43a09c5-9f21-4ac7-99cd-25300651d3a8-istio-envoy" pod "b43a09c5-9f21-4ac7-99cd-25300651d3a8" (UID: "b43a09c5-9f21-4ac7-99cd-25300651d3a8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:24:33.025894 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:33.025857    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w0-19f6ea/auth-test-model-19f6ea-kserve-b646899bf-7bmlq"]
Oct 30 19:24:33.026572 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:33.026553    4888 cpu_manager.go:414] "containerMap: removing container" podUID="bf0f21cf-8198-4fe4-8983-db083c82b34a" containerName="istio-proxy"
Oct 30 19:24:33.026572 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:33.026572    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bf0f21cf-8198-4fe4-8983-db083c82b34a" containerName="istio-proxy"
Oct 30 19:24:33.026799 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:33.026601    4888 cpu_manager.go:414] "containerMap: removing container" podUID="b43a09c5-9f21-4ac7-99cd-25300651d3a8" containerName="istio-proxy"
Oct 30 19:24:33.026799 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:33.026610    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b43a09c5-9f21-4ac7-99cd-25300651d3a8" containerName="istio-proxy"
Oct 30 19:24:33.026799 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:33.026644    4888 cpu_manager.go:414] "containerMap: removing container" podUID="bf0f21cf-8198-4fe4-8983-db083c82b34a" containerName="istio-proxy"
Oct 30 19:24:33.026799 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:33.026652    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bf0f21cf-8198-4fe4-8983-db083c82b34a" containerName="istio-proxy"
Oct 30 19:24:33.026799 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:33.026780    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="bf0f21cf-8198-4fe4-8983-db083c82b34a" containerName="istio-proxy"
Oct 30 19:24:33.026799 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:33.026791    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="b43a09c5-9f21-4ac7-99cd-25300651d3a8" containerName="istio-proxy"
Oct 30 19:24:33.026799 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:33.026799    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="bf0f21cf-8198-4fe4-8983-db083c82b34a" containerName="istio-proxy"
Oct 30 19:24:33.029738 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:33.029719    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w0-19f6ea/auth-test-model-19f6ea-kserve-b646899bf-7bmlq"
Oct 30 19:24:33.033747 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:33.033726    4888 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b43a09c5-9f21-4ac7-99cd-25300651d3a8-istio-token\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:24:33.033885 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:33.033869    4888 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b43a09c5-9f21-4ac7-99cd-25300651d3a8-istio-data\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:24:33.033961 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:33.033890    4888 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b43a09c5-9f21-4ac7-99cd-25300651d3a8-workload-socket\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:24:33.033961 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:33.033904    4888 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b43a09c5-9f21-4ac7-99cd-25300651d3a8-istio-envoy\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:24:33.033961 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:33.033919    4888 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b43a09c5-9f21-4ac7-99cd-25300651d3a8-istio-podinfo\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:24:33.033961 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:33.033936    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nrbcf\" (UniqueName: \"kubernetes.io/projected/b43a09c5-9f21-4ac7-99cd-25300651d3a8-kube-api-access-nrbcf\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:24:33.074950 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:33.074908    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w0-19f6ea/auth-test-model-19f6ea-kserve-b646899bf-7bmlq"]
Oct 30 19:24:33.084248 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:33.084227    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w0-19f6ea\"/\"openshift-service-ca.crt\""
Oct 30 19:24:33.084704 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:33.084634    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w0-19f6ea\"/\"kube-root-ca.crt\""
Oct 30 19:24:33.084704 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:33.084657    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w0-19f6ea\"/\"default-dockercfg-pjhmc\""
Oct 30 19:24:33.084893 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:33.084736    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w0-19f6ea\"/\"auth-test-model-19f6ea-kserve-self-signed-certs\""
Oct 30 19:24:33.134852 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:33.134818    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j6lrv\" (UniqueName: \"kubernetes.io/projected/53796da4-5dc9-40e5-8ce1-42a20807b8f0-kube-api-access-j6lrv\") pod \"auth-test-model-19f6ea-kserve-b646899bf-7bmlq\" (UID: \"53796da4-5dc9-40e5-8ce1-42a20807b8f0\") " pod="ai-tenant-e2e-shared-a-w0-19f6ea/auth-test-model-19f6ea-kserve-b646899bf-7bmlq"
Oct 30 19:24:33.135014 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:33.134960    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/53796da4-5dc9-40e5-8ce1-42a20807b8f0-home\") pod \"auth-test-model-19f6ea-kserve-b646899bf-7bmlq\" (UID: \"53796da4-5dc9-40e5-8ce1-42a20807b8f0\") " pod="ai-tenant-e2e-shared-a-w0-19f6ea/auth-test-model-19f6ea-kserve-b646899bf-7bmlq"
Oct 30 19:24:33.135014 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:33.135007    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/53796da4-5dc9-40e5-8ce1-42a20807b8f0-dshm\") pod \"auth-test-model-19f6ea-kserve-b646899bf-7bmlq\" (UID: \"53796da4-5dc9-40e5-8ce1-42a20807b8f0\") " pod="ai-tenant-e2e-shared-a-w0-19f6ea/auth-test-model-19f6ea-kserve-b646899bf-7bmlq"
Oct 30 19:24:33.135124 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:33.135037    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/53796da4-5dc9-40e5-8ce1-42a20807b8f0-tmp-dir\") pod \"auth-test-model-19f6ea-kserve-b646899bf-7bmlq\" (UID: \"53796da4-5dc9-40e5-8ce1-42a20807b8f0\") " pod="ai-tenant-e2e-shared-a-w0-19f6ea/auth-test-model-19f6ea-kserve-b646899bf-7bmlq"
Oct 30 19:24:33.135182 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:33.135161    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/53796da4-5dc9-40e5-8ce1-42a20807b8f0-model-cache\") pod \"auth-test-model-19f6ea-kserve-b646899bf-7bmlq\" (UID: \"53796da4-5dc9-40e5-8ce1-42a20807b8f0\") " pod="ai-tenant-e2e-shared-a-w0-19f6ea/auth-test-model-19f6ea-kserve-b646899bf-7bmlq"
Oct 30 19:24:33.135221 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:33.135197    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/53796da4-5dc9-40e5-8ce1-42a20807b8f0-tls-certs\") pod \"auth-test-model-19f6ea-kserve-b646899bf-7bmlq\" (UID: \"53796da4-5dc9-40e5-8ce1-42a20807b8f0\") " pod="ai-tenant-e2e-shared-a-w0-19f6ea/auth-test-model-19f6ea-kserve-b646899bf-7bmlq"
Oct 30 19:24:33.205536 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:33.205507    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-autores-c60f5c-openshift-default-9c4d97949-vk946"
Oct 30 19:24:33.206037 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:33.205951    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-autores-c60f5c-openshift-default-9c4d97949-vk946"
Oct 30 19:24:33.206740 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:33.206719    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-autores-c60f5c-openshift-default-9c4d97949-vk946"
Oct 30 19:24:33.236606 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:33.236581    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j6lrv\" (UniqueName: \"kubernetes.io/projected/53796da4-5dc9-40e5-8ce1-42a20807b8f0-kube-api-access-j6lrv\") pod \"auth-test-model-19f6ea-kserve-b646899bf-7bmlq\" (UID: \"53796da4-5dc9-40e5-8ce1-42a20807b8f0\") " pod="ai-tenant-e2e-shared-a-w0-19f6ea/auth-test-model-19f6ea-kserve-b646899bf-7bmlq"
Oct 30 19:24:33.236772 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:33.236662    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/53796da4-5dc9-40e5-8ce1-42a20807b8f0-home\") pod \"auth-test-model-19f6ea-kserve-b646899bf-7bmlq\" (UID: \"53796da4-5dc9-40e5-8ce1-42a20807b8f0\") " pod="ai-tenant-e2e-shared-a-w0-19f6ea/auth-test-model-19f6ea-kserve-b646899bf-7bmlq"
Oct 30 19:24:33.236772 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:33.236698    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/53796da4-5dc9-40e5-8ce1-42a20807b8f0-dshm\") pod \"auth-test-model-19f6ea-kserve-b646899bf-7bmlq\" (UID: \"53796da4-5dc9-40e5-8ce1-42a20807b8f0\") " pod="ai-tenant-e2e-shared-a-w0-19f6ea/auth-test-model-19f6ea-kserve-b646899bf-7bmlq"
Oct 30 19:24:33.236772 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:33.236723    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/53796da4-5dc9-40e5-8ce1-42a20807b8f0-tmp-dir\") pod \"auth-test-model-19f6ea-kserve-b646899bf-7bmlq\" (UID: \"53796da4-5dc9-40e5-8ce1-42a20807b8f0\") " pod="ai-tenant-e2e-shared-a-w0-19f6ea/auth-test-model-19f6ea-kserve-b646899bf-7bmlq"
Oct 30 19:24:33.236772 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:33.236743    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/53796da4-5dc9-40e5-8ce1-42a20807b8f0-model-cache\") pod \"auth-test-model-19f6ea-kserve-b646899bf-7bmlq\" (UID: \"53796da4-5dc9-40e5-8ce1-42a20807b8f0\") " pod="ai-tenant-e2e-shared-a-w0-19f6ea/auth-test-model-19f6ea-kserve-b646899bf-7bmlq"
Oct 30 19:24:33.237009 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:33.236871    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/53796da4-5dc9-40e5-8ce1-42a20807b8f0-tls-certs\") pod \"auth-test-model-19f6ea-kserve-b646899bf-7bmlq\" (UID: \"53796da4-5dc9-40e5-8ce1-42a20807b8f0\") " pod="ai-tenant-e2e-shared-a-w0-19f6ea/auth-test-model-19f6ea-kserve-b646899bf-7bmlq"
Oct 30 19:24:33.237125 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:33.237108    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/53796da4-5dc9-40e5-8ce1-42a20807b8f0-home\") pod \"auth-test-model-19f6ea-kserve-b646899bf-7bmlq\" (UID: \"53796da4-5dc9-40e5-8ce1-42a20807b8f0\") " pod="ai-tenant-e2e-shared-a-w0-19f6ea/auth-test-model-19f6ea-kserve-b646899bf-7bmlq"
Oct 30 19:24:33.237207 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:33.237187    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/53796da4-5dc9-40e5-8ce1-42a20807b8f0-model-cache\") pod \"auth-test-model-19f6ea-kserve-b646899bf-7bmlq\" (UID: \"53796da4-5dc9-40e5-8ce1-42a20807b8f0\") " pod="ai-tenant-e2e-shared-a-w0-19f6ea/auth-test-model-19f6ea-kserve-b646899bf-7bmlq"
Oct 30 19:24:33.237275 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:33.237230    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/53796da4-5dc9-40e5-8ce1-42a20807b8f0-tmp-dir\") pod \"auth-test-model-19f6ea-kserve-b646899bf-7bmlq\" (UID: \"53796da4-5dc9-40e5-8ce1-42a20807b8f0\") " pod="ai-tenant-e2e-shared-a-w0-19f6ea/auth-test-model-19f6ea-kserve-b646899bf-7bmlq"
Oct 30 19:24:33.239338 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:33.239323    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/53796da4-5dc9-40e5-8ce1-42a20807b8f0-dshm\") pod \"auth-test-model-19f6ea-kserve-b646899bf-7bmlq\" (UID: \"53796da4-5dc9-40e5-8ce1-42a20807b8f0\") " pod="ai-tenant-e2e-shared-a-w0-19f6ea/auth-test-model-19f6ea-kserve-b646899bf-7bmlq"
Oct 30 19:24:33.239486 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:33.239470    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/53796da4-5dc9-40e5-8ce1-42a20807b8f0-tls-certs\") pod \"auth-test-model-19f6ea-kserve-b646899bf-7bmlq\" (UID: \"53796da4-5dc9-40e5-8ce1-42a20807b8f0\") " pod="ai-tenant-e2e-shared-a-w0-19f6ea/auth-test-model-19f6ea-kserve-b646899bf-7bmlq"
Oct 30 19:24:33.299829 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:33.299771    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j6lrv\" (UniqueName: \"kubernetes.io/projected/53796da4-5dc9-40e5-8ce1-42a20807b8f0-kube-api-access-j6lrv\") pod \"auth-test-model-19f6ea-kserve-b646899bf-7bmlq\" (UID: \"53796da4-5dc9-40e5-8ce1-42a20807b8f0\") " pod="ai-tenant-e2e-shared-a-w0-19f6ea/auth-test-model-19f6ea-kserve-b646899bf-7bmlq"
Oct 30 19:24:33.342860 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:33.342835    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w0-19f6ea/auth-test-model-19f6ea-kserve-b646899bf-7bmlq"
Oct 30 19:24:33.448422 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:33.448387    4888 generic.go:356] "Generic (PLEG): container finished" podID="b43a09c5-9f21-4ac7-99cd-25300651d3a8" containerID="e3cbb8fa9ae750d9f607caadb0d4953bd778a6e860f96e68cbba4d4279b69d95" exitCode=0
Oct 30 19:24:33.451333 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:33.449202    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-jhlsk"
Oct 30 19:24:33.451333 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:33.450227    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-jhlsk" event={"ID":"b43a09c5-9f21-4ac7-99cd-25300651d3a8","Type":"ContainerDied","Data":"e3cbb8fa9ae750d9f607caadb0d4953bd778a6e860f96e68cbba4d4279b69d95"}
Oct 30 19:24:33.451333 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:33.450296    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-jhlsk" event={"ID":"b43a09c5-9f21-4ac7-99cd-25300651d3a8","Type":"ContainerDied","Data":"6260fe7adcd519aca28cb285325df36753e0a9cfa1012417c6428d675647bd26"}
Oct 30 19:24:33.451333 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:33.450325    4888 scope.go:122] "RemoveContainer" containerID="e3cbb8fa9ae750d9f607caadb0d4953bd778a6e860f96e68cbba4d4279b69d95"
Oct 30 19:24:33.540089 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:33.538901    4888 scope.go:122] "RemoveContainer" containerID="e3cbb8fa9ae750d9f607caadb0d4953bd778a6e860f96e68cbba4d4279b69d95"
Oct 30 19:24:33.540536 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:24:33.540510    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e3cbb8fa9ae750d9f607caadb0d4953bd778a6e860f96e68cbba4d4279b69d95\": container with ID starting with e3cbb8fa9ae750d9f607caadb0d4953bd778a6e860f96e68cbba4d4279b69d95 not found: ID does not exist" containerID="e3cbb8fa9ae750d9f607caadb0d4953bd778a6e860f96e68cbba4d4279b69d95"
Oct 30 19:24:33.540685 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:33.540542    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e3cbb8fa9ae750d9f607caadb0d4953bd778a6e860f96e68cbba4d4279b69d95"} err="failed to get container status \"e3cbb8fa9ae750d9f607caadb0d4953bd778a6e860f96e68cbba4d4279b69d95\": rpc error: code = NotFound desc = could not find container \"e3cbb8fa9ae750d9f607caadb0d4953bd778a6e860f96e68cbba4d4279b69d95\": container with ID starting with e3cbb8fa9ae750d9f607caadb0d4953bd778a6e860f96e68cbba4d4279b69d95 not found: ID does not exist"
Oct 30 19:24:33.654948 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:33.654911    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-jhlsk"]
Oct 30 19:24:33.663326 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:33.663300    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-jhlsk"]
Oct 30 19:24:33.770304 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:33.770263    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w0-19f6ea/auth-test-model-19f6ea-kserve-b646899bf-7bmlq"]
Oct 30 19:24:33.771102 ip-10-0-134-46 kubenswrapper[4888]: W1030 19:24:33.771078    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod53796da4_5dc9_40e5_8ce1_42a20807b8f0.slice/crio-ffba741576506023fc51deba9960e0bc6af170d90b316ff66470c697702b8ef9 WatchSource:0}: Error finding container ffba741576506023fc51deba9960e0bc6af170d90b316ff66470c697702b8ef9: Status 404 returned error can't find the container with id ffba741576506023fc51deba9960e0bc6af170d90b316ff66470c697702b8ef9
Oct 30 19:24:33.893396 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:33.893360    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b43a09c5-9f21-4ac7-99cd-25300651d3a8" path="/var/lib/kubelet/pods/b43a09c5-9f21-4ac7-99cd-25300651d3a8/volumes"
Oct 30 19:24:34.457483 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:34.457445    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w0-19f6ea/auth-test-model-19f6ea-kserve-b646899bf-7bmlq" event={"ID":"53796da4-5dc9-40e5-8ce1-42a20807b8f0","Type":"ContainerStarted","Data":"78b2a9c9da7d201419a810890cfe3442f4de5291a6ef886058889ac812abeb4c"}
Oct 30 19:24:34.457483 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:34.457487    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w0-19f6ea/auth-test-model-19f6ea-kserve-b646899bf-7bmlq" event={"ID":"53796da4-5dc9-40e5-8ce1-42a20807b8f0","Type":"ContainerStarted","Data":"ffba741576506023fc51deba9960e0bc6af170d90b316ff66470c697702b8ef9"}
Oct 30 19:24:34.579866 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:34.579814    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w0-19f6ea/auth-test-model-19f6ea-kserve-b646899bf-7bmlq" podStartSLOduration=2.5797999640000002 podStartE2EDuration="2.579799964s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:24:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:24:34.577648964 +0000 UTC m=+3038.825054985" watchObservedRunningTime="2026-10-30 19:24:34.579799964 +0000 UTC m=+3038.827205988"
Oct 30 19:24:35.280473 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:35.280431    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-wkl4b"]
Oct 30 19:24:35.280760 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:35.280728    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-wkl4b" podUID="f88bf4ae-259e-4a89-9c1e-70bc180d4b98" containerName="istio-proxy" containerID="cri-o://dd8d40d15069c966d73bbe0572e7f0d5c7d0f2288a7c5f5d7e081660db419f06" gracePeriod=30
Oct 30 19:24:35.343752 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:35.343720    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w0-19f6ea/auth-test-model-19f6ea-kserve-b646899bf-7bmlq"
Oct 30 19:24:35.619135 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:24:35.619099    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-jbsw6" containerName="extractor"
Oct 30 19:24:40.512710 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:40.512677    4888 generic.go:356] "Generic (PLEG): container finished" podID="f88bf4ae-259e-4a89-9c1e-70bc180d4b98" containerID="dd8d40d15069c966d73bbe0572e7f0d5c7d0f2288a7c5f5d7e081660db419f06" exitCode=0
Oct 30 19:24:40.513062 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:40.512746    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-wkl4b" event={"ID":"f88bf4ae-259e-4a89-9c1e-70bc180d4b98","Type":"ContainerDied","Data":"dd8d40d15069c966d73bbe0572e7f0d5c7d0f2288a7c5f5d7e081660db419f06"}
Oct 30 19:24:40.565502 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:40.565482    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-wkl4b"
Oct 30 19:24:40.632514 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:40.632493    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f88bf4ae-259e-4a89-9c1e-70bc180d4b98-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f88bf4ae-259e-4a89-9c1e-70bc180d4b98-credential-socket\") pod \"f88bf4ae-259e-4a89-9c1e-70bc180d4b98\" (UID: \"f88bf4ae-259e-4a89-9c1e-70bc180d4b98\") "
Oct 30 19:24:40.632657 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:40.632551    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f88bf4ae-259e-4a89-9c1e-70bc180d4b98-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f88bf4ae-259e-4a89-9c1e-70bc180d4b98-workload-certs\") pod \"f88bf4ae-259e-4a89-9c1e-70bc180d4b98\" (UID: \"f88bf4ae-259e-4a89-9c1e-70bc180d4b98\") "
Oct 30 19:24:40.632657 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:40.632576    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f88bf4ae-259e-4a89-9c1e-70bc180d4b98-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f88bf4ae-259e-4a89-9c1e-70bc180d4b98-istio-data\") pod \"f88bf4ae-259e-4a89-9c1e-70bc180d4b98\" (UID: \"f88bf4ae-259e-4a89-9c1e-70bc180d4b98\") "
Oct 30 19:24:40.632657 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:40.632617    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f88bf4ae-259e-4a89-9c1e-70bc180d4b98-kube-api-access-9d2vc\" (UniqueName: \"kubernetes.io/projected/f88bf4ae-259e-4a89-9c1e-70bc180d4b98-kube-api-access-9d2vc\") pod \"f88bf4ae-259e-4a89-9c1e-70bc180d4b98\" (UID: \"f88bf4ae-259e-4a89-9c1e-70bc180d4b98\") "
Oct 30 19:24:40.632657 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:40.632645    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f88bf4ae-259e-4a89-9c1e-70bc180d4b98-istio-token\" (UniqueName: \"kubernetes.io/projected/f88bf4ae-259e-4a89-9c1e-70bc180d4b98-istio-token\") pod \"f88bf4ae-259e-4a89-9c1e-70bc180d4b98\" (UID: \"f88bf4ae-259e-4a89-9c1e-70bc180d4b98\") "
Oct 30 19:24:40.632892 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:40.632689    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f88bf4ae-259e-4a89-9c1e-70bc180d4b98-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f88bf4ae-259e-4a89-9c1e-70bc180d4b98-istiod-ca-cert\") pod \"f88bf4ae-259e-4a89-9c1e-70bc180d4b98\" (UID: \"f88bf4ae-259e-4a89-9c1e-70bc180d4b98\") "
Oct 30 19:24:40.632892 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:40.632744    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/f88bf4ae-259e-4a89-9c1e-70bc180d4b98-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f88bf4ae-259e-4a89-9c1e-70bc180d4b98-istio-podinfo\") pod \"f88bf4ae-259e-4a89-9c1e-70bc180d4b98\" (UID: \"f88bf4ae-259e-4a89-9c1e-70bc180d4b98\") "
Oct 30 19:24:40.632892 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:40.632740    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f88bf4ae-259e-4a89-9c1e-70bc180d4b98-credential-socket" pod "f88bf4ae-259e-4a89-9c1e-70bc180d4b98" (UID: "f88bf4ae-259e-4a89-9c1e-70bc180d4b98"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:24:40.632892 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:40.632822    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f88bf4ae-259e-4a89-9c1e-70bc180d4b98-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f88bf4ae-259e-4a89-9c1e-70bc180d4b98-istio-envoy\") pod \"f88bf4ae-259e-4a89-9c1e-70bc180d4b98\" (UID: \"f88bf4ae-259e-4a89-9c1e-70bc180d4b98\") "
Oct 30 19:24:40.632892 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:40.632860    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f88bf4ae-259e-4a89-9c1e-70bc180d4b98-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f88bf4ae-259e-4a89-9c1e-70bc180d4b98-workload-socket\") pod \"f88bf4ae-259e-4a89-9c1e-70bc180d4b98\" (UID: \"f88bf4ae-259e-4a89-9c1e-70bc180d4b98\") "
Oct 30 19:24:40.633156 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:40.632940    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f88bf4ae-259e-4a89-9c1e-70bc180d4b98-workload-certs" pod "f88bf4ae-259e-4a89-9c1e-70bc180d4b98" (UID: "f88bf4ae-259e-4a89-9c1e-70bc180d4b98"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:24:40.633156 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:40.632961    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f88bf4ae-259e-4a89-9c1e-70bc180d4b98-istio-data" pod "f88bf4ae-259e-4a89-9c1e-70bc180d4b98" (UID: "f88bf4ae-259e-4a89-9c1e-70bc180d4b98"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:24:40.633263 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:40.633222    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f88bf4ae-259e-4a89-9c1e-70bc180d4b98-workload-socket" pod "f88bf4ae-259e-4a89-9c1e-70bc180d4b98" (UID: "f88bf4ae-259e-4a89-9c1e-70bc180d4b98"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:24:40.633416 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:40.633397    4888 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f88bf4ae-259e-4a89-9c1e-70bc180d4b98-workload-socket\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:24:40.633456 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:40.633423    4888 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f88bf4ae-259e-4a89-9c1e-70bc180d4b98-credential-socket\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:24:40.633456 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:40.633440    4888 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f88bf4ae-259e-4a89-9c1e-70bc180d4b98-workload-certs\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:24:40.633523 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:40.633453    4888 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f88bf4ae-259e-4a89-9c1e-70bc180d4b98-istio-data\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:24:40.633523 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:40.633451    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f88bf4ae-259e-4a89-9c1e-70bc180d4b98-istiod-ca-cert" pod "f88bf4ae-259e-4a89-9c1e-70bc180d4b98" (UID: "f88bf4ae-259e-4a89-9c1e-70bc180d4b98"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:24:40.635107 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:40.635078    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f88bf4ae-259e-4a89-9c1e-70bc180d4b98-istio-envoy" pod "f88bf4ae-259e-4a89-9c1e-70bc180d4b98" (UID: "f88bf4ae-259e-4a89-9c1e-70bc180d4b98"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:24:40.635370 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:40.635345    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f88bf4ae-259e-4a89-9c1e-70bc180d4b98-istio-podinfo" pod "f88bf4ae-259e-4a89-9c1e-70bc180d4b98" (UID: "f88bf4ae-259e-4a89-9c1e-70bc180d4b98"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:24:40.635639 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:40.635610    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f88bf4ae-259e-4a89-9c1e-70bc180d4b98-kube-api-access-9d2vc" pod "f88bf4ae-259e-4a89-9c1e-70bc180d4b98" (UID: "f88bf4ae-259e-4a89-9c1e-70bc180d4b98"). InnerVolumeSpecName "kube-api-access-9d2vc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:24:40.635639 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:40.635629    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f88bf4ae-259e-4a89-9c1e-70bc180d4b98-istio-token" pod "f88bf4ae-259e-4a89-9c1e-70bc180d4b98" (UID: "f88bf4ae-259e-4a89-9c1e-70bc180d4b98"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:24:40.734617 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:40.734589    4888 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f88bf4ae-259e-4a89-9c1e-70bc180d4b98-istio-podinfo\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:24:40.734617 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:40.734618    4888 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f88bf4ae-259e-4a89-9c1e-70bc180d4b98-istio-envoy\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:24:40.734804 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:40.734634    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9d2vc\" (UniqueName: \"kubernetes.io/projected/f88bf4ae-259e-4a89-9c1e-70bc180d4b98-kube-api-access-9d2vc\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:24:40.734804 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:40.734650    4888 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f88bf4ae-259e-4a89-9c1e-70bc180d4b98-istio-token\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:24:40.734804 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:40.734663    4888 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f88bf4ae-259e-4a89-9c1e-70bc180d4b98-istiod-ca-cert\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:24:41.521149 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:41.521119    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-wkl4b"
Oct 30 19:24:41.521755 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:41.521122    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-wkl4b" event={"ID":"f88bf4ae-259e-4a89-9c1e-70bc180d4b98","Type":"ContainerDied","Data":"a263211a43b9426344b33fbc27893f499350b2c34ac88f774d6d5df61c31117d"}
Oct 30 19:24:41.521755 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:41.521251    4888 scope.go:122] "RemoveContainer" containerID="dd8d40d15069c966d73bbe0572e7f0d5c7d0f2288a7c5f5d7e081660db419f06"
Oct 30 19:24:41.742376 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:41.742345    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-wkl4b"]
Oct 30 19:24:41.793025 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:41.792949    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-2ade09-openshift-default-785b7cf675-wkl4b"]
Oct 30 19:24:41.892910 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:41.892871    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f88bf4ae-259e-4a89-9c1e-70bc180d4b98" path="/var/lib/kubelet/pods/f88bf4ae-259e-4a89-9c1e-70bc180d4b98/volumes"
Oct 30 19:24:43.343026 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:43.342994    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w0-19f6ea/auth-test-model-19f6ea-kserve-b646899bf-7bmlq"
Oct 30 19:24:43.355609 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:43.355587    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w0-19f6ea/auth-test-model-19f6ea-kserve-b646899bf-7bmlq"
Oct 30 19:24:43.549575 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:43.549548    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w0-19f6ea/auth-test-model-19f6ea-kserve-b646899bf-7bmlq"
Oct 30 19:24:55.264423 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:24:55.264386    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-rjmdz"]
Oct 30 19:25:05.618552 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:25:05.618520    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-jbsw6" containerName="extractor"
Oct 30 19:25:07.827715 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:25:07.827680    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w2-dc5cdd/auth-test-model-dc5cdd-kserve-7cb964d99c-9znzj"]
Oct 30 19:25:07.828322 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:25:07.828301    4888 cpu_manager.go:414] "containerMap: removing container" podUID="f88bf4ae-259e-4a89-9c1e-70bc180d4b98" containerName="istio-proxy"
Oct 30 19:25:07.828322 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:25:07.828320    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f88bf4ae-259e-4a89-9c1e-70bc180d4b98" containerName="istio-proxy"
Oct 30 19:25:07.828485 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:25:07.828427    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="f88bf4ae-259e-4a89-9c1e-70bc180d4b98" containerName="istio-proxy"
Oct 30 19:25:07.831938 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:25:07.831922    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w2-dc5cdd/auth-test-model-dc5cdd-kserve-7cb964d99c-9znzj"
Oct 30 19:25:07.850731 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:25:07.850709    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f0b4d9fd-a0dd-4fb3-8607-c0af3fd3d030-tls-certs\") pod \"auth-test-model-dc5cdd-kserve-7cb964d99c-9znzj\" (UID: \"f0b4d9fd-a0dd-4fb3-8607-c0af3fd3d030\") " pod="ai-tenant-e2e-shared-b-w2-dc5cdd/auth-test-model-dc5cdd-kserve-7cb964d99c-9znzj"
Oct 30 19:25:07.850865 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:25:07.850748    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-twg8l\" (UniqueName: \"kubernetes.io/projected/f0b4d9fd-a0dd-4fb3-8607-c0af3fd3d030-kube-api-access-twg8l\") pod \"auth-test-model-dc5cdd-kserve-7cb964d99c-9znzj\" (UID: \"f0b4d9fd-a0dd-4fb3-8607-c0af3fd3d030\") " pod="ai-tenant-e2e-shared-b-w2-dc5cdd/auth-test-model-dc5cdd-kserve-7cb964d99c-9znzj"
Oct 30 19:25:07.850865 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:25:07.850830    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f0b4d9fd-a0dd-4fb3-8607-c0af3fd3d030-tmp-dir\") pod \"auth-test-model-dc5cdd-kserve-7cb964d99c-9znzj\" (UID: \"f0b4d9fd-a0dd-4fb3-8607-c0af3fd3d030\") " pod="ai-tenant-e2e-shared-b-w2-dc5cdd/auth-test-model-dc5cdd-kserve-7cb964d99c-9znzj"
Oct 30 19:25:07.850959 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:25:07.850900    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f0b4d9fd-a0dd-4fb3-8607-c0af3fd3d030-home\") pod \"auth-test-model-dc5cdd-kserve-7cb964d99c-9znzj\" (UID: \"f0b4d9fd-a0dd-4fb3-8607-c0af3fd3d030\") " pod="ai-tenant-e2e-shared-b-w2-dc5cdd/auth-test-model-dc5cdd-kserve-7cb964d99c-9znzj"
Oct 30 19:25:07.851019 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:25:07.850956    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f0b4d9fd-a0dd-4fb3-8607-c0af3fd3d030-model-cache\") pod \"auth-test-model-dc5cdd-kserve-7cb964d99c-9znzj\" (UID: \"f0b4d9fd-a0dd-4fb3-8607-c0af3fd3d030\") " pod="ai-tenant-e2e-shared-b-w2-dc5cdd/auth-test-model-dc5cdd-kserve-7cb964d99c-9znzj"
Oct 30 19:25:07.851082 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:25:07.851063    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f0b4d9fd-a0dd-4fb3-8607-c0af3fd3d030-dshm\") pod \"auth-test-model-dc5cdd-kserve-7cb964d99c-9znzj\" (UID: \"f0b4d9fd-a0dd-4fb3-8607-c0af3fd3d030\") " pod="ai-tenant-e2e-shared-b-w2-dc5cdd/auth-test-model-dc5cdd-kserve-7cb964d99c-9znzj"
Oct 30 19:25:07.938901 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:25:07.938863    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w2-dc5cdd\"/\"kube-root-ca.crt\""
Oct 30 19:25:07.938901 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:25:07.938863    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w2-dc5cdd\"/\"auth-test-model-dc5cdd-kserve-self-signed-certs\""
Oct 30 19:25:07.939135 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:25:07.938914    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w2-dc5cdd\"/\"openshift-service-ca.crt\""
Oct 30 19:25:07.940459 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:25:07.940439    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w2-dc5cdd\"/\"default-dockercfg-j9tv4\""
Oct 30 19:25:07.942609 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:25:07.942591    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w2-dc5cdd/auth-test-model-dc5cdd-kserve-7cb964d99c-9znzj"]
Oct 30 19:25:07.952205 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:25:07.952180    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f0b4d9fd-a0dd-4fb3-8607-c0af3fd3d030-tls-certs\") pod \"auth-test-model-dc5cdd-kserve-7cb964d99c-9znzj\" (UID: \"f0b4d9fd-a0dd-4fb3-8607-c0af3fd3d030\") " pod="ai-tenant-e2e-shared-b-w2-dc5cdd/auth-test-model-dc5cdd-kserve-7cb964d99c-9znzj"
Oct 30 19:25:07.952321 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:25:07.952222    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-twg8l\" (UniqueName: \"kubernetes.io/projected/f0b4d9fd-a0dd-4fb3-8607-c0af3fd3d030-kube-api-access-twg8l\") pod \"auth-test-model-dc5cdd-kserve-7cb964d99c-9znzj\" (UID: \"f0b4d9fd-a0dd-4fb3-8607-c0af3fd3d030\") " pod="ai-tenant-e2e-shared-b-w2-dc5cdd/auth-test-model-dc5cdd-kserve-7cb964d99c-9znzj"
Oct 30 19:25:07.952321 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:25:07.952257    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f0b4d9fd-a0dd-4fb3-8607-c0af3fd3d030-tmp-dir\") pod \"auth-test-model-dc5cdd-kserve-7cb964d99c-9znzj\" (UID: \"f0b4d9fd-a0dd-4fb3-8607-c0af3fd3d030\") " pod="ai-tenant-e2e-shared-b-w2-dc5cdd/auth-test-model-dc5cdd-kserve-7cb964d99c-9znzj"
Oct 30 19:25:07.952437 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:25:07.952338    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f0b4d9fd-a0dd-4fb3-8607-c0af3fd3d030-home\") pod \"auth-test-model-dc5cdd-kserve-7cb964d99c-9znzj\" (UID: \"f0b4d9fd-a0dd-4fb3-8607-c0af3fd3d030\") " pod="ai-tenant-e2e-shared-b-w2-dc5cdd/auth-test-model-dc5cdd-kserve-7cb964d99c-9znzj"
Oct 30 19:25:07.952494 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:25:07.952420    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f0b4d9fd-a0dd-4fb3-8607-c0af3fd3d030-model-cache\") pod \"auth-test-model-dc5cdd-kserve-7cb964d99c-9znzj\" (UID: \"f0b4d9fd-a0dd-4fb3-8607-c0af3fd3d030\") " pod="ai-tenant-e2e-shared-b-w2-dc5cdd/auth-test-model-dc5cdd-kserve-7cb964d99c-9znzj"
Oct 30 19:25:07.952542 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:25:07.952500    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f0b4d9fd-a0dd-4fb3-8607-c0af3fd3d030-dshm\") pod \"auth-test-model-dc5cdd-kserve-7cb964d99c-9znzj\" (UID: \"f0b4d9fd-a0dd-4fb3-8607-c0af3fd3d030\") " pod="ai-tenant-e2e-shared-b-w2-dc5cdd/auth-test-model-dc5cdd-kserve-7cb964d99c-9znzj"
Oct 30 19:25:07.952850 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:25:07.952827    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f0b4d9fd-a0dd-4fb3-8607-c0af3fd3d030-home\") pod \"auth-test-model-dc5cdd-kserve-7cb964d99c-9znzj\" (UID: \"f0b4d9fd-a0dd-4fb3-8607-c0af3fd3d030\") " pod="ai-tenant-e2e-shared-b-w2-dc5cdd/auth-test-model-dc5cdd-kserve-7cb964d99c-9znzj"
Oct 30 19:25:07.952924 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:25:07.952859    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f0b4d9fd-a0dd-4fb3-8607-c0af3fd3d030-model-cache\") pod \"auth-test-model-dc5cdd-kserve-7cb964d99c-9znzj\" (UID: \"f0b4d9fd-a0dd-4fb3-8607-c0af3fd3d030\") " pod="ai-tenant-e2e-shared-b-w2-dc5cdd/auth-test-model-dc5cdd-kserve-7cb964d99c-9znzj"
Oct 30 19:25:07.952924 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:25:07.952881    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f0b4d9fd-a0dd-4fb3-8607-c0af3fd3d030-tmp-dir\") pod \"auth-test-model-dc5cdd-kserve-7cb964d99c-9znzj\" (UID: \"f0b4d9fd-a0dd-4fb3-8607-c0af3fd3d030\") " pod="ai-tenant-e2e-shared-b-w2-dc5cdd/auth-test-model-dc5cdd-kserve-7cb964d99c-9znzj"
Oct 30 19:25:07.954983 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:25:07.954959    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f0b4d9fd-a0dd-4fb3-8607-c0af3fd3d030-dshm\") pod \"auth-test-model-dc5cdd-kserve-7cb964d99c-9znzj\" (UID: \"f0b4d9fd-a0dd-4fb3-8607-c0af3fd3d030\") " pod="ai-tenant-e2e-shared-b-w2-dc5cdd/auth-test-model-dc5cdd-kserve-7cb964d99c-9znzj"
Oct 30 19:25:07.955269 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:25:07.955252    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f0b4d9fd-a0dd-4fb3-8607-c0af3fd3d030-tls-certs\") pod \"auth-test-model-dc5cdd-kserve-7cb964d99c-9znzj\" (UID: \"f0b4d9fd-a0dd-4fb3-8607-c0af3fd3d030\") " pod="ai-tenant-e2e-shared-b-w2-dc5cdd/auth-test-model-dc5cdd-kserve-7cb964d99c-9znzj"
Oct 30 19:25:08.019291 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:25:08.019255    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-twg8l\" (UniqueName: \"kubernetes.io/projected/f0b4d9fd-a0dd-4fb3-8607-c0af3fd3d030-kube-api-access-twg8l\") pod \"auth-test-model-dc5cdd-kserve-7cb964d99c-9znzj\" (UID: \"f0b4d9fd-a0dd-4fb3-8607-c0af3fd3d030\") " pod="ai-tenant-e2e-shared-b-w2-dc5cdd/auth-test-model-dc5cdd-kserve-7cb964d99c-9znzj"
Oct 30 19:25:08.143644 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:25:08.143552    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w2-dc5cdd/auth-test-model-dc5cdd-kserve-7cb964d99c-9znzj"
Oct 30 19:25:08.529375 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:25:08.529345    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w2-dc5cdd/auth-test-model-dc5cdd-kserve-7cb964d99c-9znzj"]
Oct 30 19:25:08.529883 ip-10-0-134-46 kubenswrapper[4888]: W1030 19:25:08.529862    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf0b4d9fd_a0dd_4fb3_8607_c0af3fd3d030.slice/crio-35fd9081ae751913c2d93eafcb729be7e61bbc9155ebd466d5dc002404f3e771 WatchSource:0}: Error finding container 35fd9081ae751913c2d93eafcb729be7e61bbc9155ebd466d5dc002404f3e771: Status 404 returned error can't find the container with id 35fd9081ae751913c2d93eafcb729be7e61bbc9155ebd466d5dc002404f3e771
Oct 30 19:25:08.729524 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:25:08.729492    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w2-dc5cdd/auth-test-model-dc5cdd-kserve-7cb964d99c-9znzj" event={"ID":"f0b4d9fd-a0dd-4fb3-8607-c0af3fd3d030","Type":"ContainerStarted","Data":"65662cd374620364883ab08b496a5ab1ba5a37b631a58a04c863f7a69dcc9644"}
Oct 30 19:25:08.729691 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:25:08.729537    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w2-dc5cdd/auth-test-model-dc5cdd-kserve-7cb964d99c-9znzj" event={"ID":"f0b4d9fd-a0dd-4fb3-8607-c0af3fd3d030","Type":"ContainerStarted","Data":"35fd9081ae751913c2d93eafcb729be7e61bbc9155ebd466d5dc002404f3e771"}
Oct 30 19:25:08.872544 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:25:08.872490    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w2-dc5cdd/auth-test-model-dc5cdd-kserve-7cb964d99c-9znzj" podStartSLOduration=1.8724740469999999 podStartE2EDuration="1.872474047s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:25:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:25:08.869198596 +0000 UTC m=+3073.116604623" watchObservedRunningTime="2026-10-30 19:25:08.872474047 +0000 UTC m=+3073.119880098"
Oct 30 19:25:09.143995 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:25:09.143911    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w2-dc5cdd/auth-test-model-dc5cdd-kserve-7cb964d99c-9znzj"
Oct 30 19:25:18.144491 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:25:18.144461    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w2-dc5cdd/auth-test-model-dc5cdd-kserve-7cb964d99c-9znzj"
Oct 30 19:25:18.157446 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:25:18.157423    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w2-dc5cdd/auth-test-model-dc5cdd-kserve-7cb964d99c-9znzj"
Oct 30 19:25:18.820709 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:25:18.820677    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w2-dc5cdd/auth-test-model-dc5cdd-kserve-7cb964d99c-9znzj"
Oct 30 19:25:33.182953 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:25:33.182918    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-rjmdz"]
Oct 30 19:25:35.618102 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:25:35.618023    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-jbsw6" containerName="extractor"
Oct 30 19:25:45.640436 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:25:45.640406    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w1-d4f84d/auth-test-model-d4f84d-kserve-5b9745569c-h4pcq"]
Oct 30 19:25:45.644757 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:25:45.644736    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w1-d4f84d/auth-test-model-d4f84d-kserve-5b9745569c-h4pcq"
Oct 30 19:25:45.752890 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:25:45.752868    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w1-d4f84d\"/\"default-dockercfg-qz6fz\""
Oct 30 19:25:45.753143 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:25:45.753126    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w1-d4f84d\"/\"openshift-service-ca.crt\""
Oct 30 19:25:45.753237 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:25:45.753199    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w1-d4f84d\"/\"kube-root-ca.crt\""
Oct 30 19:25:45.753952 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:25:45.753939    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w1-d4f84d\"/\"auth-test-model-d4f84d-kserve-self-signed-certs\""
Oct 30 19:25:45.756793 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:25:45.756772    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w1-d4f84d/auth-test-model-d4f84d-kserve-5b9745569c-h4pcq"]
Oct 30 19:25:45.822168 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:25:45.822120    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/362aa1c7-3163-49ee-9ed9-6a965a4a9a56-home\") pod \"auth-test-model-d4f84d-kserve-5b9745569c-h4pcq\" (UID: \"362aa1c7-3163-49ee-9ed9-6a965a4a9a56\") " pod="ai-tenant-e2e-shared-b-w1-d4f84d/auth-test-model-d4f84d-kserve-5b9745569c-h4pcq"
Oct 30 19:25:45.822418 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:25:45.822196    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fkz9n\" (UniqueName: \"kubernetes.io/projected/362aa1c7-3163-49ee-9ed9-6a965a4a9a56-kube-api-access-fkz9n\") pod \"auth-test-model-d4f84d-kserve-5b9745569c-h4pcq\" (UID: \"362aa1c7-3163-49ee-9ed9-6a965a4a9a56\") " pod="ai-tenant-e2e-shared-b-w1-d4f84d/auth-test-model-d4f84d-kserve-5b9745569c-h4pcq"
Oct 30 19:25:45.822418 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:25:45.822239    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/362aa1c7-3163-49ee-9ed9-6a965a4a9a56-dshm\") pod \"auth-test-model-d4f84d-kserve-5b9745569c-h4pcq\" (UID: \"362aa1c7-3163-49ee-9ed9-6a965a4a9a56\") " pod="ai-tenant-e2e-shared-b-w1-d4f84d/auth-test-model-d4f84d-kserve-5b9745569c-h4pcq"
Oct 30 19:25:45.822562 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:25:45.822468    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/362aa1c7-3163-49ee-9ed9-6a965a4a9a56-model-cache\") pod \"auth-test-model-d4f84d-kserve-5b9745569c-h4pcq\" (UID: \"362aa1c7-3163-49ee-9ed9-6a965a4a9a56\") " pod="ai-tenant-e2e-shared-b-w1-d4f84d/auth-test-model-d4f84d-kserve-5b9745569c-h4pcq"
Oct 30 19:25:45.822627 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:25:45.822574    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/362aa1c7-3163-49ee-9ed9-6a965a4a9a56-tmp-dir\") pod \"auth-test-model-d4f84d-kserve-5b9745569c-h4pcq\" (UID: \"362aa1c7-3163-49ee-9ed9-6a965a4a9a56\") " pod="ai-tenant-e2e-shared-b-w1-d4f84d/auth-test-model-d4f84d-kserve-5b9745569c-h4pcq"
Oct 30 19:25:45.822627 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:25:45.822620    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/362aa1c7-3163-49ee-9ed9-6a965a4a9a56-tls-certs\") pod \"auth-test-model-d4f84d-kserve-5b9745569c-h4pcq\" (UID: \"362aa1c7-3163-49ee-9ed9-6a965a4a9a56\") " pod="ai-tenant-e2e-shared-b-w1-d4f84d/auth-test-model-d4f84d-kserve-5b9745569c-h4pcq"
Oct 30 19:25:45.924166 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:25:45.924088    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/362aa1c7-3163-49ee-9ed9-6a965a4a9a56-tmp-dir\") pod \"auth-test-model-d4f84d-kserve-5b9745569c-h4pcq\" (UID: \"362aa1c7-3163-49ee-9ed9-6a965a4a9a56\") " pod="ai-tenant-e2e-shared-b-w1-d4f84d/auth-test-model-d4f84d-kserve-5b9745569c-h4pcq"
Oct 30 19:25:45.924166 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:25:45.924127    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/362aa1c7-3163-49ee-9ed9-6a965a4a9a56-tls-certs\") pod \"auth-test-model-d4f84d-kserve-5b9745569c-h4pcq\" (UID: \"362aa1c7-3163-49ee-9ed9-6a965a4a9a56\") " pod="ai-tenant-e2e-shared-b-w1-d4f84d/auth-test-model-d4f84d-kserve-5b9745569c-h4pcq"
Oct 30 19:25:45.924414 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:25:45.924190    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/362aa1c7-3163-49ee-9ed9-6a965a4a9a56-home\") pod \"auth-test-model-d4f84d-kserve-5b9745569c-h4pcq\" (UID: \"362aa1c7-3163-49ee-9ed9-6a965a4a9a56\") " pod="ai-tenant-e2e-shared-b-w1-d4f84d/auth-test-model-d4f84d-kserve-5b9745569c-h4pcq"
Oct 30 19:25:45.924414 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:25:45.924230    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fkz9n\" (UniqueName: \"kubernetes.io/projected/362aa1c7-3163-49ee-9ed9-6a965a4a9a56-kube-api-access-fkz9n\") pod \"auth-test-model-d4f84d-kserve-5b9745569c-h4pcq\" (UID: \"362aa1c7-3163-49ee-9ed9-6a965a4a9a56\") " pod="ai-tenant-e2e-shared-b-w1-d4f84d/auth-test-model-d4f84d-kserve-5b9745569c-h4pcq"
Oct 30 19:25:45.924414 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:25:45.924254    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/362aa1c7-3163-49ee-9ed9-6a965a4a9a56-dshm\") pod \"auth-test-model-d4f84d-kserve-5b9745569c-h4pcq\" (UID: \"362aa1c7-3163-49ee-9ed9-6a965a4a9a56\") " pod="ai-tenant-e2e-shared-b-w1-d4f84d/auth-test-model-d4f84d-kserve-5b9745569c-h4pcq"
Oct 30 19:25:45.924414 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:25:45.924336    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/362aa1c7-3163-49ee-9ed9-6a965a4a9a56-model-cache\") pod \"auth-test-model-d4f84d-kserve-5b9745569c-h4pcq\" (UID: \"362aa1c7-3163-49ee-9ed9-6a965a4a9a56\") " pod="ai-tenant-e2e-shared-b-w1-d4f84d/auth-test-model-d4f84d-kserve-5b9745569c-h4pcq"
Oct 30 19:25:45.924627 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:25:45.924570    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/362aa1c7-3163-49ee-9ed9-6a965a4a9a56-tmp-dir\") pod \"auth-test-model-d4f84d-kserve-5b9745569c-h4pcq\" (UID: \"362aa1c7-3163-49ee-9ed9-6a965a4a9a56\") " pod="ai-tenant-e2e-shared-b-w1-d4f84d/auth-test-model-d4f84d-kserve-5b9745569c-h4pcq"
Oct 30 19:25:45.924627 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:25:45.924607    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/362aa1c7-3163-49ee-9ed9-6a965a4a9a56-home\") pod \"auth-test-model-d4f84d-kserve-5b9745569c-h4pcq\" (UID: \"362aa1c7-3163-49ee-9ed9-6a965a4a9a56\") " pod="ai-tenant-e2e-shared-b-w1-d4f84d/auth-test-model-d4f84d-kserve-5b9745569c-h4pcq"
Oct 30 19:25:45.924737 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:25:45.924626    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/362aa1c7-3163-49ee-9ed9-6a965a4a9a56-model-cache\") pod \"auth-test-model-d4f84d-kserve-5b9745569c-h4pcq\" (UID: \"362aa1c7-3163-49ee-9ed9-6a965a4a9a56\") " pod="ai-tenant-e2e-shared-b-w1-d4f84d/auth-test-model-d4f84d-kserve-5b9745569c-h4pcq"
Oct 30 19:25:45.926725 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:25:45.926699    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/362aa1c7-3163-49ee-9ed9-6a965a4a9a56-dshm\") pod \"auth-test-model-d4f84d-kserve-5b9745569c-h4pcq\" (UID: \"362aa1c7-3163-49ee-9ed9-6a965a4a9a56\") " pod="ai-tenant-e2e-shared-b-w1-d4f84d/auth-test-model-d4f84d-kserve-5b9745569c-h4pcq"
Oct 30 19:25:45.926960 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:25:45.926940    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/362aa1c7-3163-49ee-9ed9-6a965a4a9a56-tls-certs\") pod \"auth-test-model-d4f84d-kserve-5b9745569c-h4pcq\" (UID: \"362aa1c7-3163-49ee-9ed9-6a965a4a9a56\") " pod="ai-tenant-e2e-shared-b-w1-d4f84d/auth-test-model-d4f84d-kserve-5b9745569c-h4pcq"
Oct 30 19:25:45.987942 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:25:45.987914    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fkz9n\" (UniqueName: \"kubernetes.io/projected/362aa1c7-3163-49ee-9ed9-6a965a4a9a56-kube-api-access-fkz9n\") pod \"auth-test-model-d4f84d-kserve-5b9745569c-h4pcq\" (UID: \"362aa1c7-3163-49ee-9ed9-6a965a4a9a56\") " pod="ai-tenant-e2e-shared-b-w1-d4f84d/auth-test-model-d4f84d-kserve-5b9745569c-h4pcq"
Oct 30 19:25:46.257997 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:25:46.257914    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w1-d4f84d/auth-test-model-d4f84d-kserve-5b9745569c-h4pcq"
Oct 30 19:25:46.651048 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:25:46.651020    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w1-d4f84d/auth-test-model-d4f84d-kserve-5b9745569c-h4pcq"]
Oct 30 19:25:46.651405 ip-10-0-134-46 kubenswrapper[4888]: W1030 19:25:46.651130    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod362aa1c7_3163_49ee_9ed9_6a965a4a9a56.slice/crio-dc64a3d96c8ec344959f0970d24682c56fa04c1d96760fa4cf44605078d9dda8 WatchSource:0}: Error finding container dc64a3d96c8ec344959f0970d24682c56fa04c1d96760fa4cf44605078d9dda8: Status 404 returned error can't find the container with id dc64a3d96c8ec344959f0970d24682c56fa04c1d96760fa4cf44605078d9dda8
Oct 30 19:25:47.020751 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:25:47.020669    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w1-d4f84d/auth-test-model-d4f84d-kserve-5b9745569c-h4pcq" event={"ID":"362aa1c7-3163-49ee-9ed9-6a965a4a9a56","Type":"ContainerStarted","Data":"fdb3523ce12e2652f8b3b046ec55606c9635ae42f413b35556fb7c444b5828f1"}
Oct 30 19:25:47.020751 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:25:47.020712    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w1-d4f84d/auth-test-model-d4f84d-kserve-5b9745569c-h4pcq" event={"ID":"362aa1c7-3163-49ee-9ed9-6a965a4a9a56","Type":"ContainerStarted","Data":"dc64a3d96c8ec344959f0970d24682c56fa04c1d96760fa4cf44605078d9dda8"}
Oct 30 19:25:47.146661 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:25:47.146612    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w1-d4f84d/auth-test-model-d4f84d-kserve-5b9745569c-h4pcq" podStartSLOduration=2.146598855 podStartE2EDuration="2.146598855s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:25:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:25:47.142684275 +0000 UTC m=+3111.390090313" watchObservedRunningTime="2026-10-30 19:25:47.146598855 +0000 UTC m=+3111.394004921"
Oct 30 19:25:47.259351 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:25:47.259318    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w1-d4f84d/auth-test-model-d4f84d-kserve-5b9745569c-h4pcq"
Oct 30 19:25:56.258584 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:25:56.258546    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w1-d4f84d/auth-test-model-d4f84d-kserve-5b9745569c-h4pcq"
Oct 30 19:25:56.271003 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:25:56.270977    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w1-d4f84d/auth-test-model-d4f84d-kserve-5b9745569c-h4pcq"
Oct 30 19:25:57.103145 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:25:57.103119    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w1-d4f84d/auth-test-model-d4f84d-kserve-5b9745569c-h4pcq"
Oct 30 19:26:05.621091 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:26:05.621056    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-jbsw6" containerName="extractor"
Oct 30 19:26:12.070640 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:26:12.070595    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-rjmdz"]
Oct 30 19:26:35.618155 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:26:35.618116    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-jbsw6" containerName="extractor"
Oct 30 19:26:53.376472 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:26:53.376431    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-rjmdz"]
Oct 30 19:27:05.623900 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:27:05.623817    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-jbsw6" containerName="extractor"
Oct 30 19:27:28.941374 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:28.941342    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-vdg7j"]
Oct 30 19:27:28.941863 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:28.941542    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-vdg7j" podUID="c841723f-12c0-449f-bd04-96b459feb185" containerName="istio-proxy" containerID="cri-o://64b30a0e50ed9cf9e8046c343673c0b9c9fb27c873852f76a26eabc5c8c17368" gracePeriod=30
Oct 30 19:27:29.125963 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:29.125931    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-c60f5c-7cf756c6b-sql7j"]
Oct 30 19:27:29.240395 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:29.239202    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-c60f5c-9cc67ddcf-hr6vs"]
Oct 30 19:27:31.940787 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:31.940755    4888 patch_prober.go:29] interesting pod/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-vdg7j container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 19:27:31.941186 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:31.940815    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-vdg7j" podUID="c841723f-12c0-449f-bd04-96b459feb185" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 19:27:34.127313 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:34.126560    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-autores-c60f5c-7cf756c6b-sql7j" podUID="3a30428d-1c18-4962-b665-beb8bf3fad1d" containerName="payload-processing" containerID="cri-o://a2cd073e753fdd46fa9d2b83e63240a0201c946d83cffa05abcb2ddc1d462952" gracePeriod=25
Oct 30 19:27:34.240671 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:34.240617    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-autores-c60f5c-9cc67ddcf-hr6vs" podUID="6d598741-f60f-4c9b-b937-4035ec6943a2" containerName="payload-pre-processing" containerID="cri-o://a3091d910aa05185394b6cf534c9e0142e4c81a3c341c3e9af498a4f79b1ab50" gracePeriod=25
Oct 30 19:27:34.347105 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:34.347074    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-vdg7j"
Oct 30 19:27:34.402186 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:34.402156    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/c841723f-12c0-449f-bd04-96b459feb185-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c841723f-12c0-449f-bd04-96b459feb185-istio-podinfo\") pod \"c841723f-12c0-449f-bd04-96b459feb185\" (UID: \"c841723f-12c0-449f-bd04-96b459feb185\") "
Oct 30 19:27:34.402495 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:34.402203    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c841723f-12c0-449f-bd04-96b459feb185-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c841723f-12c0-449f-bd04-96b459feb185-istio-envoy\") pod \"c841723f-12c0-449f-bd04-96b459feb185\" (UID: \"c841723f-12c0-449f-bd04-96b459feb185\") "
Oct 30 19:27:34.402495 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:34.402249    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c841723f-12c0-449f-bd04-96b459feb185-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c841723f-12c0-449f-bd04-96b459feb185-credential-socket\") pod \"c841723f-12c0-449f-bd04-96b459feb185\" (UID: \"c841723f-12c0-449f-bd04-96b459feb185\") "
Oct 30 19:27:34.402495 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:34.402270    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c841723f-12c0-449f-bd04-96b459feb185-kube-api-access-gzh4t\" (UniqueName: \"kubernetes.io/projected/c841723f-12c0-449f-bd04-96b459feb185-kube-api-access-gzh4t\") pod \"c841723f-12c0-449f-bd04-96b459feb185\" (UID: \"c841723f-12c0-449f-bd04-96b459feb185\") "
Oct 30 19:27:34.402495 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:34.402474    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c841723f-12c0-449f-bd04-96b459feb185-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c841723f-12c0-449f-bd04-96b459feb185-workload-socket\") pod \"c841723f-12c0-449f-bd04-96b459feb185\" (UID: \"c841723f-12c0-449f-bd04-96b459feb185\") "
Oct 30 19:27:34.402932 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:34.402516    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c841723f-12c0-449f-bd04-96b459feb185-istio-token\" (UniqueName: \"kubernetes.io/projected/c841723f-12c0-449f-bd04-96b459feb185-istio-token\") pod \"c841723f-12c0-449f-bd04-96b459feb185\" (UID: \"c841723f-12c0-449f-bd04-96b459feb185\") "
Oct 30 19:27:34.402932 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:34.402602    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c841723f-12c0-449f-bd04-96b459feb185-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c841723f-12c0-449f-bd04-96b459feb185-workload-certs\") pod \"c841723f-12c0-449f-bd04-96b459feb185\" (UID: \"c841723f-12c0-449f-bd04-96b459feb185\") "
Oct 30 19:27:34.402932 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:34.402641    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c841723f-12c0-449f-bd04-96b459feb185-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c841723f-12c0-449f-bd04-96b459feb185-istio-data\") pod \"c841723f-12c0-449f-bd04-96b459feb185\" (UID: \"c841723f-12c0-449f-bd04-96b459feb185\") "
Oct 30 19:27:34.402932 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:34.402753    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c841723f-12c0-449f-bd04-96b459feb185-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c841723f-12c0-449f-bd04-96b459feb185-istiod-ca-cert\") pod \"c841723f-12c0-449f-bd04-96b459feb185\" (UID: \"c841723f-12c0-449f-bd04-96b459feb185\") "
Oct 30 19:27:34.402932 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:34.402837    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c841723f-12c0-449f-bd04-96b459feb185-workload-socket" pod "c841723f-12c0-449f-bd04-96b459feb185" (UID: "c841723f-12c0-449f-bd04-96b459feb185"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:27:34.404214 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:34.403037    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c841723f-12c0-449f-bd04-96b459feb185-workload-certs" pod "c841723f-12c0-449f-bd04-96b459feb185" (UID: "c841723f-12c0-449f-bd04-96b459feb185"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:27:34.404214 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:34.403198    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c841723f-12c0-449f-bd04-96b459feb185-istio-data" pod "c841723f-12c0-449f-bd04-96b459feb185" (UID: "c841723f-12c0-449f-bd04-96b459feb185"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:27:34.404214 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:34.403370    4888 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c841723f-12c0-449f-bd04-96b459feb185-workload-certs\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:27:34.404214 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:34.403592    4888 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c841723f-12c0-449f-bd04-96b459feb185-istio-data\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:27:34.404214 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:34.403607    4888 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c841723f-12c0-449f-bd04-96b459feb185-workload-socket\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:27:34.404214 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:34.403609    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c841723f-12c0-449f-bd04-96b459feb185-istiod-ca-cert" pod "c841723f-12c0-449f-bd04-96b459feb185" (UID: "c841723f-12c0-449f-bd04-96b459feb185"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:27:34.404214 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:34.403653    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c841723f-12c0-449f-bd04-96b459feb185-credential-socket" pod "c841723f-12c0-449f-bd04-96b459feb185" (UID: "c841723f-12c0-449f-bd04-96b459feb185"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:27:34.405149 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:34.405125    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c841723f-12c0-449f-bd04-96b459feb185-istio-envoy" pod "c841723f-12c0-449f-bd04-96b459feb185" (UID: "c841723f-12c0-449f-bd04-96b459feb185"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:27:34.405274 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:34.405128    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c841723f-12c0-449f-bd04-96b459feb185-istio-podinfo" pod "c841723f-12c0-449f-bd04-96b459feb185" (UID: "c841723f-12c0-449f-bd04-96b459feb185"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:27:34.405502 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:34.405476    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c841723f-12c0-449f-bd04-96b459feb185-istio-token" pod "c841723f-12c0-449f-bd04-96b459feb185" (UID: "c841723f-12c0-449f-bd04-96b459feb185"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:27:34.406481 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:34.406463    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c841723f-12c0-449f-bd04-96b459feb185-kube-api-access-gzh4t" pod "c841723f-12c0-449f-bd04-96b459feb185" (UID: "c841723f-12c0-449f-bd04-96b459feb185"). InnerVolumeSpecName "kube-api-access-gzh4t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:27:34.417254 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:34.417228    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-c60f5c-7cf756c6b-sql7j"
Oct 30 19:27:34.504113 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:34.504086    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3a30428d-1c18-4962-b665-beb8bf3fad1d-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3a30428d-1c18-4962-b665-beb8bf3fad1d-plugins-config-volume\") pod \"3a30428d-1c18-4962-b665-beb8bf3fad1d\" (UID: \"3a30428d-1c18-4962-b665-beb8bf3fad1d\") "
Oct 30 19:27:34.504226 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:34.504178    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3a30428d-1c18-4962-b665-beb8bf3fad1d-kube-api-access-4ktlx\" (UniqueName: \"kubernetes.io/projected/3a30428d-1c18-4962-b665-beb8bf3fad1d-kube-api-access-4ktlx\") pod \"3a30428d-1c18-4962-b665-beb8bf3fad1d\" (UID: \"3a30428d-1c18-4962-b665-beb8bf3fad1d\") "
Oct 30 19:27:34.504538 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:34.504513    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3a30428d-1c18-4962-b665-beb8bf3fad1d-plugins-config-volume" pod "3a30428d-1c18-4962-b665-beb8bf3fad1d" (UID: "3a30428d-1c18-4962-b665-beb8bf3fad1d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:27:34.504830 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:34.504739    4888 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c841723f-12c0-449f-bd04-96b459feb185-istiod-ca-cert\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:27:34.504830 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:34.504766    4888 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c841723f-12c0-449f-bd04-96b459feb185-istio-podinfo\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:27:34.504830 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:34.504779    4888 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c841723f-12c0-449f-bd04-96b459feb185-istio-envoy\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:27:34.504830 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:34.504788    4888 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3a30428d-1c18-4962-b665-beb8bf3fad1d-plugins-config-volume\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:27:34.504830 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:34.504796    4888 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c841723f-12c0-449f-bd04-96b459feb185-credential-socket\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:27:34.504830 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:34.504804    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gzh4t\" (UniqueName: \"kubernetes.io/projected/c841723f-12c0-449f-bd04-96b459feb185-kube-api-access-gzh4t\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:27:34.504830 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:34.504813    4888 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c841723f-12c0-449f-bd04-96b459feb185-istio-token\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:27:34.506592 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:34.506569    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3a30428d-1c18-4962-b665-beb8bf3fad1d-kube-api-access-4ktlx" pod "3a30428d-1c18-4962-b665-beb8bf3fad1d" (UID: "3a30428d-1c18-4962-b665-beb8bf3fad1d"). InnerVolumeSpecName "kube-api-access-4ktlx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:27:34.512530 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:34.512511    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-c60f5c-9cc67ddcf-hr6vs"
Oct 30 19:27:34.605691 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:34.605661    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6d598741-f60f-4c9b-b937-4035ec6943a2-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6d598741-f60f-4c9b-b937-4035ec6943a2-plugins-config-volume\") pod \"6d598741-f60f-4c9b-b937-4035ec6943a2\" (UID: \"6d598741-f60f-4c9b-b937-4035ec6943a2\") "
Oct 30 19:27:34.605856 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:34.605740    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6d598741-f60f-4c9b-b937-4035ec6943a2-kube-api-access-dnjhx\" (UniqueName: \"kubernetes.io/projected/6d598741-f60f-4c9b-b937-4035ec6943a2-kube-api-access-dnjhx\") pod \"6d598741-f60f-4c9b-b937-4035ec6943a2\" (UID: \"6d598741-f60f-4c9b-b937-4035ec6943a2\") "
Oct 30 19:27:34.606029 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:34.606006    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6d598741-f60f-4c9b-b937-4035ec6943a2-plugins-config-volume" pod "6d598741-f60f-4c9b-b937-4035ec6943a2" (UID: "6d598741-f60f-4c9b-b937-4035ec6943a2"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:27:34.606203 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:34.606187    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4ktlx\" (UniqueName: \"kubernetes.io/projected/3a30428d-1c18-4962-b665-beb8bf3fad1d-kube-api-access-4ktlx\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:27:34.606254 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:34.606206    4888 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6d598741-f60f-4c9b-b937-4035ec6943a2-plugins-config-volume\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:27:34.607994 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:34.607976    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6d598741-f60f-4c9b-b937-4035ec6943a2-kube-api-access-dnjhx" pod "6d598741-f60f-4c9b-b937-4035ec6943a2" (UID: "6d598741-f60f-4c9b-b937-4035ec6943a2"). InnerVolumeSpecName "kube-api-access-dnjhx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:27:34.707875 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:34.707847    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dnjhx\" (UniqueName: \"kubernetes.io/projected/6d598741-f60f-4c9b-b937-4035ec6943a2-kube-api-access-dnjhx\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:27:34.824076 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:34.824047    4888 generic.go:356] "Generic (PLEG): container finished" podID="c841723f-12c0-449f-bd04-96b459feb185" containerID="64b30a0e50ed9cf9e8046c343673c0b9c9fb27c873852f76a26eabc5c8c17368" exitCode=0
Oct 30 19:27:34.824315 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:34.824097    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-vdg7j"
Oct 30 19:27:34.824315 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:34.824130    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-vdg7j" event={"ID":"c841723f-12c0-449f-bd04-96b459feb185","Type":"ContainerDied","Data":"64b30a0e50ed9cf9e8046c343673c0b9c9fb27c873852f76a26eabc5c8c17368"}
Oct 30 19:27:34.824315 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:34.824194    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-vdg7j" event={"ID":"c841723f-12c0-449f-bd04-96b459feb185","Type":"ContainerDied","Data":"f195e8146d4e0a2c8ee44f7043d4eafec38506546836881826e6eb45cbed7c53"}
Oct 30 19:27:34.824315 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:34.824224    4888 scope.go:122] "RemoveContainer" containerID="64b30a0e50ed9cf9e8046c343673c0b9c9fb27c873852f76a26eabc5c8c17368"
Oct 30 19:27:34.825697 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:34.825670    4888 generic.go:356] "Generic (PLEG): container finished" podID="3a30428d-1c18-4962-b665-beb8bf3fad1d" containerID="a2cd073e753fdd46fa9d2b83e63240a0201c946d83cffa05abcb2ddc1d462952" exitCode=0
Oct 30 19:27:34.825807 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:34.825721    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-c60f5c-7cf756c6b-sql7j"
Oct 30 19:27:34.825807 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:34.825747    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-c60f5c-7cf756c6b-sql7j" event={"ID":"3a30428d-1c18-4962-b665-beb8bf3fad1d","Type":"ContainerDied","Data":"a2cd073e753fdd46fa9d2b83e63240a0201c946d83cffa05abcb2ddc1d462952"}
Oct 30 19:27:34.825807 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:34.825794    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-c60f5c-7cf756c6b-sql7j" event={"ID":"3a30428d-1c18-4962-b665-beb8bf3fad1d","Type":"ContainerDied","Data":"3156d8a672a8ca397907286feb95c17db8956ef086409509981702f017fb8643"}
Oct 30 19:27:34.827638 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:34.827616    4888 generic.go:356] "Generic (PLEG): container finished" podID="6d598741-f60f-4c9b-b937-4035ec6943a2" containerID="a3091d910aa05185394b6cf534c9e0142e4c81a3c341c3e9af498a4f79b1ab50" exitCode=0
Oct 30 19:27:34.827757 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:34.827674    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-c60f5c-9cc67ddcf-hr6vs"
Oct 30 19:27:34.827757 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:34.827702    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-c60f5c-9cc67ddcf-hr6vs" event={"ID":"6d598741-f60f-4c9b-b937-4035ec6943a2","Type":"ContainerDied","Data":"a3091d910aa05185394b6cf534c9e0142e4c81a3c341c3e9af498a4f79b1ab50"}
Oct 30 19:27:34.827757 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:34.827743    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-c60f5c-9cc67ddcf-hr6vs" event={"ID":"6d598741-f60f-4c9b-b937-4035ec6943a2","Type":"ContainerDied","Data":"0d9dac4bd5dc7a550f3c1592302c5a3327f8550421c1d253b8ed8e2729cc7afb"}
Oct 30 19:27:34.835973 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:34.835955    4888 scope.go:122] "RemoveContainer" containerID="64b30a0e50ed9cf9e8046c343673c0b9c9fb27c873852f76a26eabc5c8c17368"
Oct 30 19:27:34.836330 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:27:34.836301    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"64b30a0e50ed9cf9e8046c343673c0b9c9fb27c873852f76a26eabc5c8c17368\": container with ID starting with 64b30a0e50ed9cf9e8046c343673c0b9c9fb27c873852f76a26eabc5c8c17368 not found: ID does not exist" containerID="64b30a0e50ed9cf9e8046c343673c0b9c9fb27c873852f76a26eabc5c8c17368"
Oct 30 19:27:34.836491 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:34.836339    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"64b30a0e50ed9cf9e8046c343673c0b9c9fb27c873852f76a26eabc5c8c17368"} err="failed to get container status \"64b30a0e50ed9cf9e8046c343673c0b9c9fb27c873852f76a26eabc5c8c17368\": rpc error: code = NotFound desc = could not find container \"64b30a0e50ed9cf9e8046c343673c0b9c9fb27c873852f76a26eabc5c8c17368\": container with ID starting with 64b30a0e50ed9cf9e8046c343673c0b9c9fb27c873852f76a26eabc5c8c17368 not found: ID does not exist"
Oct 30 19:27:34.836491 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:34.836389    4888 scope.go:122] "RemoveContainer" containerID="a2cd073e753fdd46fa9d2b83e63240a0201c946d83cffa05abcb2ddc1d462952"
Oct 30 19:27:34.848574 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:34.848556    4888 scope.go:122] "RemoveContainer" containerID="a2cd073e753fdd46fa9d2b83e63240a0201c946d83cffa05abcb2ddc1d462952"
Oct 30 19:27:34.848931 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:27:34.848912    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a2cd073e753fdd46fa9d2b83e63240a0201c946d83cffa05abcb2ddc1d462952\": container with ID starting with a2cd073e753fdd46fa9d2b83e63240a0201c946d83cffa05abcb2ddc1d462952 not found: ID does not exist" containerID="a2cd073e753fdd46fa9d2b83e63240a0201c946d83cffa05abcb2ddc1d462952"
Oct 30 19:27:34.849041 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:34.848937    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a2cd073e753fdd46fa9d2b83e63240a0201c946d83cffa05abcb2ddc1d462952"} err="failed to get container status \"a2cd073e753fdd46fa9d2b83e63240a0201c946d83cffa05abcb2ddc1d462952\": rpc error: code = NotFound desc = could not find container \"a2cd073e753fdd46fa9d2b83e63240a0201c946d83cffa05abcb2ddc1d462952\": container with ID starting with a2cd073e753fdd46fa9d2b83e63240a0201c946d83cffa05abcb2ddc1d462952 not found: ID does not exist"
Oct 30 19:27:34.849041 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:34.848952    4888 scope.go:122] "RemoveContainer" containerID="a3091d910aa05185394b6cf534c9e0142e4c81a3c341c3e9af498a4f79b1ab50"
Oct 30 19:27:34.860307 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:34.860271    4888 scope.go:122] "RemoveContainer" containerID="a3091d910aa05185394b6cf534c9e0142e4c81a3c341c3e9af498a4f79b1ab50"
Oct 30 19:27:34.860724 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:27:34.860695    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a3091d910aa05185394b6cf534c9e0142e4c81a3c341c3e9af498a4f79b1ab50\": container with ID starting with a3091d910aa05185394b6cf534c9e0142e4c81a3c341c3e9af498a4f79b1ab50 not found: ID does not exist" containerID="a3091d910aa05185394b6cf534c9e0142e4c81a3c341c3e9af498a4f79b1ab50"
Oct 30 19:27:34.860823 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:34.860736    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a3091d910aa05185394b6cf534c9e0142e4c81a3c341c3e9af498a4f79b1ab50"} err="failed to get container status \"a3091d910aa05185394b6cf534c9e0142e4c81a3c341c3e9af498a4f79b1ab50\": rpc error: code = NotFound desc = could not find container \"a3091d910aa05185394b6cf534c9e0142e4c81a3c341c3e9af498a4f79b1ab50\": container with ID starting with a3091d910aa05185394b6cf534c9e0142e4c81a3c341c3e9af498a4f79b1ab50 not found: ID does not exist"
Oct 30 19:27:34.996466 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:34.996435    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-c60f5c-9cc67ddcf-hr6vs"]
Oct 30 19:27:35.003523 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:35.003496    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-c60f5c-9cc67ddcf-hr6vs"]
Oct 30 19:27:35.119136 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:35.119040    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-vdg7j"]
Oct 30 19:27:35.124305 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:35.124263    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-vdg7j"]
Oct 30 19:27:35.238265 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:35.238233    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-c60f5c-7cf756c6b-sql7j"]
Oct 30 19:27:35.242526 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:35.242503    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-c60f5c-7cf756c6b-sql7j"]
Oct 30 19:27:35.624716 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:27:35.624677    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-jbsw6" containerName="extractor"
Oct 30 19:27:35.890998 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:35.890916    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3a30428d-1c18-4962-b665-beb8bf3fad1d" path="/var/lib/kubelet/pods/3a30428d-1c18-4962-b665-beb8bf3fad1d/volumes"
Oct 30 19:27:35.891357 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:35.891342    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6d598741-f60f-4c9b-b937-4035ec6943a2" path="/var/lib/kubelet/pods/6d598741-f60f-4c9b-b937-4035ec6943a2/volumes"
Oct 30 19:27:35.891770 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:35.891757    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c841723f-12c0-449f-bd04-96b459feb185" path="/var/lib/kubelet/pods/c841723f-12c0-449f-bd04-96b459feb185/volumes"
Oct 30 19:27:41.860375 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:41.860346    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-autores-c60f5c-openshift-default-9c4d97949-vk946"]
Oct 30 19:27:41.860777 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:41.860581    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-autores-c60f5c-openshift-default-9c4d97949-vk946" podUID="2b71ddaa-a90a-4764-a703-91f6f81db363" containerName="istio-proxy" containerID="cri-o://783cdf54ed93425253fcd9b1bddfba051f2d0510a5f7456338e0f5e6004b631d" gracePeriod=30
Oct 30 19:27:43.179362 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:43.179322    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-explicit-200c46-openshift-default-5f4d5cd597-t47sj"]
Oct 30 19:27:43.180147 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:43.180124    4888 cpu_manager.go:414] "containerMap: removing container" podUID="c841723f-12c0-449f-bd04-96b459feb185" containerName="istio-proxy"
Oct 30 19:27:43.180147 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:43.180146    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c841723f-12c0-449f-bd04-96b459feb185" containerName="istio-proxy"
Oct 30 19:27:43.180371 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:43.180164    4888 cpu_manager.go:414] "containerMap: removing container" podUID="3a30428d-1c18-4962-b665-beb8bf3fad1d" containerName="payload-processing"
Oct 30 19:27:43.180371 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:43.180172    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3a30428d-1c18-4962-b665-beb8bf3fad1d" containerName="payload-processing"
Oct 30 19:27:43.180371 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:43.180199    4888 cpu_manager.go:414] "containerMap: removing container" podUID="6d598741-f60f-4c9b-b937-4035ec6943a2" containerName="payload-pre-processing"
Oct 30 19:27:43.180371 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:43.180207    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6d598741-f60f-4c9b-b937-4035ec6943a2" containerName="payload-pre-processing"
Oct 30 19:27:43.180582 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:43.180422    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="c841723f-12c0-449f-bd04-96b459feb185" containerName="istio-proxy"
Oct 30 19:27:43.180582 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:43.180437    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="3a30428d-1c18-4962-b665-beb8bf3fad1d" containerName="payload-processing"
Oct 30 19:27:43.180582 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:43.180452    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="6d598741-f60f-4c9b-b937-4035ec6943a2" containerName="payload-pre-processing"
Oct 30 19:27:43.185561 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:43.185538    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-200c46-openshift-default-5f4d5cd597-t47sj"
Oct 30 19:27:43.253308 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:43.248973    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-explicit-200c46-openshift-default-dockercfg-2r66w\""
Oct 30 19:27:43.299073 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:43.299037    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-explicit-200c46-openshift-default-5f4d5cd597-t47sj"]
Oct 30 19:27:43.318960 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:43.318925    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f7045fa9-0689-4317-95fc-d3bf87806ce1-istio-data\") pod \"e2e-explicit-200c46-openshift-default-5f4d5cd597-t47sj\" (UID: \"f7045fa9-0689-4317-95fc-d3bf87806ce1\") " pod="openshift-ingress/e2e-explicit-200c46-openshift-default-5f4d5cd597-t47sj"
Oct 30 19:27:43.318960 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:43.318965    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f7045fa9-0689-4317-95fc-d3bf87806ce1-workload-certs\") pod \"e2e-explicit-200c46-openshift-default-5f4d5cd597-t47sj\" (UID: \"f7045fa9-0689-4317-95fc-d3bf87806ce1\") " pod="openshift-ingress/e2e-explicit-200c46-openshift-default-5f4d5cd597-t47sj"
Oct 30 19:27:43.319226 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:43.318992    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f7045fa9-0689-4317-95fc-d3bf87806ce1-istiod-ca-cert\") pod \"e2e-explicit-200c46-openshift-default-5f4d5cd597-t47sj\" (UID: \"f7045fa9-0689-4317-95fc-d3bf87806ce1\") " pod="openshift-ingress/e2e-explicit-200c46-openshift-default-5f4d5cd597-t47sj"
Oct 30 19:27:43.319226 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:43.319050    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f7045fa9-0689-4317-95fc-d3bf87806ce1-istio-token\") pod \"e2e-explicit-200c46-openshift-default-5f4d5cd597-t47sj\" (UID: \"f7045fa9-0689-4317-95fc-d3bf87806ce1\") " pod="openshift-ingress/e2e-explicit-200c46-openshift-default-5f4d5cd597-t47sj"
Oct 30 19:27:43.319226 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:43.319096    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hgxhd\" (UniqueName: \"kubernetes.io/projected/f7045fa9-0689-4317-95fc-d3bf87806ce1-kube-api-access-hgxhd\") pod \"e2e-explicit-200c46-openshift-default-5f4d5cd597-t47sj\" (UID: \"f7045fa9-0689-4317-95fc-d3bf87806ce1\") " pod="openshift-ingress/e2e-explicit-200c46-openshift-default-5f4d5cd597-t47sj"
Oct 30 19:27:43.319226 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:43.319149    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f7045fa9-0689-4317-95fc-d3bf87806ce1-istio-podinfo\") pod \"e2e-explicit-200c46-openshift-default-5f4d5cd597-t47sj\" (UID: \"f7045fa9-0689-4317-95fc-d3bf87806ce1\") " pod="openshift-ingress/e2e-explicit-200c46-openshift-default-5f4d5cd597-t47sj"
Oct 30 19:27:43.319531 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:43.319298    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7045fa9-0689-4317-95fc-d3bf87806ce1-credential-socket\") pod \"e2e-explicit-200c46-openshift-default-5f4d5cd597-t47sj\" (UID: \"f7045fa9-0689-4317-95fc-d3bf87806ce1\") " pod="openshift-ingress/e2e-explicit-200c46-openshift-default-5f4d5cd597-t47sj"
Oct 30 19:27:43.319531 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:43.319335    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7045fa9-0689-4317-95fc-d3bf87806ce1-workload-socket\") pod \"e2e-explicit-200c46-openshift-default-5f4d5cd597-t47sj\" (UID: \"f7045fa9-0689-4317-95fc-d3bf87806ce1\") " pod="openshift-ingress/e2e-explicit-200c46-openshift-default-5f4d5cd597-t47sj"
Oct 30 19:27:43.319531 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:43.319396    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f7045fa9-0689-4317-95fc-d3bf87806ce1-istio-envoy\") pod \"e2e-explicit-200c46-openshift-default-5f4d5cd597-t47sj\" (UID: \"f7045fa9-0689-4317-95fc-d3bf87806ce1\") " pod="openshift-ingress/e2e-explicit-200c46-openshift-default-5f4d5cd597-t47sj"
Oct 30 19:27:43.420147 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:43.420107    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f7045fa9-0689-4317-95fc-d3bf87806ce1-istio-podinfo\") pod \"e2e-explicit-200c46-openshift-default-5f4d5cd597-t47sj\" (UID: \"f7045fa9-0689-4317-95fc-d3bf87806ce1\") " pod="openshift-ingress/e2e-explicit-200c46-openshift-default-5f4d5cd597-t47sj"
Oct 30 19:27:43.420381 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:43.420213    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7045fa9-0689-4317-95fc-d3bf87806ce1-credential-socket\") pod \"e2e-explicit-200c46-openshift-default-5f4d5cd597-t47sj\" (UID: \"f7045fa9-0689-4317-95fc-d3bf87806ce1\") " pod="openshift-ingress/e2e-explicit-200c46-openshift-default-5f4d5cd597-t47sj"
Oct 30 19:27:43.420381 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:43.420241    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7045fa9-0689-4317-95fc-d3bf87806ce1-workload-socket\") pod \"e2e-explicit-200c46-openshift-default-5f4d5cd597-t47sj\" (UID: \"f7045fa9-0689-4317-95fc-d3bf87806ce1\") " pod="openshift-ingress/e2e-explicit-200c46-openshift-default-5f4d5cd597-t47sj"
Oct 30 19:27:43.420381 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:43.420317    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f7045fa9-0689-4317-95fc-d3bf87806ce1-istio-envoy\") pod \"e2e-explicit-200c46-openshift-default-5f4d5cd597-t47sj\" (UID: \"f7045fa9-0689-4317-95fc-d3bf87806ce1\") " pod="openshift-ingress/e2e-explicit-200c46-openshift-default-5f4d5cd597-t47sj"
Oct 30 19:27:43.420596 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:43.420402    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f7045fa9-0689-4317-95fc-d3bf87806ce1-istio-data\") pod \"e2e-explicit-200c46-openshift-default-5f4d5cd597-t47sj\" (UID: \"f7045fa9-0689-4317-95fc-d3bf87806ce1\") " pod="openshift-ingress/e2e-explicit-200c46-openshift-default-5f4d5cd597-t47sj"
Oct 30 19:27:43.420596 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:43.420530    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f7045fa9-0689-4317-95fc-d3bf87806ce1-workload-certs\") pod \"e2e-explicit-200c46-openshift-default-5f4d5cd597-t47sj\" (UID: \"f7045fa9-0689-4317-95fc-d3bf87806ce1\") " pod="openshift-ingress/e2e-explicit-200c46-openshift-default-5f4d5cd597-t47sj"
Oct 30 19:27:43.420596 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:43.420571    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f7045fa9-0689-4317-95fc-d3bf87806ce1-istiod-ca-cert\") pod \"e2e-explicit-200c46-openshift-default-5f4d5cd597-t47sj\" (UID: \"f7045fa9-0689-4317-95fc-d3bf87806ce1\") " pod="openshift-ingress/e2e-explicit-200c46-openshift-default-5f4d5cd597-t47sj"
Oct 30 19:27:43.420760 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:43.420646    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f7045fa9-0689-4317-95fc-d3bf87806ce1-istio-token\") pod \"e2e-explicit-200c46-openshift-default-5f4d5cd597-t47sj\" (UID: \"f7045fa9-0689-4317-95fc-d3bf87806ce1\") " pod="openshift-ingress/e2e-explicit-200c46-openshift-default-5f4d5cd597-t47sj"
Oct 30 19:27:43.420760 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:43.420697    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hgxhd\" (UniqueName: \"kubernetes.io/projected/f7045fa9-0689-4317-95fc-d3bf87806ce1-kube-api-access-hgxhd\") pod \"e2e-explicit-200c46-openshift-default-5f4d5cd597-t47sj\" (UID: \"f7045fa9-0689-4317-95fc-d3bf87806ce1\") " pod="openshift-ingress/e2e-explicit-200c46-openshift-default-5f4d5cd597-t47sj"
Oct 30 19:27:43.420867 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:43.420810    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7045fa9-0689-4317-95fc-d3bf87806ce1-workload-socket\") pod \"e2e-explicit-200c46-openshift-default-5f4d5cd597-t47sj\" (UID: \"f7045fa9-0689-4317-95fc-d3bf87806ce1\") " pod="openshift-ingress/e2e-explicit-200c46-openshift-default-5f4d5cd597-t47sj"
Oct 30 19:27:43.420968 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:43.420924    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7045fa9-0689-4317-95fc-d3bf87806ce1-credential-socket\") pod \"e2e-explicit-200c46-openshift-default-5f4d5cd597-t47sj\" (UID: \"f7045fa9-0689-4317-95fc-d3bf87806ce1\") " pod="openshift-ingress/e2e-explicit-200c46-openshift-default-5f4d5cd597-t47sj"
Oct 30 19:27:43.421116 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:43.420997    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f7045fa9-0689-4317-95fc-d3bf87806ce1-workload-certs\") pod \"e2e-explicit-200c46-openshift-default-5f4d5cd597-t47sj\" (UID: \"f7045fa9-0689-4317-95fc-d3bf87806ce1\") " pod="openshift-ingress/e2e-explicit-200c46-openshift-default-5f4d5cd597-t47sj"
Oct 30 19:27:43.421116 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:43.421067    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f7045fa9-0689-4317-95fc-d3bf87806ce1-istio-data\") pod \"e2e-explicit-200c46-openshift-default-5f4d5cd597-t47sj\" (UID: \"f7045fa9-0689-4317-95fc-d3bf87806ce1\") " pod="openshift-ingress/e2e-explicit-200c46-openshift-default-5f4d5cd597-t47sj"
Oct 30 19:27:43.421667 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:43.421638    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f7045fa9-0689-4317-95fc-d3bf87806ce1-istiod-ca-cert\") pod \"e2e-explicit-200c46-openshift-default-5f4d5cd597-t47sj\" (UID: \"f7045fa9-0689-4317-95fc-d3bf87806ce1\") " pod="openshift-ingress/e2e-explicit-200c46-openshift-default-5f4d5cd597-t47sj"
Oct 30 19:27:43.422889 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:43.422869    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f7045fa9-0689-4317-95fc-d3bf87806ce1-istio-envoy\") pod \"e2e-explicit-200c46-openshift-default-5f4d5cd597-t47sj\" (UID: \"f7045fa9-0689-4317-95fc-d3bf87806ce1\") " pod="openshift-ingress/e2e-explicit-200c46-openshift-default-5f4d5cd597-t47sj"
Oct 30 19:27:43.423028 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:43.423000    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f7045fa9-0689-4317-95fc-d3bf87806ce1-istio-podinfo\") pod \"e2e-explicit-200c46-openshift-default-5f4d5cd597-t47sj\" (UID: \"f7045fa9-0689-4317-95fc-d3bf87806ce1\") " pod="openshift-ingress/e2e-explicit-200c46-openshift-default-5f4d5cd597-t47sj"
Oct 30 19:27:43.495583 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:43.495495    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hgxhd\" (UniqueName: \"kubernetes.io/projected/f7045fa9-0689-4317-95fc-d3bf87806ce1-kube-api-access-hgxhd\") pod \"e2e-explicit-200c46-openshift-default-5f4d5cd597-t47sj\" (UID: \"f7045fa9-0689-4317-95fc-d3bf87806ce1\") " pod="openshift-ingress/e2e-explicit-200c46-openshift-default-5f4d5cd597-t47sj"
Oct 30 19:27:43.499502 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:43.499464    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f7045fa9-0689-4317-95fc-d3bf87806ce1-istio-token\") pod \"e2e-explicit-200c46-openshift-default-5f4d5cd597-t47sj\" (UID: \"f7045fa9-0689-4317-95fc-d3bf87806ce1\") " pod="openshift-ingress/e2e-explicit-200c46-openshift-default-5f4d5cd597-t47sj"
Oct 30 19:27:43.535253 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:43.535220    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-200c46-openshift-default-5f4d5cd597-t47sj"
Oct 30 19:27:43.976249 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:43.976220    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-explicit-200c46-openshift-default-5f4d5cd597-t47sj"]
Oct 30 19:27:43.976660 ip-10-0-134-46 kubenswrapper[4888]: W1030 19:27:43.976629    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7045fa9_0689_4317_95fc_d3bf87806ce1.slice/crio-55a65a0e2c5db80a34ab8cc5ef7e461cdda4d33bd0e562f79fb0580dfb20ac6d WatchSource:0}: Error finding container 55a65a0e2c5db80a34ab8cc5ef7e461cdda4d33bd0e562f79fb0580dfb20ac6d: Status 404 returned error can't find the container with id 55a65a0e2c5db80a34ab8cc5ef7e461cdda4d33bd0e562f79fb0580dfb20ac6d
Oct 30 19:27:43.979543 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:43.979511    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:27:43.979683 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:43.979576    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:27:43.979683 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:43.979605    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:27:44.906405 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:44.906367    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-200c46-openshift-default-5f4d5cd597-t47sj" event={"ID":"f7045fa9-0689-4317-95fc-d3bf87806ce1","Type":"ContainerStarted","Data":"011d00ec587977bd8622e89c6434b2e80ccb712763ece18f9533bf46fed0c95e"}
Oct 30 19:27:44.906841 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:44.906436    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-200c46-openshift-default-5f4d5cd597-t47sj" event={"ID":"f7045fa9-0689-4317-95fc-d3bf87806ce1","Type":"ContainerStarted","Data":"55a65a0e2c5db80a34ab8cc5ef7e461cdda4d33bd0e562f79fb0580dfb20ac6d"}
Oct 30 19:27:45.081980 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:45.081922    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-explicit-200c46-openshift-default-5f4d5cd597-t47sj" podStartSLOduration=2.081906787 podStartE2EDuration="2.081906787s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:27:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:27:45.080698112 +0000 UTC m=+3229.328104157" watchObservedRunningTime="2026-10-30 19:27:45.081906787 +0000 UTC m=+3229.329312811"
Oct 30 19:27:45.536451 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:45.536415    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-explicit-200c46-openshift-default-5f4d5cd597-t47sj"
Oct 30 19:27:45.543515 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:45.543493    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-explicit-200c46-openshift-default-5f4d5cd597-t47sj"
Oct 30 19:27:45.913544 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:45.913512    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-explicit-200c46-openshift-default-5f4d5cd597-t47sj"
Oct 30 19:27:45.914395 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:45.914378    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-explicit-200c46-openshift-default-5f4d5cd597-t47sj"
Oct 30 19:27:47.142504 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:47.142480    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-c60f5c-openshift-default-9c4d97949-vk946"
Oct 30 19:27:47.279964 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:47.279939    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2b71ddaa-a90a-4764-a703-91f6f81db363-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2b71ddaa-a90a-4764-a703-91f6f81db363-istio-data\") pod \"2b71ddaa-a90a-4764-a703-91f6f81db363\" (UID: \"2b71ddaa-a90a-4764-a703-91f6f81db363\") "
Oct 30 19:27:47.280122 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:47.279996    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2b71ddaa-a90a-4764-a703-91f6f81db363-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2b71ddaa-a90a-4764-a703-91f6f81db363-workload-socket\") pod \"2b71ddaa-a90a-4764-a703-91f6f81db363\" (UID: \"2b71ddaa-a90a-4764-a703-91f6f81db363\") "
Oct 30 19:27:47.280122 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:47.280045    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2b71ddaa-a90a-4764-a703-91f6f81db363-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2b71ddaa-a90a-4764-a703-91f6f81db363-workload-certs\") pod \"2b71ddaa-a90a-4764-a703-91f6f81db363\" (UID: \"2b71ddaa-a90a-4764-a703-91f6f81db363\") "
Oct 30 19:27:47.280122 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:47.280066    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2b71ddaa-a90a-4764-a703-91f6f81db363-kube-api-access-bqkw4\" (UniqueName: \"kubernetes.io/projected/2b71ddaa-a90a-4764-a703-91f6f81db363-kube-api-access-bqkw4\") pod \"2b71ddaa-a90a-4764-a703-91f6f81db363\" (UID: \"2b71ddaa-a90a-4764-a703-91f6f81db363\") "
Oct 30 19:27:47.280122 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:47.280089    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2b71ddaa-a90a-4764-a703-91f6f81db363-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2b71ddaa-a90a-4764-a703-91f6f81db363-istiod-ca-cert\") pod \"2b71ddaa-a90a-4764-a703-91f6f81db363\" (UID: \"2b71ddaa-a90a-4764-a703-91f6f81db363\") "
Oct 30 19:27:47.280122 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:47.280113    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2b71ddaa-a90a-4764-a703-91f6f81db363-istio-token\" (UniqueName: \"kubernetes.io/projected/2b71ddaa-a90a-4764-a703-91f6f81db363-istio-token\") pod \"2b71ddaa-a90a-4764-a703-91f6f81db363\" (UID: \"2b71ddaa-a90a-4764-a703-91f6f81db363\") "
Oct 30 19:27:47.280414 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:47.280174    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/2b71ddaa-a90a-4764-a703-91f6f81db363-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2b71ddaa-a90a-4764-a703-91f6f81db363-istio-podinfo\") pod \"2b71ddaa-a90a-4764-a703-91f6f81db363\" (UID: \"2b71ddaa-a90a-4764-a703-91f6f81db363\") "
Oct 30 19:27:47.280414 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:47.280227    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2b71ddaa-a90a-4764-a703-91f6f81db363-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2b71ddaa-a90a-4764-a703-91f6f81db363-istio-envoy\") pod \"2b71ddaa-a90a-4764-a703-91f6f81db363\" (UID: \"2b71ddaa-a90a-4764-a703-91f6f81db363\") "
Oct 30 19:27:47.280414 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:47.280262    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2b71ddaa-a90a-4764-a703-91f6f81db363-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2b71ddaa-a90a-4764-a703-91f6f81db363-credential-socket\") pod \"2b71ddaa-a90a-4764-a703-91f6f81db363\" (UID: \"2b71ddaa-a90a-4764-a703-91f6f81db363\") "
Oct 30 19:27:47.280414 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:47.280258    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2b71ddaa-a90a-4764-a703-91f6f81db363-istio-data" pod "2b71ddaa-a90a-4764-a703-91f6f81db363" (UID: "2b71ddaa-a90a-4764-a703-91f6f81db363"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:27:47.280414 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:47.280335    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2b71ddaa-a90a-4764-a703-91f6f81db363-workload-socket" pod "2b71ddaa-a90a-4764-a703-91f6f81db363" (UID: "2b71ddaa-a90a-4764-a703-91f6f81db363"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:27:47.280414 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:47.280398    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2b71ddaa-a90a-4764-a703-91f6f81db363-workload-certs" pod "2b71ddaa-a90a-4764-a703-91f6f81db363" (UID: "2b71ddaa-a90a-4764-a703-91f6f81db363"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:27:47.280716 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:47.280610    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2b71ddaa-a90a-4764-a703-91f6f81db363-credential-socket" pod "2b71ddaa-a90a-4764-a703-91f6f81db363" (UID: "2b71ddaa-a90a-4764-a703-91f6f81db363"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:27:47.280813 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:47.280791    4888 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2b71ddaa-a90a-4764-a703-91f6f81db363-credential-socket\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:27:47.280880 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:47.280820    4888 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2b71ddaa-a90a-4764-a703-91f6f81db363-istio-data\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:27:47.280880 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:47.280836    4888 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2b71ddaa-a90a-4764-a703-91f6f81db363-workload-socket\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:27:47.280880 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:47.280847    4888 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2b71ddaa-a90a-4764-a703-91f6f81db363-workload-certs\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:27:47.281048 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:47.280912    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2b71ddaa-a90a-4764-a703-91f6f81db363-istiod-ca-cert" pod "2b71ddaa-a90a-4764-a703-91f6f81db363" (UID: "2b71ddaa-a90a-4764-a703-91f6f81db363"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:27:47.282720 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:47.282690    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2b71ddaa-a90a-4764-a703-91f6f81db363-istio-token" pod "2b71ddaa-a90a-4764-a703-91f6f81db363" (UID: "2b71ddaa-a90a-4764-a703-91f6f81db363"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:27:47.282841 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:47.282718    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2b71ddaa-a90a-4764-a703-91f6f81db363-kube-api-access-bqkw4" pod "2b71ddaa-a90a-4764-a703-91f6f81db363" (UID: "2b71ddaa-a90a-4764-a703-91f6f81db363"). InnerVolumeSpecName "kube-api-access-bqkw4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:27:47.282957 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:47.282939    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2b71ddaa-a90a-4764-a703-91f6f81db363-istio-podinfo" pod "2b71ddaa-a90a-4764-a703-91f6f81db363" (UID: "2b71ddaa-a90a-4764-a703-91f6f81db363"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:27:47.283023 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:47.282963    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2b71ddaa-a90a-4764-a703-91f6f81db363-istio-envoy" pod "2b71ddaa-a90a-4764-a703-91f6f81db363" (UID: "2b71ddaa-a90a-4764-a703-91f6f81db363"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:27:47.382165 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:47.382089    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bqkw4\" (UniqueName: \"kubernetes.io/projected/2b71ddaa-a90a-4764-a703-91f6f81db363-kube-api-access-bqkw4\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:27:47.382165 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:47.382121    4888 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2b71ddaa-a90a-4764-a703-91f6f81db363-istiod-ca-cert\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:27:47.382165 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:47.382134    4888 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2b71ddaa-a90a-4764-a703-91f6f81db363-istio-token\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:27:47.382165 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:47.382145    4888 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2b71ddaa-a90a-4764-a703-91f6f81db363-istio-podinfo\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:27:47.382165 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:47.382159    4888 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2b71ddaa-a90a-4764-a703-91f6f81db363-istio-envoy\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:27:47.929163 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:47.929126    4888 generic.go:356] "Generic (PLEG): container finished" podID="2b71ddaa-a90a-4764-a703-91f6f81db363" containerID="783cdf54ed93425253fcd9b1bddfba051f2d0510a5f7456338e0f5e6004b631d" exitCode=0
Oct 30 19:27:47.929349 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:47.929177    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-c60f5c-openshift-default-9c4d97949-vk946"
Oct 30 19:27:47.929349 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:47.929201    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-c60f5c-openshift-default-9c4d97949-vk946" event={"ID":"2b71ddaa-a90a-4764-a703-91f6f81db363","Type":"ContainerDied","Data":"783cdf54ed93425253fcd9b1bddfba051f2d0510a5f7456338e0f5e6004b631d"}
Oct 30 19:27:47.929349 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:47.929253    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-c60f5c-openshift-default-9c4d97949-vk946" event={"ID":"2b71ddaa-a90a-4764-a703-91f6f81db363","Type":"ContainerDied","Data":"c0d72e9035a002a82f2234a40d147cb0afffea57f1e19006e6707ede8fe81d51"}
Oct 30 19:27:47.929349 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:47.929276    4888 scope.go:122] "RemoveContainer" containerID="783cdf54ed93425253fcd9b1bddfba051f2d0510a5f7456338e0f5e6004b631d"
Oct 30 19:27:47.939746 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:47.939728    4888 scope.go:122] "RemoveContainer" containerID="a078a02fc2a846bde79c2a47d37550014aa112ebe408d23f4b0a85e6f74f1aa3"
Oct 30 19:27:47.949164 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:47.949144    4888 scope.go:122] "RemoveContainer" containerID="783cdf54ed93425253fcd9b1bddfba051f2d0510a5f7456338e0f5e6004b631d"
Oct 30 19:27:47.949450 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:27:47.949431    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"783cdf54ed93425253fcd9b1bddfba051f2d0510a5f7456338e0f5e6004b631d\": container with ID starting with 783cdf54ed93425253fcd9b1bddfba051f2d0510a5f7456338e0f5e6004b631d not found: ID does not exist" containerID="783cdf54ed93425253fcd9b1bddfba051f2d0510a5f7456338e0f5e6004b631d"
Oct 30 19:27:47.949515 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:47.949458    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"783cdf54ed93425253fcd9b1bddfba051f2d0510a5f7456338e0f5e6004b631d"} err="failed to get container status \"783cdf54ed93425253fcd9b1bddfba051f2d0510a5f7456338e0f5e6004b631d\": rpc error: code = NotFound desc = could not find container \"783cdf54ed93425253fcd9b1bddfba051f2d0510a5f7456338e0f5e6004b631d\": container with ID starting with 783cdf54ed93425253fcd9b1bddfba051f2d0510a5f7456338e0f5e6004b631d not found: ID does not exist"
Oct 30 19:27:47.949515 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:47.949478    4888 scope.go:122] "RemoveContainer" containerID="a078a02fc2a846bde79c2a47d37550014aa112ebe408d23f4b0a85e6f74f1aa3"
Oct 30 19:27:47.949732 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:27:47.949715    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a078a02fc2a846bde79c2a47d37550014aa112ebe408d23f4b0a85e6f74f1aa3\": container with ID starting with a078a02fc2a846bde79c2a47d37550014aa112ebe408d23f4b0a85e6f74f1aa3 not found: ID does not exist" containerID="a078a02fc2a846bde79c2a47d37550014aa112ebe408d23f4b0a85e6f74f1aa3"
Oct 30 19:27:47.949773 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:47.949738    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a078a02fc2a846bde79c2a47d37550014aa112ebe408d23f4b0a85e6f74f1aa3"} err="failed to get container status \"a078a02fc2a846bde79c2a47d37550014aa112ebe408d23f4b0a85e6f74f1aa3\": rpc error: code = NotFound desc = could not find container \"a078a02fc2a846bde79c2a47d37550014aa112ebe408d23f4b0a85e6f74f1aa3\": container with ID starting with a078a02fc2a846bde79c2a47d37550014aa112ebe408d23f4b0a85e6f74f1aa3 not found: ID does not exist"
Oct 30 19:27:48.103247 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:48.103218    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-autores-c60f5c-openshift-default-9c4d97949-vk946"]
Oct 30 19:27:48.109041 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:48.109015    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-autores-c60f5c-openshift-default-9c4d97949-vk946"]
Oct 30 19:27:49.890998 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:49.890965    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2b71ddaa-a90a-4764-a703-91f6f81db363" path="/var/lib/kubelet/pods/2b71ddaa-a90a-4764-a703-91f6f81db363/volumes"
Oct 30 19:27:51.880310 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:51.879944    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-200c46-6fb66fd44d-blvbz"]
Oct 30 19:27:51.880831 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:51.880537    4888 cpu_manager.go:414] "containerMap: removing container" podUID="2b71ddaa-a90a-4764-a703-91f6f81db363" containerName="istio-proxy"
Oct 30 19:27:51.880831 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:51.880549    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2b71ddaa-a90a-4764-a703-91f6f81db363" containerName="istio-proxy"
Oct 30 19:27:51.880831 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:51.880563    4888 cpu_manager.go:414] "containerMap: removing container" podUID="2b71ddaa-a90a-4764-a703-91f6f81db363" containerName="istio-proxy"
Oct 30 19:27:51.880831 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:51.880569    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2b71ddaa-a90a-4764-a703-91f6f81db363" containerName="istio-proxy"
Oct 30 19:27:51.880831 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:51.880681    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="2b71ddaa-a90a-4764-a703-91f6f81db363" containerName="istio-proxy"
Oct 30 19:27:51.880831 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:51.880689    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="2b71ddaa-a90a-4764-a703-91f6f81db363" containerName="istio-proxy"
Oct 30 19:27:51.889277 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:51.889248    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-200c46-6fb66fd44d-blvbz"
Oct 30 19:27:51.942001 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:51.941965    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-200c46-6fb66fd44d-blvbz"]
Oct 30 19:27:51.988668 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:51.988627    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-explicit-200c46\""
Oct 30 19:27:51.988861 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:51.988733    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-explicit-200c46-dockercfg-mhk4v\""
Oct 30 19:27:52.040208 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:52.040172    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mcx54\" (UniqueName: \"kubernetes.io/projected/b8ad2219-bda0-4eb5-b2d8-4f922611b50a-kube-api-access-mcx54\") pod \"payload-processing-e2e-explicit-200c46-6fb66fd44d-blvbz\" (UID: \"b8ad2219-bda0-4eb5-b2d8-4f922611b50a\") " pod="openshift-ingress/payload-processing-e2e-explicit-200c46-6fb66fd44d-blvbz"
Oct 30 19:27:52.040428 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:52.040313    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b8ad2219-bda0-4eb5-b2d8-4f922611b50a-plugins-config-volume\") pod \"payload-processing-e2e-explicit-200c46-6fb66fd44d-blvbz\" (UID: \"b8ad2219-bda0-4eb5-b2d8-4f922611b50a\") " pod="openshift-ingress/payload-processing-e2e-explicit-200c46-6fb66fd44d-blvbz"
Oct 30 19:27:52.141504 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:52.141412    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mcx54\" (UniqueName: \"kubernetes.io/projected/b8ad2219-bda0-4eb5-b2d8-4f922611b50a-kube-api-access-mcx54\") pod \"payload-processing-e2e-explicit-200c46-6fb66fd44d-blvbz\" (UID: \"b8ad2219-bda0-4eb5-b2d8-4f922611b50a\") " pod="openshift-ingress/payload-processing-e2e-explicit-200c46-6fb66fd44d-blvbz"
Oct 30 19:27:52.141504 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:52.141467    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b8ad2219-bda0-4eb5-b2d8-4f922611b50a-plugins-config-volume\") pod \"payload-processing-e2e-explicit-200c46-6fb66fd44d-blvbz\" (UID: \"b8ad2219-bda0-4eb5-b2d8-4f922611b50a\") " pod="openshift-ingress/payload-processing-e2e-explicit-200c46-6fb66fd44d-blvbz"
Oct 30 19:27:52.142278 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:52.142258    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b8ad2219-bda0-4eb5-b2d8-4f922611b50a-plugins-config-volume\") pod \"payload-processing-e2e-explicit-200c46-6fb66fd44d-blvbz\" (UID: \"b8ad2219-bda0-4eb5-b2d8-4f922611b50a\") " pod="openshift-ingress/payload-processing-e2e-explicit-200c46-6fb66fd44d-blvbz"
Oct 30 19:27:52.204654 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:52.204621    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mcx54\" (UniqueName: \"kubernetes.io/projected/b8ad2219-bda0-4eb5-b2d8-4f922611b50a-kube-api-access-mcx54\") pod \"payload-processing-e2e-explicit-200c46-6fb66fd44d-blvbz\" (UID: \"b8ad2219-bda0-4eb5-b2d8-4f922611b50a\") " pod="openshift-ingress/payload-processing-e2e-explicit-200c46-6fb66fd44d-blvbz"
Oct 30 19:27:52.211491 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:52.211462    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-200c46-6fb66fd44d-blvbz"
Oct 30 19:27:52.598774 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:52.598744    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-200c46-6fb66fd44d-blvbz"]
Oct 30 19:27:52.599266 ip-10-0-134-46 kubenswrapper[4888]: W1030 19:27:52.599241    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8ad2219_bda0_4eb5_b2d8_4f922611b50a.slice/crio-d9d65448646b7f77e323b06d8c2cc11ae7dd9adafd688557498010d170793250 WatchSource:0}: Error finding container d9d65448646b7f77e323b06d8c2cc11ae7dd9adafd688557498010d170793250: Status 404 returned error can't find the container with id d9d65448646b7f77e323b06d8c2cc11ae7dd9adafd688557498010d170793250
Oct 30 19:27:52.970313 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:52.970187    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-200c46-6fb66fd44d-blvbz" event={"ID":"b8ad2219-bda0-4eb5-b2d8-4f922611b50a","Type":"ContainerStarted","Data":"b8c9321d71fb3596406cf9fa63d3ede45e13aebe612155a002c648729115e0e9"}
Oct 30 19:27:52.970761 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:52.970342    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-200c46-6fb66fd44d-blvbz" event={"ID":"b8ad2219-bda0-4eb5-b2d8-4f922611b50a","Type":"ContainerStarted","Data":"d9d65448646b7f77e323b06d8c2cc11ae7dd9adafd688557498010d170793250"}
Oct 30 19:27:52.970761 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:52.970378    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-explicit-200c46-6fb66fd44d-blvbz"
Oct 30 19:27:53.094318 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:27:53.092138    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-explicit-200c46-6fb66fd44d-blvbz" podStartSLOduration=2.092122087 podStartE2EDuration="2.092122087s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:27:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:27:53.090363156 +0000 UTC m=+3237.337769182" watchObservedRunningTime="2026-10-30 19:27:53.092122087 +0000 UTC m=+3237.339528106"
Oct 30 19:28:03.981154 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:28:03.981119    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-explicit-200c46-6fb66fd44d-blvbz"
Oct 30 19:28:05.621710 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:28:05.621680    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-jbsw6" containerName="extractor"
Oct 30 19:28:09.242140 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:28:09.242105    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-rjmdz"]
Oct 30 19:28:35.621577 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:28:35.621497    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-jbsw6" containerName="extractor"
Oct 30 19:28:55.288134 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:28:55.288096    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-81d66e-7bf78c4974-d779b"]
Oct 30 19:28:55.288747 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:28:55.288307    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-81d66e-7bf78c4974-d779b" podUID="cde83ed8-4964-4ba0-93c8-d71f10bdbcba" containerName="maas-api" containerID="cri-o://ff6080ca18d69c4442d019a04011ab885a7519cba317a21d1e2474f517b0baee" gracePeriod=30
Oct 30 19:28:55.419922 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:28:55.419885    4888 generic.go:356] "Generic (PLEG): container finished" podID="cde83ed8-4964-4ba0-93c8-d71f10bdbcba" containerID="ff6080ca18d69c4442d019a04011ab885a7519cba317a21d1e2474f517b0baee" exitCode=0
Oct 30 19:28:55.420123 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:28:55.419966    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-81d66e-7bf78c4974-d779b" event={"ID":"cde83ed8-4964-4ba0-93c8-d71f10bdbcba","Type":"ContainerDied","Data":"ff6080ca18d69c4442d019a04011ab885a7519cba317a21d1e2474f517b0baee"}
Oct 30 19:28:55.566778 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:28:55.566755    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-81d66e-7bf78c4974-d779b"
Oct 30 19:28:55.627876 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:28:55.627853    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/cde83ed8-4964-4ba0-93c8-d71f10bdbcba-metrics-certs\" (UniqueName: \"kubernetes.io/secret/cde83ed8-4964-4ba0-93c8-d71f10bdbcba-metrics-certs\") pod \"cde83ed8-4964-4ba0-93c8-d71f10bdbcba\" (UID: \"cde83ed8-4964-4ba0-93c8-d71f10bdbcba\") "
Oct 30 19:28:55.628013 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:28:55.627905    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/cde83ed8-4964-4ba0-93c8-d71f10bdbcba-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cde83ed8-4964-4ba0-93c8-d71f10bdbcba-openshift-service-ca\") pod \"cde83ed8-4964-4ba0-93c8-d71f10bdbcba\" (UID: \"cde83ed8-4964-4ba0-93c8-d71f10bdbcba\") "
Oct 30 19:28:55.628013 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:28:55.627992    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cde83ed8-4964-4ba0-93c8-d71f10bdbcba-kube-api-access-2pr6n\" (UniqueName: \"kubernetes.io/projected/cde83ed8-4964-4ba0-93c8-d71f10bdbcba-kube-api-access-2pr6n\") pod \"cde83ed8-4964-4ba0-93c8-d71f10bdbcba\" (UID: \"cde83ed8-4964-4ba0-93c8-d71f10bdbcba\") "
Oct 30 19:28:55.628144 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:28:55.628131    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/cde83ed8-4964-4ba0-93c8-d71f10bdbcba-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cde83ed8-4964-4ba0-93c8-d71f10bdbcba-maas-api-tls\") pod \"cde83ed8-4964-4ba0-93c8-d71f10bdbcba\" (UID: \"cde83ed8-4964-4ba0-93c8-d71f10bdbcba\") "
Oct 30 19:28:55.628356 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:28:55.628327    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cde83ed8-4964-4ba0-93c8-d71f10bdbcba-openshift-service-ca" pod "cde83ed8-4964-4ba0-93c8-d71f10bdbcba" (UID: "cde83ed8-4964-4ba0-93c8-d71f10bdbcba"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:28:55.628621 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:28:55.628602    4888 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cde83ed8-4964-4ba0-93c8-d71f10bdbcba-openshift-service-ca\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:28:55.630158 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:28:55.630129    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cde83ed8-4964-4ba0-93c8-d71f10bdbcba-metrics-certs" pod "cde83ed8-4964-4ba0-93c8-d71f10bdbcba" (UID: "cde83ed8-4964-4ba0-93c8-d71f10bdbcba"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:28:55.630265 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:28:55.630249    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cde83ed8-4964-4ba0-93c8-d71f10bdbcba-maas-api-tls" pod "cde83ed8-4964-4ba0-93c8-d71f10bdbcba" (UID: "cde83ed8-4964-4ba0-93c8-d71f10bdbcba"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:28:55.630384 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:28:55.630369    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cde83ed8-4964-4ba0-93c8-d71f10bdbcba-kube-api-access-2pr6n" pod "cde83ed8-4964-4ba0-93c8-d71f10bdbcba" (UID: "cde83ed8-4964-4ba0-93c8-d71f10bdbcba"). InnerVolumeSpecName "kube-api-access-2pr6n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:28:55.730111 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:28:55.730064    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2pr6n\" (UniqueName: \"kubernetes.io/projected/cde83ed8-4964-4ba0-93c8-d71f10bdbcba-kube-api-access-2pr6n\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:28:55.730111 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:28:55.730106    4888 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cde83ed8-4964-4ba0-93c8-d71f10bdbcba-maas-api-tls\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:28:55.730398 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:28:55.730124    4888 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cde83ed8-4964-4ba0-93c8-d71f10bdbcba-metrics-certs\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:28:56.428252 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:28:56.428215    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-81d66e-7bf78c4974-d779b"
Oct 30 19:28:56.428746 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:28:56.428207    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-81d66e-7bf78c4974-d779b" event={"ID":"cde83ed8-4964-4ba0-93c8-d71f10bdbcba","Type":"ContainerDied","Data":"f7dc09d1faf1a43b82327c8f4cb71738d2fffe89db1e82c423f6dd5d7ad15d7c"}
Oct 30 19:28:56.428746 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:28:56.428344    4888 scope.go:122] "RemoveContainer" containerID="ff6080ca18d69c4442d019a04011ab885a7519cba317a21d1e2474f517b0baee"
Oct 30 19:28:56.600590 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:28:56.600562    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-81d66e-7bf78c4974-d779b"]
Oct 30 19:28:56.603610 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:28:56.603591    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-81d66e-7bf78c4974-d779b"]
Oct 30 19:28:56.757104 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:28:56.757035    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-7x4rb_382e8b31-a7fa-43f3-9f15-4c61627b0c04/console-operator/2.log"
Oct 30 19:28:56.765346 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:28:56.765325    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-7x4rb_382e8b31-a7fa-43f3-9f15-4c61627b0c04/console-operator/2.log"
Oct 30 19:28:56.765495 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:28:56.765331    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xfcdl_3f3fe764-2652-46c9-8e54-7882c0e6bcf1/ovn-acl-logging/0.log"
Oct 30 19:28:56.774306 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:28:56.774274    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xfcdl_3f3fe764-2652-46c9-8e54-7882c0e6bcf1/ovn-acl-logging/0.log"
Oct 30 19:28:57.892695 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:28:57.892662    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="cde83ed8-4964-4ba0-93c8-d71f10bdbcba" path="/var/lib/kubelet/pods/cde83ed8-4964-4ba0-93c8-d71f10bdbcba/volumes"
Oct 30 19:29:05.620665 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:29:05.620630    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-jbsw6" containerName="extractor"
Oct 30 19:29:09.436041 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:29:09.436001    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-rjmdz"]
Oct 30 19:29:20.638352 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:29:20.638322    4888 generic.go:356] "Generic (PLEG): container finished" podID="f7045fa9-0689-4317-95fc-d3bf87806ce1" containerID="011d00ec587977bd8622e89c6434b2e80ccb712763ece18f9533bf46fed0c95e" exitCode=0
Oct 30 19:29:20.638841 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:29:20.638406    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-200c46-openshift-default-5f4d5cd597-t47sj" event={"ID":"f7045fa9-0689-4317-95fc-d3bf87806ce1","Type":"ContainerDied","Data":"011d00ec587977bd8622e89c6434b2e80ccb712763ece18f9533bf46fed0c95e"}
Oct 30 19:29:20.638841 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:29:20.638835    4888 scope.go:122] "RemoveContainer" containerID="011d00ec587977bd8622e89c6434b2e80ccb712763ece18f9533bf46fed0c95e"
Oct 30 19:29:20.640345 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:29:20.640320    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:29:20.640435 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:29:20.640381    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:29:20.640435 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:29:20.640415    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:29:21.647194 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:29:21.647159    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-200c46-openshift-default-5f4d5cd597-t47sj" event={"ID":"f7045fa9-0689-4317-95fc-d3bf87806ce1","Type":"ContainerStarted","Data":"603a8e613de33663f6326a93227cb1e07514511f0e0c20a124f3a33c63e0b0d1"}
Oct 30 19:29:22.535763 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:29:22.535729    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-explicit-200c46-openshift-default-5f4d5cd597-t47sj"
Oct 30 19:29:22.535970 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:29:22.535897    4888 patch_prober.go:29] interesting pod/e2e-explicit-200c46-openshift-default-5f4d5cd597-t47sj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.154:15021/healthz/ready\": dial tcp 10.132.0.154:15021: connect: connection refused" start-of-body=
Oct 30 19:29:22.536054 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:29:22.535969    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-200c46-openshift-default-5f4d5cd597-t47sj" podUID="f7045fa9-0689-4317-95fc-d3bf87806ce1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.154:15021/healthz/ready\": dial tcp 10.132.0.154:15021: connect: connection refused"
Oct 30 19:29:23.536494 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:29:23.536463    4888 patch_prober.go:29] interesting pod/e2e-explicit-200c46-openshift-default-5f4d5cd597-t47sj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.154:15021/healthz/ready\": dial tcp 10.132.0.154:15021: connect: connection refused" start-of-body=
Oct 30 19:29:23.536882 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:29:23.536517    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-200c46-openshift-default-5f4d5cd597-t47sj" podUID="f7045fa9-0689-4317-95fc-d3bf87806ce1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.154:15021/healthz/ready\": dial tcp 10.132.0.154:15021: connect: connection refused"
Oct 30 19:29:24.536140 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:29:24.536103    4888 patch_prober.go:29] interesting pod/e2e-explicit-200c46-openshift-default-5f4d5cd597-t47sj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.154:15021/healthz/ready\": dial tcp 10.132.0.154:15021: connect: connection refused" start-of-body=
Oct 30 19:29:24.536351 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:29:24.536156    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-200c46-openshift-default-5f4d5cd597-t47sj" podUID="f7045fa9-0689-4317-95fc-d3bf87806ce1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.154:15021/healthz/ready\": dial tcp 10.132.0.154:15021: connect: connection refused"
Oct 30 19:29:25.535905 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:29:25.535864    4888 patch_prober.go:29] interesting pod/e2e-explicit-200c46-openshift-default-5f4d5cd597-t47sj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.154:15021/healthz/ready\": dial tcp 10.132.0.154:15021: connect: connection refused" start-of-body=
Oct 30 19:29:25.536342 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:29:25.535921    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-200c46-openshift-default-5f4d5cd597-t47sj" podUID="f7045fa9-0689-4317-95fc-d3bf87806ce1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.154:15021/healthz/ready\": dial tcp 10.132.0.154:15021: connect: connection refused"
Oct 30 19:29:26.536455 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:29:26.536419    4888 patch_prober.go:29] interesting pod/e2e-explicit-200c46-openshift-default-5f4d5cd597-t47sj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.154:15021/healthz/ready\": dial tcp 10.132.0.154:15021: connect: connection refused" start-of-body=
Oct 30 19:29:26.536860 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:29:26.536494    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-200c46-openshift-default-5f4d5cd597-t47sj" podUID="f7045fa9-0689-4317-95fc-d3bf87806ce1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.154:15021/healthz/ready\": dial tcp 10.132.0.154:15021: connect: connection refused"
Oct 30 19:29:27.541917 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:29:27.541891    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-explicit-200c46-openshift-default-5f4d5cd597-t47sj"
Oct 30 19:29:27.542557 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:29:27.542532    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-explicit-200c46-openshift-default-5f4d5cd597-t47sj"
Oct 30 19:29:27.543369 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:29:27.543331    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-explicit-200c46-openshift-default-5f4d5cd597-t47sj"
Oct 30 19:29:29.929219 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:29:29.929187    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-praxis-a-81d66e-openshift-default-6595ddff49-nhkjb"]
Oct 30 19:29:29.929787 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:29:29.929407    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-praxis-a-81d66e-openshift-default-6595ddff49-nhkjb" podUID="0da0531b-5d26-4468-9343-d071d984f66a" containerName="istio-proxy" containerID="cri-o://6139ff209ed15408d686eb69f0d2a7815a21ff7b5c436e487756a53552cc017d" gracePeriod=30
Oct 30 19:29:35.214660 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:29:35.214629    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-a-81d66e-openshift-default-6595ddff49-nhkjb"
Oct 30 19:29:35.386046 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:29:35.386018    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0da0531b-5d26-4468-9343-d071d984f66a-kube-api-access-4n7t7\" (UniqueName: \"kubernetes.io/projected/0da0531b-5d26-4468-9343-d071d984f66a-kube-api-access-4n7t7\") pod \"0da0531b-5d26-4468-9343-d071d984f66a\" (UID: \"0da0531b-5d26-4468-9343-d071d984f66a\") "
Oct 30 19:29:35.386233 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:29:35.386065    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0da0531b-5d26-4468-9343-d071d984f66a-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0da0531b-5d26-4468-9343-d071d984f66a-workload-certs\") pod \"0da0531b-5d26-4468-9343-d071d984f66a\" (UID: \"0da0531b-5d26-4468-9343-d071d984f66a\") "
Oct 30 19:29:35.386233 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:29:35.386086    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0da0531b-5d26-4468-9343-d071d984f66a-istio-token\" (UniqueName: \"kubernetes.io/projected/0da0531b-5d26-4468-9343-d071d984f66a-istio-token\") pod \"0da0531b-5d26-4468-9343-d071d984f66a\" (UID: \"0da0531b-5d26-4468-9343-d071d984f66a\") "
Oct 30 19:29:35.386233 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:29:35.386120    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0da0531b-5d26-4468-9343-d071d984f66a-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0da0531b-5d26-4468-9343-d071d984f66a-workload-socket\") pod \"0da0531b-5d26-4468-9343-d071d984f66a\" (UID: \"0da0531b-5d26-4468-9343-d071d984f66a\") "
Oct 30 19:29:35.386233 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:29:35.386210    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0da0531b-5d26-4468-9343-d071d984f66a-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0da0531b-5d26-4468-9343-d071d984f66a-istiod-ca-cert\") pod \"0da0531b-5d26-4468-9343-d071d984f66a\" (UID: \"0da0531b-5d26-4468-9343-d071d984f66a\") "
Oct 30 19:29:35.386518 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:29:35.386256    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0da0531b-5d26-4468-9343-d071d984f66a-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0da0531b-5d26-4468-9343-d071d984f66a-credential-socket\") pod \"0da0531b-5d26-4468-9343-d071d984f66a\" (UID: \"0da0531b-5d26-4468-9343-d071d984f66a\") "
Oct 30 19:29:35.386518 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:29:35.386337    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0da0531b-5d26-4468-9343-d071d984f66a-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0da0531b-5d26-4468-9343-d071d984f66a-istio-data\") pod \"0da0531b-5d26-4468-9343-d071d984f66a\" (UID: \"0da0531b-5d26-4468-9343-d071d984f66a\") "
Oct 30 19:29:35.386518 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:29:35.386333    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0da0531b-5d26-4468-9343-d071d984f66a-workload-certs" pod "0da0531b-5d26-4468-9343-d071d984f66a" (UID: "0da0531b-5d26-4468-9343-d071d984f66a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:29:35.386518 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:29:35.386493    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0da0531b-5d26-4468-9343-d071d984f66a-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0da0531b-5d26-4468-9343-d071d984f66a-istio-envoy\") pod \"0da0531b-5d26-4468-9343-d071d984f66a\" (UID: \"0da0531b-5d26-4468-9343-d071d984f66a\") "
Oct 30 19:29:35.386744 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:29:35.386558    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0da0531b-5d26-4468-9343-d071d984f66a-istio-data" pod "0da0531b-5d26-4468-9343-d071d984f66a" (UID: "0da0531b-5d26-4468-9343-d071d984f66a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:29:35.386744 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:29:35.386568    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0da0531b-5d26-4468-9343-d071d984f66a-credential-socket" pod "0da0531b-5d26-4468-9343-d071d984f66a" (UID: "0da0531b-5d26-4468-9343-d071d984f66a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:29:35.386744 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:29:35.386693    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0da0531b-5d26-4468-9343-d071d984f66a-istiod-ca-cert" pod "0da0531b-5d26-4468-9343-d071d984f66a" (UID: "0da0531b-5d26-4468-9343-d071d984f66a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:29:35.386744 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:29:35.386704    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0da0531b-5d26-4468-9343-d071d984f66a-workload-socket" pod "0da0531b-5d26-4468-9343-d071d984f66a" (UID: "0da0531b-5d26-4468-9343-d071d984f66a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:29:35.387169 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:29:35.387125    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/0da0531b-5d26-4468-9343-d071d984f66a-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0da0531b-5d26-4468-9343-d071d984f66a-istio-podinfo\") pod \"0da0531b-5d26-4468-9343-d071d984f66a\" (UID: \"0da0531b-5d26-4468-9343-d071d984f66a\") "
Oct 30 19:29:35.388665 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:29:35.388620    4888 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0da0531b-5d26-4468-9343-d071d984f66a-workload-certs\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:29:35.388665 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:29:35.388645    4888 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0da0531b-5d26-4468-9343-d071d984f66a-workload-socket\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:29:35.388665 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:29:35.388663    4888 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0da0531b-5d26-4468-9343-d071d984f66a-istiod-ca-cert\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:29:35.388886 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:29:35.388679    4888 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0da0531b-5d26-4468-9343-d071d984f66a-credential-socket\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:29:35.388886 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:29:35.388693    4888 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0da0531b-5d26-4468-9343-d071d984f66a-istio-data\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:29:35.388886 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:29:35.388839    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0da0531b-5d26-4468-9343-d071d984f66a-kube-api-access-4n7t7" pod "0da0531b-5d26-4468-9343-d071d984f66a" (UID: "0da0531b-5d26-4468-9343-d071d984f66a"). InnerVolumeSpecName "kube-api-access-4n7t7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:29:35.389429 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:29:35.389401    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0da0531b-5d26-4468-9343-d071d984f66a-istio-podinfo" pod "0da0531b-5d26-4468-9343-d071d984f66a" (UID: "0da0531b-5d26-4468-9343-d071d984f66a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:29:35.389526 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:29:35.389452    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0da0531b-5d26-4468-9343-d071d984f66a-istio-token" pod "0da0531b-5d26-4468-9343-d071d984f66a" (UID: "0da0531b-5d26-4468-9343-d071d984f66a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:29:35.389690 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:29:35.389672    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0da0531b-5d26-4468-9343-d071d984f66a-istio-envoy" pod "0da0531b-5d26-4468-9343-d071d984f66a" (UID: "0da0531b-5d26-4468-9343-d071d984f66a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:29:35.489545 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:29:35.489468    4888 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0da0531b-5d26-4468-9343-d071d984f66a-istio-envoy\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:29:35.489545 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:29:35.489500    4888 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0da0531b-5d26-4468-9343-d071d984f66a-istio-podinfo\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:29:35.489545 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:29:35.489515    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4n7t7\" (UniqueName: \"kubernetes.io/projected/0da0531b-5d26-4468-9343-d071d984f66a-kube-api-access-4n7t7\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:29:35.489545 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:29:35.489531    4888 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0da0531b-5d26-4468-9343-d071d984f66a-istio-token\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:29:35.620679 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:29:35.620638    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-jbsw6" containerName="extractor"
Oct 30 19:29:35.773718 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:29:35.773631    4888 generic.go:356] "Generic (PLEG): container finished" podID="0da0531b-5d26-4468-9343-d071d984f66a" containerID="6139ff209ed15408d686eb69f0d2a7815a21ff7b5c436e487756a53552cc017d" exitCode=0
Oct 30 19:29:35.773718 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:29:35.773685    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-a-81d66e-openshift-default-6595ddff49-nhkjb"
Oct 30 19:29:35.773914 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:29:35.773714    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-81d66e-openshift-default-6595ddff49-nhkjb" event={"ID":"0da0531b-5d26-4468-9343-d071d984f66a","Type":"ContainerDied","Data":"6139ff209ed15408d686eb69f0d2a7815a21ff7b5c436e487756a53552cc017d"}
Oct 30 19:29:35.773914 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:29:35.773767    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-a-81d66e-openshift-default-6595ddff49-nhkjb" event={"ID":"0da0531b-5d26-4468-9343-d071d984f66a","Type":"ContainerDied","Data":"5779a13da2ac320744909ab6efd129e7302b36eb655cec2e663b44e69fc75592"}
Oct 30 19:29:35.773914 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:29:35.773795    4888 scope.go:122] "RemoveContainer" containerID="6139ff209ed15408d686eb69f0d2a7815a21ff7b5c436e487756a53552cc017d"
Oct 30 19:29:35.785352 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:29:35.785333    4888 scope.go:122] "RemoveContainer" containerID="f2cc3e4ef1a98a82945d2ae0413bc975ccbd02f3d1cad4912d65607ba3f3979e"
Oct 30 19:29:35.807355 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:29:35.807330    4888 scope.go:122] "RemoveContainer" containerID="6139ff209ed15408d686eb69f0d2a7815a21ff7b5c436e487756a53552cc017d"
Oct 30 19:29:35.807703 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:29:35.807681    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6139ff209ed15408d686eb69f0d2a7815a21ff7b5c436e487756a53552cc017d\": container with ID starting with 6139ff209ed15408d686eb69f0d2a7815a21ff7b5c436e487756a53552cc017d not found: ID does not exist" containerID="6139ff209ed15408d686eb69f0d2a7815a21ff7b5c436e487756a53552cc017d"
Oct 30 19:29:35.807791 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:29:35.807723    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6139ff209ed15408d686eb69f0d2a7815a21ff7b5c436e487756a53552cc017d"} err="failed to get container status \"6139ff209ed15408d686eb69f0d2a7815a21ff7b5c436e487756a53552cc017d\": rpc error: code = NotFound desc = could not find container \"6139ff209ed15408d686eb69f0d2a7815a21ff7b5c436e487756a53552cc017d\": container with ID starting with 6139ff209ed15408d686eb69f0d2a7815a21ff7b5c436e487756a53552cc017d not found: ID does not exist"
Oct 30 19:29:35.807791 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:29:35.807748    4888 scope.go:122] "RemoveContainer" containerID="f2cc3e4ef1a98a82945d2ae0413bc975ccbd02f3d1cad4912d65607ba3f3979e"
Oct 30 19:29:35.808044 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:29:35.808026    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f2cc3e4ef1a98a82945d2ae0413bc975ccbd02f3d1cad4912d65607ba3f3979e\": container with ID starting with f2cc3e4ef1a98a82945d2ae0413bc975ccbd02f3d1cad4912d65607ba3f3979e not found: ID does not exist" containerID="f2cc3e4ef1a98a82945d2ae0413bc975ccbd02f3d1cad4912d65607ba3f3979e"
Oct 30 19:29:35.808114 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:29:35.808061    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f2cc3e4ef1a98a82945d2ae0413bc975ccbd02f3d1cad4912d65607ba3f3979e"} err="failed to get container status \"f2cc3e4ef1a98a82945d2ae0413bc975ccbd02f3d1cad4912d65607ba3f3979e\": rpc error: code = NotFound desc = could not find container \"f2cc3e4ef1a98a82945d2ae0413bc975ccbd02f3d1cad4912d65607ba3f3979e\": container with ID starting with f2cc3e4ef1a98a82945d2ae0413bc975ccbd02f3d1cad4912d65607ba3f3979e not found: ID does not exist"
Oct 30 19:29:35.944801 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:29:35.944767    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-praxis-a-81d66e-openshift-default-6595ddff49-nhkjb"]
Oct 30 19:29:35.947857 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:29:35.947833    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-praxis-a-81d66e-openshift-default-6595ddff49-nhkjb"]
Oct 30 19:29:37.890812 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:29:37.890779    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0da0531b-5d26-4468-9343-d071d984f66a" path="/var/lib/kubelet/pods/0da0531b-5d26-4468-9343-d071d984f66a/volumes"
Oct 30 19:29:52.049753 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:29:52.049703    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-rjmdz"]
Oct 30 19:30:00.175203 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:00.175164    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-8f9c6b-29889810-bcs5p"]
Oct 30 19:30:00.175788 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:00.175773    4888 cpu_manager.go:414] "containerMap: removing container" podUID="0da0531b-5d26-4468-9343-d071d984f66a" containerName="istio-proxy"
Oct 30 19:30:00.175788 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:00.175788    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0da0531b-5d26-4468-9343-d071d984f66a" containerName="istio-proxy"
Oct 30 19:30:00.175906 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:00.175806    4888 cpu_manager.go:414] "containerMap: removing container" podUID="0da0531b-5d26-4468-9343-d071d984f66a" containerName="istio-proxy"
Oct 30 19:30:00.175906 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:00.175812    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0da0531b-5d26-4468-9343-d071d984f66a" containerName="istio-proxy"
Oct 30 19:30:00.175906 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:00.175841    4888 cpu_manager.go:414] "containerMap: removing container" podUID="cde83ed8-4964-4ba0-93c8-d71f10bdbcba" containerName="maas-api"
Oct 30 19:30:00.175906 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:00.175846    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cde83ed8-4964-4ba0-93c8-d71f10bdbcba" containerName="maas-api"
Oct 30 19:30:00.176110 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:00.175924    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="0da0531b-5d26-4468-9343-d071d984f66a" containerName="istio-proxy"
Oct 30 19:30:00.176110 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:00.175938    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="cde83ed8-4964-4ba0-93c8-d71f10bdbcba" containerName="maas-api"
Oct 30 19:30:00.180510 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:00.180486    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-8f9c6b-29889810-bcs5p"
Oct 30 19:30:00.209351 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:00.209318    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-fd61e3-29889810-gkhfl"]
Oct 30 19:30:00.210721 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:00.210698    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="0da0531b-5d26-4468-9343-d071d984f66a" containerName="istio-proxy"
Oct 30 19:30:00.214611 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:00.214587    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-explicit-200c46-29889810-hvmcj"]
Oct 30 19:30:00.214778 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:00.214758    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-fd61e3-29889810-gkhfl"
Oct 30 19:30:00.218409 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:00.218385    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-explicit-200c46-29889810-hvmcj"
Oct 30 19:30:00.234278 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:00.234253    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889810-wxpvs"]
Oct 30 19:30:00.239452 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:00.239432    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-2ade09-29889810-fz97s"]
Oct 30 19:30:00.239575 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:00.239554    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889810-wxpvs"
Oct 30 19:30:00.243234 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:00.243215    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-2ade09-29889810-fz97s"
Oct 30 19:30:00.259405 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:00.259382    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-2qg7n\""
Oct 30 19:30:00.276474 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:00.276452    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-8f9c6b-29889810-bcs5p"]
Oct 30 19:30:00.280857 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:00.280823    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1354e989-63a9-423d-947c-6a1f3e38162d-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-explicit-200c46-29889810-hvmcj\" (UID: \"1354e989-63a9-423d-947c-6a1f3e38162d\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-explicit-200c46-29889810-hvmcj"
Oct 30 19:30:00.280960 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:00.280875    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hhh29\" (UniqueName: \"kubernetes.io/projected/733efe29-d46f-44e7-883f-0609c99e4531-kube-api-access-hhh29\") pod \"maas-api-key-cleanup-e2e-worker-w1-fd61e3-29889810-gkhfl\" (UID: \"733efe29-d46f-44e7-883f-0609c99e4531\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-fd61e3-29889810-gkhfl"
Oct 30 19:30:00.281060 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:00.281037    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/120337e2-9322-456b-ab8e-a29e285b8bba-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-a-w2-8f9c6b-29889810-bcs5p\" (UID: \"120337e2-9322-456b-ab8e-a29e285b8bba\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-8f9c6b-29889810-bcs5p"
Oct 30 19:30:00.281181 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:00.281161    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dd2gg\" (UniqueName: \"kubernetes.io/projected/120337e2-9322-456b-ab8e-a29e285b8bba-kube-api-access-dd2gg\") pod \"maas-api-key-cleanup-e2e-shared-a-w2-8f9c6b-29889810-bcs5p\" (UID: \"120337e2-9322-456b-ab8e-a29e285b8bba\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-8f9c6b-29889810-bcs5p"
Oct 30 19:30:00.281318 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:00.281274    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mmbb5\" (UniqueName: \"kubernetes.io/projected/1354e989-63a9-423d-947c-6a1f3e38162d-kube-api-access-mmbb5\") pod \"maas-api-key-cleanup-e2e-explicit-200c46-29889810-hvmcj\" (UID: \"1354e989-63a9-423d-947c-6a1f3e38162d\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-explicit-200c46-29889810-hvmcj"
Oct 30 19:30:00.281395 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:00.281377    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/733efe29-d46f-44e7-883f-0609c99e4531-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w1-fd61e3-29889810-gkhfl\" (UID: \"733efe29-d46f-44e7-883f-0609c99e4531\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-fd61e3-29889810-gkhfl"
Oct 30 19:30:00.335800 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:00.335767    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-explicit-200c46-29889810-hvmcj"]
Oct 30 19:30:00.342240 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:00.342208    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-fd61e3-29889810-gkhfl"]
Oct 30 19:30:00.372771 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:00.372741    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-2ade09-29889810-fz97s"]
Oct 30 19:30:00.384129 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:00.384098    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dxg76\" (UniqueName: \"kubernetes.io/projected/526209fb-5947-47b1-81e2-218d65c87301-kube-api-access-dxg76\") pod \"maas-api-key-cleanup-e2e-worker-w0-2ade09-29889810-fz97s\" (UID: \"526209fb-5947-47b1-81e2-218d65c87301\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-2ade09-29889810-fz97s"
Oct 30 19:30:00.384317 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:00.384148    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/120337e2-9322-456b-ab8e-a29e285b8bba-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-a-w2-8f9c6b-29889810-bcs5p\" (UID: \"120337e2-9322-456b-ab8e-a29e285b8bba\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-8f9c6b-29889810-bcs5p"
Oct 30 19:30:00.384317 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:00.384214    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/526209fb-5947-47b1-81e2-218d65c87301-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w0-2ade09-29889810-fz97s\" (UID: \"526209fb-5947-47b1-81e2-218d65c87301\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-2ade09-29889810-fz97s"
Oct 30 19:30:00.385017 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:00.384992    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/120337e2-9322-456b-ab8e-a29e285b8bba-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-a-w2-8f9c6b-29889810-bcs5p\" (UID: \"120337e2-9322-456b-ab8e-a29e285b8bba\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-8f9c6b-29889810-bcs5p"
Oct 30 19:30:00.385108 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:00.385092    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dd2gg\" (UniqueName: \"kubernetes.io/projected/120337e2-9322-456b-ab8e-a29e285b8bba-kube-api-access-dd2gg\") pod \"maas-api-key-cleanup-e2e-shared-a-w2-8f9c6b-29889810-bcs5p\" (UID: \"120337e2-9322-456b-ab8e-a29e285b8bba\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-8f9c6b-29889810-bcs5p"
Oct 30 19:30:00.385176 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:00.385163    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ac050065-1900-4689-a024-2d499fc6e145-maas-api-service-ca\") pod \"maas-api-key-cleanup-29889810-wxpvs\" (UID: \"ac050065-1900-4689-a024-2d499fc6e145\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889810-wxpvs"
Oct 30 19:30:00.385240 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:00.385227    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mmbb5\" (UniqueName: \"kubernetes.io/projected/1354e989-63a9-423d-947c-6a1f3e38162d-kube-api-access-mmbb5\") pod \"maas-api-key-cleanup-e2e-explicit-200c46-29889810-hvmcj\" (UID: \"1354e989-63a9-423d-947c-6a1f3e38162d\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-explicit-200c46-29889810-hvmcj"
Oct 30 19:30:00.385276 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:00.385264    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/733efe29-d46f-44e7-883f-0609c99e4531-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w1-fd61e3-29889810-gkhfl\" (UID: \"733efe29-d46f-44e7-883f-0609c99e4531\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-fd61e3-29889810-gkhfl"
Oct 30 19:30:00.385365 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:00.385351    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6jnz8\" (UniqueName: \"kubernetes.io/projected/ac050065-1900-4689-a024-2d499fc6e145-kube-api-access-6jnz8\") pod \"maas-api-key-cleanup-29889810-wxpvs\" (UID: \"ac050065-1900-4689-a024-2d499fc6e145\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889810-wxpvs"
Oct 30 19:30:00.385410 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:00.385398    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1354e989-63a9-423d-947c-6a1f3e38162d-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-explicit-200c46-29889810-hvmcj\" (UID: \"1354e989-63a9-423d-947c-6a1f3e38162d\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-explicit-200c46-29889810-hvmcj"
Oct 30 19:30:00.385449 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:00.385435    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hhh29\" (UniqueName: \"kubernetes.io/projected/733efe29-d46f-44e7-883f-0609c99e4531-kube-api-access-hhh29\") pod \"maas-api-key-cleanup-e2e-worker-w1-fd61e3-29889810-gkhfl\" (UID: \"733efe29-d46f-44e7-883f-0609c99e4531\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-fd61e3-29889810-gkhfl"
Oct 30 19:30:00.386753 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:00.386732    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/733efe29-d46f-44e7-883f-0609c99e4531-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w1-fd61e3-29889810-gkhfl\" (UID: \"733efe29-d46f-44e7-883f-0609c99e4531\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-fd61e3-29889810-gkhfl"
Oct 30 19:30:00.387378 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:00.387360    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1354e989-63a9-423d-947c-6a1f3e38162d-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-explicit-200c46-29889810-hvmcj\" (UID: \"1354e989-63a9-423d-947c-6a1f3e38162d\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-explicit-200c46-29889810-hvmcj"
Oct 30 19:30:00.394726 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:00.394701    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889810-wxpvs"]
Oct 30 19:30:00.448766 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:00.448698    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hhh29\" (UniqueName: \"kubernetes.io/projected/733efe29-d46f-44e7-883f-0609c99e4531-kube-api-access-hhh29\") pod \"maas-api-key-cleanup-e2e-worker-w1-fd61e3-29889810-gkhfl\" (UID: \"733efe29-d46f-44e7-883f-0609c99e4531\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-fd61e3-29889810-gkhfl"
Oct 30 19:30:00.452395 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:00.452366    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dd2gg\" (UniqueName: \"kubernetes.io/projected/120337e2-9322-456b-ab8e-a29e285b8bba-kube-api-access-dd2gg\") pod \"maas-api-key-cleanup-e2e-shared-a-w2-8f9c6b-29889810-bcs5p\" (UID: \"120337e2-9322-456b-ab8e-a29e285b8bba\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-8f9c6b-29889810-bcs5p"
Oct 30 19:30:00.452538 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:00.452451    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mmbb5\" (UniqueName: \"kubernetes.io/projected/1354e989-63a9-423d-947c-6a1f3e38162d-kube-api-access-mmbb5\") pod \"maas-api-key-cleanup-e2e-explicit-200c46-29889810-hvmcj\" (UID: \"1354e989-63a9-423d-947c-6a1f3e38162d\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-explicit-200c46-29889810-hvmcj"
Oct 30 19:30:00.486727 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:00.486689    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dxg76\" (UniqueName: \"kubernetes.io/projected/526209fb-5947-47b1-81e2-218d65c87301-kube-api-access-dxg76\") pod \"maas-api-key-cleanup-e2e-worker-w0-2ade09-29889810-fz97s\" (UID: \"526209fb-5947-47b1-81e2-218d65c87301\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-2ade09-29889810-fz97s"
Oct 30 19:30:00.486921 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:00.486833    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/526209fb-5947-47b1-81e2-218d65c87301-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w0-2ade09-29889810-fz97s\" (UID: \"526209fb-5947-47b1-81e2-218d65c87301\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-2ade09-29889810-fz97s"
Oct 30 19:30:00.486995 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:00.486955    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ac050065-1900-4689-a024-2d499fc6e145-maas-api-service-ca\") pod \"maas-api-key-cleanup-29889810-wxpvs\" (UID: \"ac050065-1900-4689-a024-2d499fc6e145\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889810-wxpvs"
Oct 30 19:30:00.487104 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:00.487085    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6jnz8\" (UniqueName: \"kubernetes.io/projected/ac050065-1900-4689-a024-2d499fc6e145-kube-api-access-6jnz8\") pod \"maas-api-key-cleanup-29889810-wxpvs\" (UID: \"ac050065-1900-4689-a024-2d499fc6e145\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889810-wxpvs"
Oct 30 19:30:00.487679 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:00.487655    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/526209fb-5947-47b1-81e2-218d65c87301-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w0-2ade09-29889810-fz97s\" (UID: \"526209fb-5947-47b1-81e2-218d65c87301\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-2ade09-29889810-fz97s"
Oct 30 19:30:00.488169 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:00.488148    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ac050065-1900-4689-a024-2d499fc6e145-maas-api-service-ca\") pod \"maas-api-key-cleanup-29889810-wxpvs\" (UID: \"ac050065-1900-4689-a024-2d499fc6e145\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889810-wxpvs"
Oct 30 19:30:00.497484 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:00.497455    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-8f9c6b-29889810-bcs5p"
Oct 30 19:30:00.530477 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:00.530440    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-fd61e3-29889810-gkhfl"
Oct 30 19:30:00.538999 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:00.538968    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-explicit-200c46-29889810-hvmcj"
Oct 30 19:30:00.554987 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:00.554962    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dxg76\" (UniqueName: \"kubernetes.io/projected/526209fb-5947-47b1-81e2-218d65c87301-kube-api-access-dxg76\") pod \"maas-api-key-cleanup-e2e-worker-w0-2ade09-29889810-fz97s\" (UID: \"526209fb-5947-47b1-81e2-218d65c87301\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-2ade09-29889810-fz97s"
Oct 30 19:30:00.556090 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:00.556068    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6jnz8\" (UniqueName: \"kubernetes.io/projected/ac050065-1900-4689-a024-2d499fc6e145-kube-api-access-6jnz8\") pod \"maas-api-key-cleanup-29889810-wxpvs\" (UID: \"ac050065-1900-4689-a024-2d499fc6e145\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889810-wxpvs"
Oct 30 19:30:00.565605 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:00.563616    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-2ade09-29889810-fz97s"
Oct 30 19:30:00.854625 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:00.854597    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889810-wxpvs"
Oct 30 19:30:00.911943 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:00.911917    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-8f9c6b-29889810-bcs5p"]
Oct 30 19:30:00.976043 ip-10-0-134-46 kubenswrapper[4888]: W1030 19:30:00.975991    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod733efe29_d46f_44e7_883f_0609c99e4531.slice/crio-701d779220d12fcdaf273c6b66110c173f555d40c27bb8fe131f1b7d9b4b8200 WatchSource:0}: Error finding container 701d779220d12fcdaf273c6b66110c173f555d40c27bb8fe131f1b7d9b4b8200: Status 404 returned error can't find the container with id 701d779220d12fcdaf273c6b66110c173f555d40c27bb8fe131f1b7d9b4b8200
Oct 30 19:30:00.977409 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:00.977374    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-8f9c6b-29889810-bcs5p" event={"ID":"120337e2-9322-456b-ab8e-a29e285b8bba","Type":"ContainerStarted","Data":"8b39d849a25392c3f4e9e45ec02831fdc6c5d3c2934756bf3f1f6a345263fbcd"}
Oct 30 19:30:00.980472 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:00.980449    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-fd61e3-29889810-gkhfl"]
Oct 30 19:30:01.025457 ip-10-0-134-46 kubenswrapper[4888]: W1030 19:30:01.025427    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod526209fb_5947_47b1_81e2_218d65c87301.slice/crio-513b0e0cdb1166e377770a44a2fad6c116469ca49fef52445c00a534e0d07162 WatchSource:0}: Error finding container 513b0e0cdb1166e377770a44a2fad6c116469ca49fef52445c00a534e0d07162: Status 404 returned error can't find the container with id 513b0e0cdb1166e377770a44a2fad6c116469ca49fef52445c00a534e0d07162
Oct 30 19:30:01.025610 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:01.025591    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-2ade09-29889810-fz97s"]
Oct 30 19:30:01.026303 ip-10-0-134-46 kubenswrapper[4888]: W1030 19:30:01.026261    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1354e989_63a9_423d_947c_6a1f3e38162d.slice/crio-28975159ae212d57701e54a77e1577eb0d82795fa91fbf0273a39f3e53a857c1 WatchSource:0}: Error finding container 28975159ae212d57701e54a77e1577eb0d82795fa91fbf0273a39f3e53a857c1: Status 404 returned error can't find the container with id 28975159ae212d57701e54a77e1577eb0d82795fa91fbf0273a39f3e53a857c1
Oct 30 19:30:01.029258 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:01.029237    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-explicit-200c46-29889810-hvmcj"]
Oct 30 19:30:01.247539 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:01.247513    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889810-wxpvs"]
Oct 30 19:30:01.248361 ip-10-0-134-46 kubenswrapper[4888]: W1030 19:30:01.248336    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac050065_1900_4689_a024_2d499fc6e145.slice/crio-26e893e6aa7bbdefffc55b04a21de5310c7227c83ea923a1317fd2c35c604079 WatchSource:0}: Error finding container 26e893e6aa7bbdefffc55b04a21de5310c7227c83ea923a1317fd2c35c604079: Status 404 returned error can't find the container with id 26e893e6aa7bbdefffc55b04a21de5310c7227c83ea923a1317fd2c35c604079
Oct 30 19:30:01.992423 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:01.992103    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889810-wxpvs" event={"ID":"ac050065-1900-4689-a024-2d499fc6e145","Type":"ContainerStarted","Data":"f16a47da480dadca33401bc7861a386baae0afc1d6512ec62b178f8190c90fcf"}
Oct 30 19:30:01.992423 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:01.992188    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889810-wxpvs" event={"ID":"ac050065-1900-4689-a024-2d499fc6e145","Type":"ContainerStarted","Data":"26e893e6aa7bbdefffc55b04a21de5310c7227c83ea923a1317fd2c35c604079"}
Oct 30 19:30:02.001789 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:01.999475    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-fd61e3-29889810-gkhfl" event={"ID":"733efe29-d46f-44e7-883f-0609c99e4531","Type":"ContainerStarted","Data":"112c53d888e5c8a05131d0cbd19ed1822a24e15204c68afa129a6c8f2f107625"}
Oct 30 19:30:02.001789 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:01.999577    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-fd61e3-29889810-gkhfl" event={"ID":"733efe29-d46f-44e7-883f-0609c99e4531","Type":"ContainerStarted","Data":"701d779220d12fcdaf273c6b66110c173f555d40c27bb8fe131f1b7d9b4b8200"}
Oct 30 19:30:02.005985 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:02.002820    4888 generic.go:356] "Generic (PLEG): container finished" podID="526209fb-5947-47b1-81e2-218d65c87301" containerID="cea814847a1c2b2bcc64040700ffa5e298c349070e41e794f1e37abaa9afd611" exitCode=60
Oct 30 19:30:02.005985 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:02.002962    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-2ade09-29889810-fz97s" event={"ID":"526209fb-5947-47b1-81e2-218d65c87301","Type":"ContainerDied","Data":"cea814847a1c2b2bcc64040700ffa5e298c349070e41e794f1e37abaa9afd611"}
Oct 30 19:30:02.005985 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:02.003013    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-2ade09-29889810-fz97s" event={"ID":"526209fb-5947-47b1-81e2-218d65c87301","Type":"ContainerStarted","Data":"513b0e0cdb1166e377770a44a2fad6c116469ca49fef52445c00a534e0d07162"}
Oct 30 19:30:02.005985 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:02.003445    4888 scope.go:122] "RemoveContainer" containerID="cea814847a1c2b2bcc64040700ffa5e298c349070e41e794f1e37abaa9afd611"
Oct 30 19:30:02.012558 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:02.012526    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-explicit-200c46-29889810-hvmcj" event={"ID":"1354e989-63a9-423d-947c-6a1f3e38162d","Type":"ContainerStarted","Data":"c982de1346c49138388468c33e19b138ec06f6479b2c64b059b1669f4e8bf322"}
Oct 30 19:30:02.012740 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:02.012723    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-explicit-200c46-29889810-hvmcj" event={"ID":"1354e989-63a9-423d-947c-6a1f3e38162d","Type":"ContainerStarted","Data":"28975159ae212d57701e54a77e1577eb0d82795fa91fbf0273a39f3e53a857c1"}
Oct 30 19:30:02.015718 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:02.015696    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-8f9c6b-29889810-bcs5p" event={"ID":"120337e2-9322-456b-ab8e-a29e285b8bba","Type":"ContainerStarted","Data":"8220db15b31463ab551fc2487daabc0799f7dcf896127f545c8406244ac5c530"}
Oct 30 19:30:02.016126 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:02.016106    4888 scope.go:122] "RemoveContainer" containerID="8220db15b31463ab551fc2487daabc0799f7dcf896127f545c8406244ac5c530"
Oct 30 19:30:02.113058 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:02.113014    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889810-wxpvs" podStartSLOduration=2.113000243 podStartE2EDuration="2.113000243s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:30:02.111415347 +0000 UTC m=+3366.358821373" watchObservedRunningTime="2026-10-30 19:30:02.113000243 +0000 UTC m=+3366.360406268"
Oct 30 19:30:02.349077 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:02.349027    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-fd61e3-29889810-gkhfl" podStartSLOduration=2.349014026 podStartE2EDuration="2.349014026s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:30:02.347237663 +0000 UTC m=+3366.594643688" watchObservedRunningTime="2026-10-30 19:30:02.349014026 +0000 UTC m=+3366.596420051"
Oct 30 19:30:02.589427 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:02.589173    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-explicit-200c46-29889810-hvmcj" podStartSLOduration=2.589154326 podStartE2EDuration="2.589154326s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:30:02.583627591 +0000 UTC m=+3366.831033618" watchObservedRunningTime="2026-10-30 19:30:02.589154326 +0000 UTC m=+3366.836560352"
Oct 30 19:30:03.025750 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:03.025665    4888 generic.go:356] "Generic (PLEG): container finished" podID="ac050065-1900-4689-a024-2d499fc6e145" containerID="f16a47da480dadca33401bc7861a386baae0afc1d6512ec62b178f8190c90fcf" exitCode=60
Oct 30 19:30:03.025932 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:03.025763    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889810-wxpvs" event={"ID":"ac050065-1900-4689-a024-2d499fc6e145","Type":"ContainerDied","Data":"f16a47da480dadca33401bc7861a386baae0afc1d6512ec62b178f8190c90fcf"}
Oct 30 19:30:03.026057 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:03.026037    4888 scope.go:122] "RemoveContainer" containerID="f16a47da480dadca33401bc7861a386baae0afc1d6512ec62b178f8190c90fcf"
Oct 30 19:30:03.027404 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:03.027383    4888 generic.go:356] "Generic (PLEG): container finished" podID="733efe29-d46f-44e7-883f-0609c99e4531" containerID="112c53d888e5c8a05131d0cbd19ed1822a24e15204c68afa129a6c8f2f107625" exitCode=60
Oct 30 19:30:03.027526 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:03.027467    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-fd61e3-29889810-gkhfl" event={"ID":"733efe29-d46f-44e7-883f-0609c99e4531","Type":"ContainerDied","Data":"112c53d888e5c8a05131d0cbd19ed1822a24e15204c68afa129a6c8f2f107625"}
Oct 30 19:30:03.027683 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:03.027666    4888 scope.go:122] "RemoveContainer" containerID="112c53d888e5c8a05131d0cbd19ed1822a24e15204c68afa129a6c8f2f107625"
Oct 30 19:30:03.029577 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:03.029553    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-2ade09-29889810-fz97s" event={"ID":"526209fb-5947-47b1-81e2-218d65c87301","Type":"ContainerStarted","Data":"73b0c483b9cac73eed8297a9493e7e2ae082b8af2e9353a5e9618a854bb96d38"}
Oct 30 19:30:03.031202 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:03.031184    4888 generic.go:356] "Generic (PLEG): container finished" podID="1354e989-63a9-423d-947c-6a1f3e38162d" containerID="c982de1346c49138388468c33e19b138ec06f6479b2c64b059b1669f4e8bf322" exitCode=60
Oct 30 19:30:03.031278 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:03.031254    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-explicit-200c46-29889810-hvmcj" event={"ID":"1354e989-63a9-423d-947c-6a1f3e38162d","Type":"ContainerDied","Data":"c982de1346c49138388468c33e19b138ec06f6479b2c64b059b1669f4e8bf322"}
Oct 30 19:30:03.031463 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:03.031445    4888 scope.go:122] "RemoveContainer" containerID="c982de1346c49138388468c33e19b138ec06f6479b2c64b059b1669f4e8bf322"
Oct 30 19:30:03.033181 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:03.033164    4888 generic.go:356] "Generic (PLEG): container finished" podID="120337e2-9322-456b-ab8e-a29e285b8bba" containerID="8220db15b31463ab551fc2487daabc0799f7dcf896127f545c8406244ac5c530" exitCode=60
Oct 30 19:30:03.033304 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:03.033225    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-8f9c6b-29889810-bcs5p" event={"ID":"120337e2-9322-456b-ab8e-a29e285b8bba","Type":"ContainerDied","Data":"8220db15b31463ab551fc2487daabc0799f7dcf896127f545c8406244ac5c530"}
Oct 30 19:30:03.033304 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:03.033263    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-8f9c6b-29889810-bcs5p" event={"ID":"120337e2-9322-456b-ab8e-a29e285b8bba","Type":"ContainerStarted","Data":"c6ff42bc67f3dc819013873c46a729bbaec4a47b5debbde4d9dde5368c064c41"}
Oct 30 19:30:03.385426 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:03.385369    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-8f9c6b-29889810-bcs5p" podStartSLOduration=3.385355659 podStartE2EDuration="3.385355659s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:30:03.384084708 +0000 UTC m=+3367.631490733" watchObservedRunningTime="2026-10-30 19:30:03.385355659 +0000 UTC m=+3367.632761683"
Oct 30 19:30:03.622975 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:03.622922    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-2ade09-29889810-fz97s" podStartSLOduration=3.622908705 podStartE2EDuration="3.622908705s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:30:03.621104441 +0000 UTC m=+3367.868510467" watchObservedRunningTime="2026-10-30 19:30:03.622908705 +0000 UTC m=+3367.870314729"
Oct 30 19:30:04.042690 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:04.042597    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-fd61e3-29889810-gkhfl" event={"ID":"733efe29-d46f-44e7-883f-0609c99e4531","Type":"ContainerStarted","Data":"d294c7375c1e9bedeb5a582a866aeb0c486947c4248dcaf97ef6b79629197cb8"}
Oct 30 19:30:04.044553 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:04.044526    4888 generic.go:356] "Generic (PLEG): container finished" podID="526209fb-5947-47b1-81e2-218d65c87301" containerID="73b0c483b9cac73eed8297a9493e7e2ae082b8af2e9353a5e9618a854bb96d38" exitCode=60
Oct 30 19:30:04.044690 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:04.044603    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-2ade09-29889810-fz97s" event={"ID":"526209fb-5947-47b1-81e2-218d65c87301","Type":"ContainerDied","Data":"73b0c483b9cac73eed8297a9493e7e2ae082b8af2e9353a5e9618a854bb96d38"}
Oct 30 19:30:04.044690 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:04.044652    4888 scope.go:122] "RemoveContainer" containerID="cea814847a1c2b2bcc64040700ffa5e298c349070e41e794f1e37abaa9afd611"
Oct 30 19:30:04.044810 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:04.044802    4888 scope.go:122] "RemoveContainer" containerID="73b0c483b9cac73eed8297a9493e7e2ae082b8af2e9353a5e9618a854bb96d38"
Oct 30 19:30:04.045112 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:30:04.045085    4888 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w0-2ade09-29889810-fz97s_odh-ai-gateway-infra(526209fb-5947-47b1-81e2-218d65c87301)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-2ade09-29889810-fz97s" podUID="526209fb-5947-47b1-81e2-218d65c87301"
Oct 30 19:30:04.046796 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:04.046775    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-explicit-200c46-29889810-hvmcj" event={"ID":"1354e989-63a9-423d-947c-6a1f3e38162d","Type":"ContainerStarted","Data":"10a6b2a7fbec4879d39451b10d6a54c4603cca4a7659f89c503044944a5d1e1b"}
Oct 30 19:30:04.048657 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:04.048635    4888 generic.go:356] "Generic (PLEG): container finished" podID="120337e2-9322-456b-ab8e-a29e285b8bba" containerID="c6ff42bc67f3dc819013873c46a729bbaec4a47b5debbde4d9dde5368c064c41" exitCode=60
Oct 30 19:30:04.048751 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:04.048723    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-8f9c6b-29889810-bcs5p" event={"ID":"120337e2-9322-456b-ab8e-a29e285b8bba","Type":"ContainerDied","Data":"c6ff42bc67f3dc819013873c46a729bbaec4a47b5debbde4d9dde5368c064c41"}
Oct 30 19:30:04.048894 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:04.048880    4888 scope.go:122] "RemoveContainer" containerID="c6ff42bc67f3dc819013873c46a729bbaec4a47b5debbde4d9dde5368c064c41"
Oct 30 19:30:04.049149 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:30:04.049125    4888 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-a-w2-8f9c6b-29889810-bcs5p_odh-ai-gateway-infra(120337e2-9322-456b-ab8e-a29e285b8bba)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-8f9c6b-29889810-bcs5p" podUID="120337e2-9322-456b-ab8e-a29e285b8bba"
Oct 30 19:30:04.050662 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:04.050641    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889810-wxpvs" event={"ID":"ac050065-1900-4689-a024-2d499fc6e145","Type":"ContainerStarted","Data":"f02a6f8bbd37076132d2972a7fed3dd1ca75118043e1ea6bf65821a0f0e5a727"}
Oct 30 19:30:04.058497 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:04.058480    4888 scope.go:122] "RemoveContainer" containerID="8220db15b31463ab551fc2487daabc0799f7dcf896127f545c8406244ac5c530"
Oct 30 19:30:05.059458 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:05.059425    4888 generic.go:356] "Generic (PLEG): container finished" podID="1354e989-63a9-423d-947c-6a1f3e38162d" containerID="10a6b2a7fbec4879d39451b10d6a54c4603cca4a7659f89c503044944a5d1e1b" exitCode=60
Oct 30 19:30:05.059958 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:05.059514    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-explicit-200c46-29889810-hvmcj" event={"ID":"1354e989-63a9-423d-947c-6a1f3e38162d","Type":"ContainerDied","Data":"10a6b2a7fbec4879d39451b10d6a54c4603cca4a7659f89c503044944a5d1e1b"}
Oct 30 19:30:05.059958 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:05.059577    4888 scope.go:122] "RemoveContainer" containerID="c982de1346c49138388468c33e19b138ec06f6479b2c64b059b1669f4e8bf322"
Oct 30 19:30:05.059958 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:05.059804    4888 scope.go:122] "RemoveContainer" containerID="10a6b2a7fbec4879d39451b10d6a54c4603cca4a7659f89c503044944a5d1e1b"
Oct 30 19:30:05.060178 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:30:05.060153    4888 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-explicit-200c46-29889810-hvmcj_odh-ai-gateway-infra(1354e989-63a9-423d-947c-6a1f3e38162d)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-explicit-200c46-29889810-hvmcj" podUID="1354e989-63a9-423d-947c-6a1f3e38162d"
Oct 30 19:30:05.061464 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:05.061448    4888 scope.go:122] "RemoveContainer" containerID="c6ff42bc67f3dc819013873c46a729bbaec4a47b5debbde4d9dde5368c064c41"
Oct 30 19:30:05.061733 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:30:05.061713    4888 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-a-w2-8f9c6b-29889810-bcs5p_odh-ai-gateway-infra(120337e2-9322-456b-ab8e-a29e285b8bba)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-8f9c6b-29889810-bcs5p" podUID="120337e2-9322-456b-ab8e-a29e285b8bba"
Oct 30 19:30:05.063038 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:05.063020    4888 generic.go:356] "Generic (PLEG): container finished" podID="ac050065-1900-4689-a024-2d499fc6e145" containerID="f02a6f8bbd37076132d2972a7fed3dd1ca75118043e1ea6bf65821a0f0e5a727" exitCode=60
Oct 30 19:30:05.063128 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:05.063108    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889810-wxpvs" event={"ID":"ac050065-1900-4689-a024-2d499fc6e145","Type":"ContainerDied","Data":"f02a6f8bbd37076132d2972a7fed3dd1ca75118043e1ea6bf65821a0f0e5a727"}
Oct 30 19:30:05.063338 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:05.063320    4888 scope.go:122] "RemoveContainer" containerID="f02a6f8bbd37076132d2972a7fed3dd1ca75118043e1ea6bf65821a0f0e5a727"
Oct 30 19:30:05.063605 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:30:05.063582    4888 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-29889810-wxpvs_odh-ai-gateway-infra(ac050065-1900-4689-a024-2d499fc6e145)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889810-wxpvs" podUID="ac050065-1900-4689-a024-2d499fc6e145"
Oct 30 19:30:05.064924 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:05.064901    4888 generic.go:356] "Generic (PLEG): container finished" podID="733efe29-d46f-44e7-883f-0609c99e4531" containerID="d294c7375c1e9bedeb5a582a866aeb0c486947c4248dcaf97ef6b79629197cb8" exitCode=60
Oct 30 19:30:05.065009 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:05.064987    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-fd61e3-29889810-gkhfl" event={"ID":"733efe29-d46f-44e7-883f-0609c99e4531","Type":"ContainerDied","Data":"d294c7375c1e9bedeb5a582a866aeb0c486947c4248dcaf97ef6b79629197cb8"}
Oct 30 19:30:05.065149 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:05.065136    4888 scope.go:122] "RemoveContainer" containerID="d294c7375c1e9bedeb5a582a866aeb0c486947c4248dcaf97ef6b79629197cb8"
Oct 30 19:30:05.065345 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:30:05.065323    4888 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w1-fd61e3-29889810-gkhfl_odh-ai-gateway-infra(733efe29-d46f-44e7-883f-0609c99e4531)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-fd61e3-29889810-gkhfl" podUID="733efe29-d46f-44e7-883f-0609c99e4531"
Oct 30 19:30:05.066781 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:05.066768    4888 scope.go:122] "RemoveContainer" containerID="73b0c483b9cac73eed8297a9493e7e2ae082b8af2e9353a5e9618a854bb96d38"
Oct 30 19:30:05.066992 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:30:05.066973    4888 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w0-2ade09-29889810-fz97s_odh-ai-gateway-infra(526209fb-5947-47b1-81e2-218d65c87301)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-2ade09-29889810-fz97s" podUID="526209fb-5947-47b1-81e2-218d65c87301"
Oct 30 19:30:05.073120 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:05.073102    4888 scope.go:122] "RemoveContainer" containerID="f16a47da480dadca33401bc7861a386baae0afc1d6512ec62b178f8190c90fcf"
Oct 30 19:30:05.088771 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:05.088750    4888 scope.go:122] "RemoveContainer" containerID="112c53d888e5c8a05131d0cbd19ed1822a24e15204c68afa129a6c8f2f107625"
Oct 30 19:30:05.619882 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:30:05.619800    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-jbsw6" containerName="extractor"
Oct 30 19:30:06.081660 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:06.081629    4888 scope.go:122] "RemoveContainer" containerID="10a6b2a7fbec4879d39451b10d6a54c4603cca4a7659f89c503044944a5d1e1b"
Oct 30 19:30:06.082154 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:30:06.081955    4888 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-explicit-200c46-29889810-hvmcj_odh-ai-gateway-infra(1354e989-63a9-423d-947c-6a1f3e38162d)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-explicit-200c46-29889810-hvmcj" podUID="1354e989-63a9-423d-947c-6a1f3e38162d"
Oct 30 19:30:06.084188 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:06.084165    4888 scope.go:122] "RemoveContainer" containerID="f02a6f8bbd37076132d2972a7fed3dd1ca75118043e1ea6bf65821a0f0e5a727"
Oct 30 19:30:06.084637 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:30:06.084612    4888 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-29889810-wxpvs_odh-ai-gateway-infra(ac050065-1900-4689-a024-2d499fc6e145)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889810-wxpvs" podUID="ac050065-1900-4689-a024-2d499fc6e145"
Oct 30 19:30:06.086167 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:06.086146    4888 scope.go:122] "RemoveContainer" containerID="d294c7375c1e9bedeb5a582a866aeb0c486947c4248dcaf97ef6b79629197cb8"
Oct 30 19:30:06.086470 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:30:06.086438    4888 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w1-fd61e3-29889810-gkhfl_odh-ai-gateway-infra(733efe29-d46f-44e7-883f-0609c99e4531)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-fd61e3-29889810-gkhfl" podUID="733efe29-d46f-44e7-883f-0609c99e4531"
Oct 30 19:30:14.883490 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:14.883449    4888 scope.go:122] "RemoveContainer" containerID="c6ff42bc67f3dc819013873c46a729bbaec4a47b5debbde4d9dde5368c064c41"
Oct 30 19:30:14.883904 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:14.883552    4888 scope.go:122] "RemoveContainer" containerID="73b0c483b9cac73eed8297a9493e7e2ae082b8af2e9353a5e9618a854bb96d38"
Oct 30 19:30:15.125646 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:15.125612    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w2-dc5cdd/auth-test-model-dc5cdd-kserve-7cb964d99c-9znzj"]
Oct 30 19:30:15.887570 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:15.887552    4888 scope.go:122] "RemoveContainer" containerID="f02a6f8bbd37076132d2972a7fed3dd1ca75118043e1ea6bf65821a0f0e5a727"
Oct 30 19:30:15.887933 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:15.887699    4888 scope.go:122] "RemoveContainer" containerID="d294c7375c1e9bedeb5a582a866aeb0c486947c4248dcaf97ef6b79629197cb8"
Oct 30 19:30:16.159726 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:16.159697    4888 generic.go:356] "Generic (PLEG): container finished" podID="526209fb-5947-47b1-81e2-218d65c87301" containerID="5ab88bddcf3ef46234ca59e426900dab8b12b263463f779623e0d4162ee52af5" exitCode=60
Oct 30 19:30:16.159889 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:16.159774    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-2ade09-29889810-fz97s" event={"ID":"526209fb-5947-47b1-81e2-218d65c87301","Type":"ContainerDied","Data":"5ab88bddcf3ef46234ca59e426900dab8b12b263463f779623e0d4162ee52af5"}
Oct 30 19:30:16.159889 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:16.159829    4888 scope.go:122] "RemoveContainer" containerID="73b0c483b9cac73eed8297a9493e7e2ae082b8af2e9353a5e9618a854bb96d38"
Oct 30 19:30:16.160230 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:16.160211    4888 scope.go:122] "RemoveContainer" containerID="5ab88bddcf3ef46234ca59e426900dab8b12b263463f779623e0d4162ee52af5"
Oct 30 19:30:16.160592 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:30:16.160559    4888 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w0-2ade09-29889810-fz97s_odh-ai-gateway-infra(526209fb-5947-47b1-81e2-218d65c87301)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-2ade09-29889810-fz97s" podUID="526209fb-5947-47b1-81e2-218d65c87301"
Oct 30 19:30:16.162105 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:16.162083    4888 generic.go:356] "Generic (PLEG): container finished" podID="120337e2-9322-456b-ab8e-a29e285b8bba" containerID="e695321ff4bbb9b028cc1dbcb6e9a8088d585f68820f303b50c495a5478600b3" exitCode=60
Oct 30 19:30:16.162211 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:16.162107    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-8f9c6b-29889810-bcs5p" event={"ID":"120337e2-9322-456b-ab8e-a29e285b8bba","Type":"ContainerDied","Data":"e695321ff4bbb9b028cc1dbcb6e9a8088d585f68820f303b50c495a5478600b3"}
Oct 30 19:30:16.162464 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:16.162442    4888 scope.go:122] "RemoveContainer" containerID="e695321ff4bbb9b028cc1dbcb6e9a8088d585f68820f303b50c495a5478600b3"
Oct 30 19:30:16.162717 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:30:16.162696    4888 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-a-w2-8f9c6b-29889810-bcs5p_odh-ai-gateway-infra(120337e2-9322-456b-ab8e-a29e285b8bba)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-8f9c6b-29889810-bcs5p" podUID="120337e2-9322-456b-ab8e-a29e285b8bba"
Oct 30 19:30:16.175074 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:16.175053    4888 scope.go:122] "RemoveContainer" containerID="c6ff42bc67f3dc819013873c46a729bbaec4a47b5debbde4d9dde5368c064c41"
Oct 30 19:30:17.170705 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:17.170672    4888 generic.go:356] "Generic (PLEG): container finished" podID="733efe29-d46f-44e7-883f-0609c99e4531" containerID="f285880fb8b841dba86e10d0033a44c0a76d651d183a74fe775bf01726db7d1d" exitCode=60
Oct 30 19:30:17.171205 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:17.170740    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-fd61e3-29889810-gkhfl" event={"ID":"733efe29-d46f-44e7-883f-0609c99e4531","Type":"ContainerDied","Data":"f285880fb8b841dba86e10d0033a44c0a76d651d183a74fe775bf01726db7d1d"}
Oct 30 19:30:17.171205 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:17.170821    4888 scope.go:122] "RemoveContainer" containerID="d294c7375c1e9bedeb5a582a866aeb0c486947c4248dcaf97ef6b79629197cb8"
Oct 30 19:30:17.171351 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:17.171217    4888 scope.go:122] "RemoveContainer" containerID="f285880fb8b841dba86e10d0033a44c0a76d651d183a74fe775bf01726db7d1d"
Oct 30 19:30:17.171567 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:30:17.171545    4888 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w1-fd61e3-29889810-gkhfl_odh-ai-gateway-infra(733efe29-d46f-44e7-883f-0609c99e4531)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-fd61e3-29889810-gkhfl" podUID="733efe29-d46f-44e7-883f-0609c99e4531"
Oct 30 19:30:17.176078 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:17.176054    4888 generic.go:356] "Generic (PLEG): container finished" podID="ac050065-1900-4689-a024-2d499fc6e145" containerID="ebdfe765de92d77be3827d8e367d42e443b7b5685c1b2562322c6bdcfada9131" exitCode=60
Oct 30 19:30:17.176195 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:17.176083    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889810-wxpvs" event={"ID":"ac050065-1900-4689-a024-2d499fc6e145","Type":"ContainerDied","Data":"ebdfe765de92d77be3827d8e367d42e443b7b5685c1b2562322c6bdcfada9131"}
Oct 30 19:30:17.176401 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:17.176386    4888 scope.go:122] "RemoveContainer" containerID="ebdfe765de92d77be3827d8e367d42e443b7b5685c1b2562322c6bdcfada9131"
Oct 30 19:30:17.176643 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:30:17.176625    4888 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-29889810-wxpvs_odh-ai-gateway-infra(ac050065-1900-4689-a024-2d499fc6e145)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889810-wxpvs" podUID="ac050065-1900-4689-a024-2d499fc6e145"
Oct 30 19:30:17.185839 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:17.185814    4888 scope.go:122] "RemoveContainer" containerID="f02a6f8bbd37076132d2972a7fed3dd1ca75118043e1ea6bf65821a0f0e5a727"
Oct 30 19:30:17.289913 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:17.289884    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-2ade09-29889810-fz97s"]
Oct 30 19:30:17.416988 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:17.416959    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-8f9c6b-29889810-bcs5p"]
Oct 30 19:30:17.448223 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:17.448199    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-2ade09-29889810-fz97s"
Oct 30 19:30:17.509194 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:17.509169    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/526209fb-5947-47b1-81e2-218d65c87301-kube-api-access-dxg76\" (UniqueName: \"kubernetes.io/projected/526209fb-5947-47b1-81e2-218d65c87301-kube-api-access-dxg76\") pod \"526209fb-5947-47b1-81e2-218d65c87301\" (UID: \"526209fb-5947-47b1-81e2-218d65c87301\") "
Oct 30 19:30:17.509396 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:17.509362    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/526209fb-5947-47b1-81e2-218d65c87301-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/526209fb-5947-47b1-81e2-218d65c87301-maas-api-service-ca\") pod \"526209fb-5947-47b1-81e2-218d65c87301\" (UID: \"526209fb-5947-47b1-81e2-218d65c87301\") "
Oct 30 19:30:17.509745 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:17.509723    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/526209fb-5947-47b1-81e2-218d65c87301-maas-api-service-ca" pod "526209fb-5947-47b1-81e2-218d65c87301" (UID: "526209fb-5947-47b1-81e2-218d65c87301"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:30:17.509861 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:17.509846    4888 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/526209fb-5947-47b1-81e2-218d65c87301-maas-api-service-ca\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:30:17.512089 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:17.512062    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/526209fb-5947-47b1-81e2-218d65c87301-kube-api-access-dxg76" pod "526209fb-5947-47b1-81e2-218d65c87301" (UID: "526209fb-5947-47b1-81e2-218d65c87301"). InnerVolumeSpecName "kube-api-access-dxg76". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:30:17.572109 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:17.572086    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-8f9c6b-29889810-bcs5p"
Oct 30 19:30:17.610662 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:17.610633    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/120337e2-9322-456b-ab8e-a29e285b8bba-kube-api-access-dd2gg\" (UniqueName: \"kubernetes.io/projected/120337e2-9322-456b-ab8e-a29e285b8bba-kube-api-access-dd2gg\") pod \"120337e2-9322-456b-ab8e-a29e285b8bba\" (UID: \"120337e2-9322-456b-ab8e-a29e285b8bba\") "
Oct 30 19:30:17.610796 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:17.610781    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/120337e2-9322-456b-ab8e-a29e285b8bba-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/120337e2-9322-456b-ab8e-a29e285b8bba-maas-api-service-ca\") pod \"120337e2-9322-456b-ab8e-a29e285b8bba\" (UID: \"120337e2-9322-456b-ab8e-a29e285b8bba\") "
Oct 30 19:30:17.611132 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:17.611111    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/120337e2-9322-456b-ab8e-a29e285b8bba-maas-api-service-ca" pod "120337e2-9322-456b-ab8e-a29e285b8bba" (UID: "120337e2-9322-456b-ab8e-a29e285b8bba"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:30:17.611195 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:17.611180    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dxg76\" (UniqueName: \"kubernetes.io/projected/526209fb-5947-47b1-81e2-218d65c87301-kube-api-access-dxg76\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:30:17.612953 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:17.612931    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/120337e2-9322-456b-ab8e-a29e285b8bba-kube-api-access-dd2gg" pod "120337e2-9322-456b-ab8e-a29e285b8bba" (UID: "120337e2-9322-456b-ab8e-a29e285b8bba"). InnerVolumeSpecName "kube-api-access-dd2gg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:30:17.712016 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:17.711946    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dd2gg\" (UniqueName: \"kubernetes.io/projected/120337e2-9322-456b-ab8e-a29e285b8bba-kube-api-access-dd2gg\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:30:17.712016 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:17.711969    4888 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/120337e2-9322-456b-ab8e-a29e285b8bba-maas-api-service-ca\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:30:17.883343 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:17.883314    4888 scope.go:122] "RemoveContainer" containerID="10a6b2a7fbec4879d39451b10d6a54c4603cca4a7659f89c503044944a5d1e1b"
Oct 30 19:30:18.206196 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:18.206175    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-8f9c6b-29889810-bcs5p"
Oct 30 19:30:18.206656 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:18.206209    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-8f9c6b-29889810-bcs5p" event={"ID":"120337e2-9322-456b-ab8e-a29e285b8bba","Type":"ContainerDied","Data":"8b39d849a25392c3f4e9e45ec02831fdc6c5d3c2934756bf3f1f6a345263fbcd"}
Oct 30 19:30:18.206656 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:18.206261    4888 scope.go:122] "RemoveContainer" containerID="e695321ff4bbb9b028cc1dbcb6e9a8088d585f68820f303b50c495a5478600b3"
Oct 30 19:30:18.211320 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:18.211303    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-2ade09-29889810-fz97s"
Oct 30 19:30:18.211320 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:18.211311    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-2ade09-29889810-fz97s" event={"ID":"526209fb-5947-47b1-81e2-218d65c87301","Type":"ContainerDied","Data":"513b0e0cdb1166e377770a44a2fad6c116469ca49fef52445c00a534e0d07162"}
Oct 30 19:30:18.217907 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:18.217892    4888 scope.go:122] "RemoveContainer" containerID="5ab88bddcf3ef46234ca59e426900dab8b12b263463f779623e0d4162ee52af5"
Oct 30 19:30:18.305848 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:18.305812    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-fd61e3-29889810-gkhfl"]
Oct 30 19:30:18.380389 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:18.380351    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-8f9c6b-29889810-bcs5p"]
Oct 30 19:30:18.382587 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:18.382549    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-8f9c6b-29889810-bcs5p"]
Oct 30 19:30:18.458408 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:18.458382    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-fd61e3-29889810-gkhfl"
Oct 30 19:30:18.494216 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:18.494191    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-2ade09-29889810-fz97s"]
Oct 30 19:30:18.501045 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:18.501024    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-2ade09-29889810-fz97s"]
Oct 30 19:30:18.523230 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:18.523205    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/733efe29-d46f-44e7-883f-0609c99e4531-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/733efe29-d46f-44e7-883f-0609c99e4531-maas-api-service-ca\") pod \"733efe29-d46f-44e7-883f-0609c99e4531\" (UID: \"733efe29-d46f-44e7-883f-0609c99e4531\") "
Oct 30 19:30:18.523352 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:18.523242    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/733efe29-d46f-44e7-883f-0609c99e4531-kube-api-access-hhh29\" (UniqueName: \"kubernetes.io/projected/733efe29-d46f-44e7-883f-0609c99e4531-kube-api-access-hhh29\") pod \"733efe29-d46f-44e7-883f-0609c99e4531\" (UID: \"733efe29-d46f-44e7-883f-0609c99e4531\") "
Oct 30 19:30:18.523630 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:18.523607    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/733efe29-d46f-44e7-883f-0609c99e4531-maas-api-service-ca" pod "733efe29-d46f-44e7-883f-0609c99e4531" (UID: "733efe29-d46f-44e7-883f-0609c99e4531"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:30:18.523769 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:18.523756    4888 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/733efe29-d46f-44e7-883f-0609c99e4531-maas-api-service-ca\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:30:18.525574 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:18.525557    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/733efe29-d46f-44e7-883f-0609c99e4531-kube-api-access-hhh29" pod "733efe29-d46f-44e7-883f-0609c99e4531" (UID: "733efe29-d46f-44e7-883f-0609c99e4531"). InnerVolumeSpecName "kube-api-access-hhh29". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:30:18.539759 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:18.539737    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889810-wxpvs"]
Oct 30 19:30:18.627751 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:18.627679    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hhh29\" (UniqueName: \"kubernetes.io/projected/733efe29-d46f-44e7-883f-0609c99e4531-kube-api-access-hhh29\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:30:18.691756 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:18.691734    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889810-wxpvs"
Oct 30 19:30:18.728468 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:18.728445    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ac050065-1900-4689-a024-2d499fc6e145-kube-api-access-6jnz8\" (UniqueName: \"kubernetes.io/projected/ac050065-1900-4689-a024-2d499fc6e145-kube-api-access-6jnz8\") pod \"ac050065-1900-4689-a024-2d499fc6e145\" (UID: \"ac050065-1900-4689-a024-2d499fc6e145\") "
Oct 30 19:30:18.728673 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:18.728658    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ac050065-1900-4689-a024-2d499fc6e145-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ac050065-1900-4689-a024-2d499fc6e145-maas-api-service-ca\") pod \"ac050065-1900-4689-a024-2d499fc6e145\" (UID: \"ac050065-1900-4689-a024-2d499fc6e145\") "
Oct 30 19:30:18.729000 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:18.728981    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ac050065-1900-4689-a024-2d499fc6e145-maas-api-service-ca" pod "ac050065-1900-4689-a024-2d499fc6e145" (UID: "ac050065-1900-4689-a024-2d499fc6e145"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:30:18.730632 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:18.730612    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ac050065-1900-4689-a024-2d499fc6e145-kube-api-access-6jnz8" pod "ac050065-1900-4689-a024-2d499fc6e145" (UID: "ac050065-1900-4689-a024-2d499fc6e145"). InnerVolumeSpecName "kube-api-access-6jnz8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:30:18.830217 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:18.830190    4888 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ac050065-1900-4689-a024-2d499fc6e145-maas-api-service-ca\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:30:18.830217 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:18.830216    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6jnz8\" (UniqueName: \"kubernetes.io/projected/ac050065-1900-4689-a024-2d499fc6e145-kube-api-access-6jnz8\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:30:19.222696 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:19.222669    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889810-wxpvs"
Oct 30 19:30:19.223216 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:19.222705    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889810-wxpvs" event={"ID":"ac050065-1900-4689-a024-2d499fc6e145","Type":"ContainerDied","Data":"26e893e6aa7bbdefffc55b04a21de5310c7227c83ea923a1317fd2c35c604079"}
Oct 30 19:30:19.223216 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:19.222776    4888 scope.go:122] "RemoveContainer" containerID="ebdfe765de92d77be3827d8e367d42e443b7b5685c1b2562322c6bdcfada9131"
Oct 30 19:30:19.224452 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:19.224432    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-fd61e3-29889810-gkhfl"
Oct 30 19:30:19.224587 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:19.224473    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-fd61e3-29889810-gkhfl" event={"ID":"733efe29-d46f-44e7-883f-0609c99e4531","Type":"ContainerDied","Data":"701d779220d12fcdaf273c6b66110c173f555d40c27bb8fe131f1b7d9b4b8200"}
Oct 30 19:30:19.231890 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:19.231860    4888 generic.go:356] "Generic (PLEG): container finished" podID="1354e989-63a9-423d-947c-6a1f3e38162d" containerID="3ac8f1a9a932bc9bbe3b25fb30215f47b8dce7f0ca4018fa3162942b2d388536" exitCode=60
Oct 30 19:30:19.232051 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:19.231978    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-explicit-200c46-29889810-hvmcj" event={"ID":"1354e989-63a9-423d-947c-6a1f3e38162d","Type":"ContainerDied","Data":"3ac8f1a9a932bc9bbe3b25fb30215f47b8dce7f0ca4018fa3162942b2d388536"}
Oct 30 19:30:19.232502 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:19.232477    4888 scope.go:122] "RemoveContainer" containerID="3ac8f1a9a932bc9bbe3b25fb30215f47b8dce7f0ca4018fa3162942b2d388536"
Oct 30 19:30:19.232870 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:30:19.232843    4888 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-explicit-200c46-29889810-hvmcj_odh-ai-gateway-infra(1354e989-63a9-423d-947c-6a1f3e38162d)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-explicit-200c46-29889810-hvmcj" podUID="1354e989-63a9-423d-947c-6a1f3e38162d"
Oct 30 19:30:19.238269 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:19.238247    4888 scope.go:122] "RemoveContainer" containerID="f285880fb8b841dba86e10d0033a44c0a76d651d183a74fe775bf01726db7d1d"
Oct 30 19:30:19.251990 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:19.251965    4888 scope.go:122] "RemoveContainer" containerID="10a6b2a7fbec4879d39451b10d6a54c4603cca4a7659f89c503044944a5d1e1b"
Oct 30 19:30:19.393044 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:19.393010    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889810-wxpvs"]
Oct 30 19:30:19.398355 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:19.398328    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889810-wxpvs"]
Oct 30 19:30:19.510268 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:19.510193    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-fd61e3-29889810-gkhfl"]
Oct 30 19:30:19.514957 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:19.514934    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-fd61e3-29889810-gkhfl"]
Oct 30 19:30:19.897959 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:19.897919    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="120337e2-9322-456b-ab8e-a29e285b8bba" path="/var/lib/kubelet/pods/120337e2-9322-456b-ab8e-a29e285b8bba/volumes"
Oct 30 19:30:19.898627 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:19.898600    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="526209fb-5947-47b1-81e2-218d65c87301" path="/var/lib/kubelet/pods/526209fb-5947-47b1-81e2-218d65c87301/volumes"
Oct 30 19:30:19.899161 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:19.899143    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="733efe29-d46f-44e7-883f-0609c99e4531" path="/var/lib/kubelet/pods/733efe29-d46f-44e7-883f-0609c99e4531/volumes"
Oct 30 19:30:19.900136 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:19.900115    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ac050065-1900-4689-a024-2d499fc6e145" path="/var/lib/kubelet/pods/ac050065-1900-4689-a024-2d499fc6e145/volumes"
Oct 30 19:30:20.642411 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:20.642378    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-explicit-200c46-29889810-hvmcj"]
Oct 30 19:30:20.849603 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:20.849574    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-explicit-200c46-29889810-hvmcj"
Oct 30 19:30:20.955267 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:20.955181    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1354e989-63a9-423d-947c-6a1f3e38162d-kube-api-access-mmbb5\" (UniqueName: \"kubernetes.io/projected/1354e989-63a9-423d-947c-6a1f3e38162d-kube-api-access-mmbb5\") pod \"1354e989-63a9-423d-947c-6a1f3e38162d\" (UID: \"1354e989-63a9-423d-947c-6a1f3e38162d\") "
Oct 30 19:30:20.955267 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:20.955221    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1354e989-63a9-423d-947c-6a1f3e38162d-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1354e989-63a9-423d-947c-6a1f3e38162d-maas-api-service-ca\") pod \"1354e989-63a9-423d-947c-6a1f3e38162d\" (UID: \"1354e989-63a9-423d-947c-6a1f3e38162d\") "
Oct 30 19:30:20.955656 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:20.955631    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1354e989-63a9-423d-947c-6a1f3e38162d-maas-api-service-ca" pod "1354e989-63a9-423d-947c-6a1f3e38162d" (UID: "1354e989-63a9-423d-947c-6a1f3e38162d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:30:20.955822 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:20.955806    4888 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1354e989-63a9-423d-947c-6a1f3e38162d-maas-api-service-ca\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:30:20.957582 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:20.957561    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1354e989-63a9-423d-947c-6a1f3e38162d-kube-api-access-mmbb5" pod "1354e989-63a9-423d-947c-6a1f3e38162d" (UID: "1354e989-63a9-423d-947c-6a1f3e38162d"). InnerVolumeSpecName "kube-api-access-mmbb5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:30:21.057626 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:21.057591    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mmbb5\" (UniqueName: \"kubernetes.io/projected/1354e989-63a9-423d-947c-6a1f3e38162d-kube-api-access-mmbb5\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:30:21.259095 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:21.259023    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-explicit-200c46-29889810-hvmcj"
Oct 30 19:30:21.259095 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:21.259039    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-explicit-200c46-29889810-hvmcj" event={"ID":"1354e989-63a9-423d-947c-6a1f3e38162d","Type":"ContainerDied","Data":"28975159ae212d57701e54a77e1577eb0d82795fa91fbf0273a39f3e53a857c1"}
Oct 30 19:30:21.259349 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:21.259099    4888 scope.go:122] "RemoveContainer" containerID="3ac8f1a9a932bc9bbe3b25fb30215f47b8dce7f0ca4018fa3162942b2d388536"
Oct 30 19:30:21.431687 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:21.431623    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-explicit-200c46-29889810-hvmcj"]
Oct 30 19:30:21.434716 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:21.434691    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-explicit-200c46-29889810-hvmcj"]
Oct 30 19:30:21.893557 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:21.893521    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1354e989-63a9-423d-947c-6a1f3e38162d" path="/var/lib/kubelet/pods/1354e989-63a9-423d-947c-6a1f3e38162d/volumes"
Oct 30 19:30:30.147123 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:30.147064    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w2-dc5cdd/auth-test-model-dc5cdd-kserve-7cb964d99c-9znzj" podUID="f0b4d9fd-a0dd-4fb3-8607-c0af3fd3d030" containerName="main" containerID="cri-o://65662cd374620364883ab08b496a5ab1ba5a37b631a58a04c863f7a69dcc9644" gracePeriod=45
Oct 30 19:30:30.168697 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:30.168671    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-praxis-b-f5dfb8-openshift-default-d59df9d89-28vsl"]
Oct 30 19:30:30.168923 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:30.168883    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-praxis-b-f5dfb8-openshift-default-d59df9d89-28vsl" podUID="b1598dab-80ac-408c-b65a-bc3cde3b01d5" containerName="istio-proxy" containerID="cri-o://9c0edb68221716f42b960b076aa36b86f73e97f779f65368ee89418000a387f3" gracePeriod=30
Oct 30 19:30:30.334313 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:30.334252    4888 generic.go:356] "Generic (PLEG): container finished" podID="f0b4d9fd-a0dd-4fb3-8607-c0af3fd3d030" containerID="65662cd374620364883ab08b496a5ab1ba5a37b631a58a04c863f7a69dcc9644" exitCode=0
Oct 30 19:30:30.334490 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:30.334319    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w2-dc5cdd/auth-test-model-dc5cdd-kserve-7cb964d99c-9znzj" event={"ID":"f0b4d9fd-a0dd-4fb3-8607-c0af3fd3d030","Type":"ContainerDied","Data":"65662cd374620364883ab08b496a5ab1ba5a37b631a58a04c863f7a69dcc9644"}
Oct 30 19:30:30.448632 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:30.448608    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w2-dc5cdd/auth-test-model-dc5cdd-kserve-7cb964d99c-9znzj"
Oct 30 19:30:30.585959 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:30.585931    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f0b4d9fd-a0dd-4fb3-8607-c0af3fd3d030-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f0b4d9fd-a0dd-4fb3-8607-c0af3fd3d030-model-cache\") pod \"f0b4d9fd-a0dd-4fb3-8607-c0af3fd3d030\" (UID: \"f0b4d9fd-a0dd-4fb3-8607-c0af3fd3d030\") "
Oct 30 19:30:30.586143 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:30.586048    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f0b4d9fd-a0dd-4fb3-8607-c0af3fd3d030-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f0b4d9fd-a0dd-4fb3-8607-c0af3fd3d030-tmp-dir\") pod \"f0b4d9fd-a0dd-4fb3-8607-c0af3fd3d030\" (UID: \"f0b4d9fd-a0dd-4fb3-8607-c0af3fd3d030\") "
Oct 30 19:30:30.586143 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:30.586121    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f0b4d9fd-a0dd-4fb3-8607-c0af3fd3d030-home\" (UniqueName: \"kubernetes.io/empty-dir/f0b4d9fd-a0dd-4fb3-8607-c0af3fd3d030-home\") pod \"f0b4d9fd-a0dd-4fb3-8607-c0af3fd3d030\" (UID: \"f0b4d9fd-a0dd-4fb3-8607-c0af3fd3d030\") "
Oct 30 19:30:30.586275 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:30.586206    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0b4d9fd-a0dd-4fb3-8607-c0af3fd3d030-model-cache" pod "f0b4d9fd-a0dd-4fb3-8607-c0af3fd3d030" (UID: "f0b4d9fd-a0dd-4fb3-8607-c0af3fd3d030"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:30:30.586275 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:30.586244    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f0b4d9fd-a0dd-4fb3-8607-c0af3fd3d030-tls-certs\" (UniqueName: \"kubernetes.io/secret/f0b4d9fd-a0dd-4fb3-8607-c0af3fd3d030-tls-certs\") pod \"f0b4d9fd-a0dd-4fb3-8607-c0af3fd3d030\" (UID: \"f0b4d9fd-a0dd-4fb3-8607-c0af3fd3d030\") "
Oct 30 19:30:30.586406 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:30.586275    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f0b4d9fd-a0dd-4fb3-8607-c0af3fd3d030-kube-api-access-twg8l\" (UniqueName: \"kubernetes.io/projected/f0b4d9fd-a0dd-4fb3-8607-c0af3fd3d030-kube-api-access-twg8l\") pod \"f0b4d9fd-a0dd-4fb3-8607-c0af3fd3d030\" (UID: \"f0b4d9fd-a0dd-4fb3-8607-c0af3fd3d030\") "
Oct 30 19:30:30.586406 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:30.586345    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f0b4d9fd-a0dd-4fb3-8607-c0af3fd3d030-dshm\" (UniqueName: \"kubernetes.io/empty-dir/f0b4d9fd-a0dd-4fb3-8607-c0af3fd3d030-dshm\") pod \"f0b4d9fd-a0dd-4fb3-8607-c0af3fd3d030\" (UID: \"f0b4d9fd-a0dd-4fb3-8607-c0af3fd3d030\") "
Oct 30 19:30:30.586519 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:30.586372    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0b4d9fd-a0dd-4fb3-8607-c0af3fd3d030-tmp-dir" pod "f0b4d9fd-a0dd-4fb3-8607-c0af3fd3d030" (UID: "f0b4d9fd-a0dd-4fb3-8607-c0af3fd3d030"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:30:30.586519 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:30.586444    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0b4d9fd-a0dd-4fb3-8607-c0af3fd3d030-home" pod "f0b4d9fd-a0dd-4fb3-8607-c0af3fd3d030" (UID: "f0b4d9fd-a0dd-4fb3-8607-c0af3fd3d030"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:30:30.586851 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:30.586828    4888 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f0b4d9fd-a0dd-4fb3-8607-c0af3fd3d030-model-cache\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:30:30.586949 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:30.586857    4888 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f0b4d9fd-a0dd-4fb3-8607-c0af3fd3d030-tmp-dir\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:30:30.586949 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:30.586872    4888 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f0b4d9fd-a0dd-4fb3-8607-c0af3fd3d030-home\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:30:30.589393 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:30.589371    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f0b4d9fd-a0dd-4fb3-8607-c0af3fd3d030-kube-api-access-twg8l" pod "f0b4d9fd-a0dd-4fb3-8607-c0af3fd3d030" (UID: "f0b4d9fd-a0dd-4fb3-8607-c0af3fd3d030"). InnerVolumeSpecName "kube-api-access-twg8l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:30:30.589393 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:30.589381    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f0b4d9fd-a0dd-4fb3-8607-c0af3fd3d030-tls-certs" pod "f0b4d9fd-a0dd-4fb3-8607-c0af3fd3d030" (UID: "f0b4d9fd-a0dd-4fb3-8607-c0af3fd3d030"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:30:30.590385 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:30.590367    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0b4d9fd-a0dd-4fb3-8607-c0af3fd3d030-dshm" pod "f0b4d9fd-a0dd-4fb3-8607-c0af3fd3d030" (UID: "f0b4d9fd-a0dd-4fb3-8607-c0af3fd3d030"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:30:30.687967 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:30.687900    4888 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f0b4d9fd-a0dd-4fb3-8607-c0af3fd3d030-tls-certs\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:30:30.687967 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:30.687925    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-twg8l\" (UniqueName: \"kubernetes.io/projected/f0b4d9fd-a0dd-4fb3-8607-c0af3fd3d030-kube-api-access-twg8l\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:30:30.687967 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:30.687937    4888 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f0b4d9fd-a0dd-4fb3-8607-c0af3fd3d030-dshm\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:30:31.344680 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:31.344655    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w2-dc5cdd/auth-test-model-dc5cdd-kserve-7cb964d99c-9znzj"
Oct 30 19:30:31.345120 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:31.344653    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w2-dc5cdd/auth-test-model-dc5cdd-kserve-7cb964d99c-9znzj" event={"ID":"f0b4d9fd-a0dd-4fb3-8607-c0af3fd3d030","Type":"ContainerDied","Data":"35fd9081ae751913c2d93eafcb729be7e61bbc9155ebd466d5dc002404f3e771"}
Oct 30 19:30:31.345120 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:31.344778    4888 scope.go:122] "RemoveContainer" containerID="65662cd374620364883ab08b496a5ab1ba5a37b631a58a04c863f7a69dcc9644"
Oct 30 19:30:31.514745 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:31.514713    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w2-dc5cdd/auth-test-model-dc5cdd-kserve-7cb964d99c-9znzj"]
Oct 30 19:30:31.575338 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:31.567799    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w2-dc5cdd/auth-test-model-dc5cdd-kserve-7cb964d99c-9znzj"]
Oct 30 19:30:31.835882 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:31.835847    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w4-da1b8b-openshift-default-7876fcf7c5-p4wc5"]
Oct 30 19:30:31.836485 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:31.836464    4888 cpu_manager.go:414] "containerMap: removing container" podUID="733efe29-d46f-44e7-883f-0609c99e4531" containerName="cleanup"
Oct 30 19:30:31.836485 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:31.836483    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="733efe29-d46f-44e7-883f-0609c99e4531" containerName="cleanup"
Oct 30 19:30:31.836672 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:31.836492    4888 cpu_manager.go:414] "containerMap: removing container" podUID="ac050065-1900-4689-a024-2d499fc6e145" containerName="cleanup"
Oct 30 19:30:31.836672 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:31.836498    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ac050065-1900-4689-a024-2d499fc6e145" containerName="cleanup"
Oct 30 19:30:31.836672 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:31.836506    4888 cpu_manager.go:414] "containerMap: removing container" podUID="733efe29-d46f-44e7-883f-0609c99e4531" containerName="cleanup"
Oct 30 19:30:31.836672 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:31.836511    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="733efe29-d46f-44e7-883f-0609c99e4531" containerName="cleanup"
Oct 30 19:30:31.836672 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:31.836528    4888 cpu_manager.go:414] "containerMap: removing container" podUID="ac050065-1900-4689-a024-2d499fc6e145" containerName="cleanup"
Oct 30 19:30:31.836672 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:31.836533    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ac050065-1900-4689-a024-2d499fc6e145" containerName="cleanup"
Oct 30 19:30:31.836672 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:31.836541    4888 cpu_manager.go:414] "containerMap: removing container" podUID="1354e989-63a9-423d-947c-6a1f3e38162d" containerName="cleanup"
Oct 30 19:30:31.836672 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:31.836546    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1354e989-63a9-423d-947c-6a1f3e38162d" containerName="cleanup"
Oct 30 19:30:31.836672 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:31.836555    4888 cpu_manager.go:414] "containerMap: removing container" podUID="f0b4d9fd-a0dd-4fb3-8607-c0af3fd3d030" containerName="main"
Oct 30 19:30:31.836672 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:31.836559    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f0b4d9fd-a0dd-4fb3-8607-c0af3fd3d030" containerName="main"
Oct 30 19:30:31.836672 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:31.836566    4888 cpu_manager.go:414] "containerMap: removing container" podUID="120337e2-9322-456b-ab8e-a29e285b8bba" containerName="cleanup"
Oct 30 19:30:31.836672 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:31.836570    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="120337e2-9322-456b-ab8e-a29e285b8bba" containerName="cleanup"
Oct 30 19:30:31.836672 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:31.836576    4888 cpu_manager.go:414] "containerMap: removing container" podUID="526209fb-5947-47b1-81e2-218d65c87301" containerName="cleanup"
Oct 30 19:30:31.836672 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:31.836581    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="526209fb-5947-47b1-81e2-218d65c87301" containerName="cleanup"
Oct 30 19:30:31.836672 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:31.836591    4888 cpu_manager.go:414] "containerMap: removing container" podUID="ac050065-1900-4689-a024-2d499fc6e145" containerName="cleanup"
Oct 30 19:30:31.836672 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:31.836595    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ac050065-1900-4689-a024-2d499fc6e145" containerName="cleanup"
Oct 30 19:30:31.836672 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:31.836602    4888 cpu_manager.go:414] "containerMap: removing container" podUID="733efe29-d46f-44e7-883f-0609c99e4531" containerName="cleanup"
Oct 30 19:30:31.836672 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:31.836606    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="733efe29-d46f-44e7-883f-0609c99e4531" containerName="cleanup"
Oct 30 19:30:31.836672 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:31.836615    4888 cpu_manager.go:414] "containerMap: removing container" podUID="1354e989-63a9-423d-947c-6a1f3e38162d" containerName="cleanup"
Oct 30 19:30:31.836672 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:31.836619    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1354e989-63a9-423d-947c-6a1f3e38162d" containerName="cleanup"
Oct 30 19:30:31.836672 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:31.836630    4888 cpu_manager.go:414] "containerMap: removing container" podUID="1354e989-63a9-423d-947c-6a1f3e38162d" containerName="cleanup"
Oct 30 19:30:31.836672 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:31.836634    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1354e989-63a9-423d-947c-6a1f3e38162d" containerName="cleanup"
Oct 30 19:30:31.836672 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:31.836649    4888 cpu_manager.go:414] "containerMap: removing container" podUID="120337e2-9322-456b-ab8e-a29e285b8bba" containerName="cleanup"
Oct 30 19:30:31.836672 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:31.836653    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="120337e2-9322-456b-ab8e-a29e285b8bba" containerName="cleanup"
Oct 30 19:30:31.836672 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:31.836659    4888 cpu_manager.go:414] "containerMap: removing container" podUID="120337e2-9322-456b-ab8e-a29e285b8bba" containerName="cleanup"
Oct 30 19:30:31.836672 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:31.836663    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="120337e2-9322-456b-ab8e-a29e285b8bba" containerName="cleanup"
Oct 30 19:30:31.836672 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:31.836681    4888 cpu_manager.go:414] "containerMap: removing container" podUID="526209fb-5947-47b1-81e2-218d65c87301" containerName="cleanup"
Oct 30 19:30:31.836672 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:31.836685    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="526209fb-5947-47b1-81e2-218d65c87301" containerName="cleanup"
Oct 30 19:30:31.837551 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:31.836771    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="120337e2-9322-456b-ab8e-a29e285b8bba" containerName="cleanup"
Oct 30 19:30:31.837551 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:31.836777    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="120337e2-9322-456b-ab8e-a29e285b8bba" containerName="cleanup"
Oct 30 19:30:31.837551 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:31.836785    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="526209fb-5947-47b1-81e2-218d65c87301" containerName="cleanup"
Oct 30 19:30:31.837551 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:31.836791    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="733efe29-d46f-44e7-883f-0609c99e4531" containerName="cleanup"
Oct 30 19:30:31.837551 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:31.836796    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="ac050065-1900-4689-a024-2d499fc6e145" containerName="cleanup"
Oct 30 19:30:31.837551 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:31.836801    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="733efe29-d46f-44e7-883f-0609c99e4531" containerName="cleanup"
Oct 30 19:30:31.837551 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:31.836807    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="ac050065-1900-4689-a024-2d499fc6e145" containerName="cleanup"
Oct 30 19:30:31.837551 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:31.836813    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="1354e989-63a9-423d-947c-6a1f3e38162d" containerName="cleanup"
Oct 30 19:30:31.837551 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:31.836820    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="f0b4d9fd-a0dd-4fb3-8607-c0af3fd3d030" containerName="main"
Oct 30 19:30:31.837551 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:31.836829    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="ac050065-1900-4689-a024-2d499fc6e145" containerName="cleanup"
Oct 30 19:30:31.837551 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:31.836837    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="1354e989-63a9-423d-947c-6a1f3e38162d" containerName="cleanup"
Oct 30 19:30:31.837551 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:31.836844    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="526209fb-5947-47b1-81e2-218d65c87301" containerName="cleanup"
Oct 30 19:30:31.837551 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:31.836849    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="1354e989-63a9-423d-947c-6a1f3e38162d" containerName="cleanup"
Oct 30 19:30:31.841601 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:31.841583    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w4-da1b8b-openshift-default-7876fcf7c5-p4wc5"
Oct 30 19:30:31.893140 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:31.893105    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f0b4d9fd-a0dd-4fb3-8607-c0af3fd3d030" path="/var/lib/kubelet/pods/f0b4d9fd-a0dd-4fb3-8607-c0af3fd3d030/volumes"
Oct 30 19:30:31.915866 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:31.915836    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w4-da1b8b-openshift-default-7876fcf7c5-p4wc5"]
Oct 30 19:30:31.946895 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:31.946869    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w4-da1b8b-openshift-default-dockercfg-h84dl\""
Oct 30 19:30:32.003928 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:32.003898    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/73a3358c-ef67-4b35-a010-46c168a95f1e-istio-token\") pod \"e2e-shared-a-w4-da1b8b-openshift-default-7876fcf7c5-p4wc5\" (UID: \"73a3358c-ef67-4b35-a010-46c168a95f1e\") " pod="openshift-ingress/e2e-shared-a-w4-da1b8b-openshift-default-7876fcf7c5-p4wc5"
Oct 30 19:30:32.004105 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:32.003942    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/73a3358c-ef67-4b35-a010-46c168a95f1e-credential-socket\") pod \"e2e-shared-a-w4-da1b8b-openshift-default-7876fcf7c5-p4wc5\" (UID: \"73a3358c-ef67-4b35-a010-46c168a95f1e\") " pod="openshift-ingress/e2e-shared-a-w4-da1b8b-openshift-default-7876fcf7c5-p4wc5"
Oct 30 19:30:32.004155 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:32.004124    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/73a3358c-ef67-4b35-a010-46c168a95f1e-istiod-ca-cert\") pod \"e2e-shared-a-w4-da1b8b-openshift-default-7876fcf7c5-p4wc5\" (UID: \"73a3358c-ef67-4b35-a010-46c168a95f1e\") " pod="openshift-ingress/e2e-shared-a-w4-da1b8b-openshift-default-7876fcf7c5-p4wc5"
Oct 30 19:30:32.004199 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:32.004177    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/73a3358c-ef67-4b35-a010-46c168a95f1e-workload-certs\") pod \"e2e-shared-a-w4-da1b8b-openshift-default-7876fcf7c5-p4wc5\" (UID: \"73a3358c-ef67-4b35-a010-46c168a95f1e\") " pod="openshift-ingress/e2e-shared-a-w4-da1b8b-openshift-default-7876fcf7c5-p4wc5"
Oct 30 19:30:32.004254 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:32.004220    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/73a3358c-ef67-4b35-a010-46c168a95f1e-istio-podinfo\") pod \"e2e-shared-a-w4-da1b8b-openshift-default-7876fcf7c5-p4wc5\" (UID: \"73a3358c-ef67-4b35-a010-46c168a95f1e\") " pod="openshift-ingress/e2e-shared-a-w4-da1b8b-openshift-default-7876fcf7c5-p4wc5"
Oct 30 19:30:32.004357 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:32.004325    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/73a3358c-ef67-4b35-a010-46c168a95f1e-istio-data\") pod \"e2e-shared-a-w4-da1b8b-openshift-default-7876fcf7c5-p4wc5\" (UID: \"73a3358c-ef67-4b35-a010-46c168a95f1e\") " pod="openshift-ingress/e2e-shared-a-w4-da1b8b-openshift-default-7876fcf7c5-p4wc5"
Oct 30 19:30:32.004492 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:32.004467    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/73a3358c-ef67-4b35-a010-46c168a95f1e-istio-envoy\") pod \"e2e-shared-a-w4-da1b8b-openshift-default-7876fcf7c5-p4wc5\" (UID: \"73a3358c-ef67-4b35-a010-46c168a95f1e\") " pod="openshift-ingress/e2e-shared-a-w4-da1b8b-openshift-default-7876fcf7c5-p4wc5"
Oct 30 19:30:32.004624 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:32.004533    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/73a3358c-ef67-4b35-a010-46c168a95f1e-workload-socket\") pod \"e2e-shared-a-w4-da1b8b-openshift-default-7876fcf7c5-p4wc5\" (UID: \"73a3358c-ef67-4b35-a010-46c168a95f1e\") " pod="openshift-ingress/e2e-shared-a-w4-da1b8b-openshift-default-7876fcf7c5-p4wc5"
Oct 30 19:30:32.004688 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:32.004645    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ww4n4\" (UniqueName: \"kubernetes.io/projected/73a3358c-ef67-4b35-a010-46c168a95f1e-kube-api-access-ww4n4\") pod \"e2e-shared-a-w4-da1b8b-openshift-default-7876fcf7c5-p4wc5\" (UID: \"73a3358c-ef67-4b35-a010-46c168a95f1e\") " pod="openshift-ingress/e2e-shared-a-w4-da1b8b-openshift-default-7876fcf7c5-p4wc5"
Oct 30 19:30:32.106401 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:32.106313    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/73a3358c-ef67-4b35-a010-46c168a95f1e-istio-podinfo\") pod \"e2e-shared-a-w4-da1b8b-openshift-default-7876fcf7c5-p4wc5\" (UID: \"73a3358c-ef67-4b35-a010-46c168a95f1e\") " pod="openshift-ingress/e2e-shared-a-w4-da1b8b-openshift-default-7876fcf7c5-p4wc5"
Oct 30 19:30:32.106401 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:32.106379    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/73a3358c-ef67-4b35-a010-46c168a95f1e-istio-data\") pod \"e2e-shared-a-w4-da1b8b-openshift-default-7876fcf7c5-p4wc5\" (UID: \"73a3358c-ef67-4b35-a010-46c168a95f1e\") " pod="openshift-ingress/e2e-shared-a-w4-da1b8b-openshift-default-7876fcf7c5-p4wc5"
Oct 30 19:30:32.106648 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:32.106485    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/73a3358c-ef67-4b35-a010-46c168a95f1e-istio-envoy\") pod \"e2e-shared-a-w4-da1b8b-openshift-default-7876fcf7c5-p4wc5\" (UID: \"73a3358c-ef67-4b35-a010-46c168a95f1e\") " pod="openshift-ingress/e2e-shared-a-w4-da1b8b-openshift-default-7876fcf7c5-p4wc5"
Oct 30 19:30:32.106648 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:32.106522    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/73a3358c-ef67-4b35-a010-46c168a95f1e-workload-socket\") pod \"e2e-shared-a-w4-da1b8b-openshift-default-7876fcf7c5-p4wc5\" (UID: \"73a3358c-ef67-4b35-a010-46c168a95f1e\") " pod="openshift-ingress/e2e-shared-a-w4-da1b8b-openshift-default-7876fcf7c5-p4wc5"
Oct 30 19:30:32.106648 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:32.106581    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ww4n4\" (UniqueName: \"kubernetes.io/projected/73a3358c-ef67-4b35-a010-46c168a95f1e-kube-api-access-ww4n4\") pod \"e2e-shared-a-w4-da1b8b-openshift-default-7876fcf7c5-p4wc5\" (UID: \"73a3358c-ef67-4b35-a010-46c168a95f1e\") " pod="openshift-ingress/e2e-shared-a-w4-da1b8b-openshift-default-7876fcf7c5-p4wc5"
Oct 30 19:30:32.106811 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:32.106661    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/73a3358c-ef67-4b35-a010-46c168a95f1e-istio-token\") pod \"e2e-shared-a-w4-da1b8b-openshift-default-7876fcf7c5-p4wc5\" (UID: \"73a3358c-ef67-4b35-a010-46c168a95f1e\") " pod="openshift-ingress/e2e-shared-a-w4-da1b8b-openshift-default-7876fcf7c5-p4wc5"
Oct 30 19:30:32.106811 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:32.106705    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/73a3358c-ef67-4b35-a010-46c168a95f1e-credential-socket\") pod \"e2e-shared-a-w4-da1b8b-openshift-default-7876fcf7c5-p4wc5\" (UID: \"73a3358c-ef67-4b35-a010-46c168a95f1e\") " pod="openshift-ingress/e2e-shared-a-w4-da1b8b-openshift-default-7876fcf7c5-p4wc5"
Oct 30 19:30:32.106811 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:32.106788    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/73a3358c-ef67-4b35-a010-46c168a95f1e-istiod-ca-cert\") pod \"e2e-shared-a-w4-da1b8b-openshift-default-7876fcf7c5-p4wc5\" (UID: \"73a3358c-ef67-4b35-a010-46c168a95f1e\") " pod="openshift-ingress/e2e-shared-a-w4-da1b8b-openshift-default-7876fcf7c5-p4wc5"
Oct 30 19:30:32.106811 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:32.106786    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/73a3358c-ef67-4b35-a010-46c168a95f1e-istio-data\") pod \"e2e-shared-a-w4-da1b8b-openshift-default-7876fcf7c5-p4wc5\" (UID: \"73a3358c-ef67-4b35-a010-46c168a95f1e\") " pod="openshift-ingress/e2e-shared-a-w4-da1b8b-openshift-default-7876fcf7c5-p4wc5"
Oct 30 19:30:32.107022 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:32.106820    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/73a3358c-ef67-4b35-a010-46c168a95f1e-workload-certs\") pod \"e2e-shared-a-w4-da1b8b-openshift-default-7876fcf7c5-p4wc5\" (UID: \"73a3358c-ef67-4b35-a010-46c168a95f1e\") " pod="openshift-ingress/e2e-shared-a-w4-da1b8b-openshift-default-7876fcf7c5-p4wc5"
Oct 30 19:30:32.107138 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:32.107111    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/73a3358c-ef67-4b35-a010-46c168a95f1e-credential-socket\") pod \"e2e-shared-a-w4-da1b8b-openshift-default-7876fcf7c5-p4wc5\" (UID: \"73a3358c-ef67-4b35-a010-46c168a95f1e\") " pod="openshift-ingress/e2e-shared-a-w4-da1b8b-openshift-default-7876fcf7c5-p4wc5"
Oct 30 19:30:32.107383 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:32.107202    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/73a3358c-ef67-4b35-a010-46c168a95f1e-workload-certs\") pod \"e2e-shared-a-w4-da1b8b-openshift-default-7876fcf7c5-p4wc5\" (UID: \"73a3358c-ef67-4b35-a010-46c168a95f1e\") " pod="openshift-ingress/e2e-shared-a-w4-da1b8b-openshift-default-7876fcf7c5-p4wc5"
Oct 30 19:30:32.107383 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:32.107222    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/73a3358c-ef67-4b35-a010-46c168a95f1e-workload-socket\") pod \"e2e-shared-a-w4-da1b8b-openshift-default-7876fcf7c5-p4wc5\" (UID: \"73a3358c-ef67-4b35-a010-46c168a95f1e\") " pod="openshift-ingress/e2e-shared-a-w4-da1b8b-openshift-default-7876fcf7c5-p4wc5"
Oct 30 19:30:32.107963 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:32.107932    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/73a3358c-ef67-4b35-a010-46c168a95f1e-istiod-ca-cert\") pod \"e2e-shared-a-w4-da1b8b-openshift-default-7876fcf7c5-p4wc5\" (UID: \"73a3358c-ef67-4b35-a010-46c168a95f1e\") " pod="openshift-ingress/e2e-shared-a-w4-da1b8b-openshift-default-7876fcf7c5-p4wc5"
Oct 30 19:30:32.109081 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:32.109054    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/73a3358c-ef67-4b35-a010-46c168a95f1e-istio-envoy\") pod \"e2e-shared-a-w4-da1b8b-openshift-default-7876fcf7c5-p4wc5\" (UID: \"73a3358c-ef67-4b35-a010-46c168a95f1e\") " pod="openshift-ingress/e2e-shared-a-w4-da1b8b-openshift-default-7876fcf7c5-p4wc5"
Oct 30 19:30:32.109239 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:32.109218    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/73a3358c-ef67-4b35-a010-46c168a95f1e-istio-podinfo\") pod \"e2e-shared-a-w4-da1b8b-openshift-default-7876fcf7c5-p4wc5\" (UID: \"73a3358c-ef67-4b35-a010-46c168a95f1e\") " pod="openshift-ingress/e2e-shared-a-w4-da1b8b-openshift-default-7876fcf7c5-p4wc5"
Oct 30 19:30:32.175682 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:32.175654    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ww4n4\" (UniqueName: \"kubernetes.io/projected/73a3358c-ef67-4b35-a010-46c168a95f1e-kube-api-access-ww4n4\") pod \"e2e-shared-a-w4-da1b8b-openshift-default-7876fcf7c5-p4wc5\" (UID: \"73a3358c-ef67-4b35-a010-46c168a95f1e\") " pod="openshift-ingress/e2e-shared-a-w4-da1b8b-openshift-default-7876fcf7c5-p4wc5"
Oct 30 19:30:32.175860 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:32.175704    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/73a3358c-ef67-4b35-a010-46c168a95f1e-istio-token\") pod \"e2e-shared-a-w4-da1b8b-openshift-default-7876fcf7c5-p4wc5\" (UID: \"73a3358c-ef67-4b35-a010-46c168a95f1e\") " pod="openshift-ingress/e2e-shared-a-w4-da1b8b-openshift-default-7876fcf7c5-p4wc5"
Oct 30 19:30:32.456024 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:32.455945    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w4-da1b8b-openshift-default-7876fcf7c5-p4wc5"
Oct 30 19:30:32.893094 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:32.893067    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w4-da1b8b-openshift-default-7876fcf7c5-p4wc5"]
Oct 30 19:30:32.893871 ip-10-0-134-46 kubenswrapper[4888]: W1030 19:30:32.893842    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73a3358c_ef67_4b35_a010_46c168a95f1e.slice/crio-1141c20b0e65bcfeb8bb40010c32cb030baf19a23eab9afff19bced9582ea610 WatchSource:0}: Error finding container 1141c20b0e65bcfeb8bb40010c32cb030baf19a23eab9afff19bced9582ea610: Status 404 returned error can't find the container with id 1141c20b0e65bcfeb8bb40010c32cb030baf19a23eab9afff19bced9582ea610
Oct 30 19:30:32.896659 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:32.896625    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:30:32.896788 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:32.896699    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:30:32.896788 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:32.896730    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:30:33.362017 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:33.361980    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w4-da1b8b-openshift-default-7876fcf7c5-p4wc5" event={"ID":"73a3358c-ef67-4b35-a010-46c168a95f1e","Type":"ContainerStarted","Data":"0cd319ad64495916f49bc6f0c96bfb426c5f4908cc00f00238416829cd194448"}
Oct 30 19:30:33.362179 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:33.362062    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w4-da1b8b-openshift-default-7876fcf7c5-p4wc5" event={"ID":"73a3358c-ef67-4b35-a010-46c168a95f1e","Type":"ContainerStarted","Data":"1141c20b0e65bcfeb8bb40010c32cb030baf19a23eab9afff19bced9582ea610"}
Oct 30 19:30:33.456553 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:33.456519    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w4-da1b8b-openshift-default-7876fcf7c5-p4wc5"
Oct 30 19:30:33.462270 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:33.462245    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w4-da1b8b-openshift-default-7876fcf7c5-p4wc5"
Oct 30 19:30:33.470159 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:33.470136    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-explicit-200c46/explicit-model-200c46-kserve-67d6bc85b5-hzc9r"]
Oct 30 19:30:33.470809 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:33.470794    4888 cpu_manager.go:414] "containerMap: removing container" podUID="526209fb-5947-47b1-81e2-218d65c87301" containerName="cleanup"
Oct 30 19:30:33.470871 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:33.470811    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="526209fb-5947-47b1-81e2-218d65c87301" containerName="cleanup"
Oct 30 19:30:33.470922 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:33.470913    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="733efe29-d46f-44e7-883f-0609c99e4531" containerName="cleanup"
Oct 30 19:30:33.470995 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:33.470932    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="120337e2-9322-456b-ab8e-a29e285b8bba" containerName="cleanup"
Oct 30 19:30:33.470995 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:33.470941    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="526209fb-5947-47b1-81e2-218d65c87301" containerName="cleanup"
Oct 30 19:30:33.474263 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:33.474246    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-200c46/explicit-model-200c46-kserve-67d6bc85b5-hzc9r"
Oct 30 19:30:33.517122 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:33.517064    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w4-da1b8b-openshift-default-7876fcf7c5-p4wc5" podStartSLOduration=2.517046138 podStartE2EDuration="2.517046138s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:30:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:30:33.516483273 +0000 UTC m=+3397.763889310" watchObservedRunningTime="2026-10-30 19:30:33.517046138 +0000 UTC m=+3397.764452164"
Oct 30 19:30:33.528495 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:33.528467    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-explicit-200c46/explicit-model-200c46-kserve-67d6bc85b5-hzc9r"]
Oct 30 19:30:33.566651 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:33.566625    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-explicit-200c46\"/\"kube-root-ca.crt\""
Oct 30 19:30:33.566651 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:33.566625    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-explicit-200c46\"/\"default-dockercfg-6jqbv\""
Oct 30 19:30:33.632946 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:33.627040    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4e284413-68bd-4e63-883b-59c5a0c9f13c-home\") pod \"explicit-model-200c46-kserve-67d6bc85b5-hzc9r\" (UID: \"4e284413-68bd-4e63-883b-59c5a0c9f13c\") " pod="ai-tenant-e2e-explicit-200c46/explicit-model-200c46-kserve-67d6bc85b5-hzc9r"
Oct 30 19:30:33.632946 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:33.627139    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4e284413-68bd-4e63-883b-59c5a0c9f13c-tls-certs\") pod \"explicit-model-200c46-kserve-67d6bc85b5-hzc9r\" (UID: \"4e284413-68bd-4e63-883b-59c5a0c9f13c\") " pod="ai-tenant-e2e-explicit-200c46/explicit-model-200c46-kserve-67d6bc85b5-hzc9r"
Oct 30 19:30:33.632946 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:33.627188    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4e284413-68bd-4e63-883b-59c5a0c9f13c-tmp-dir\") pod \"explicit-model-200c46-kserve-67d6bc85b5-hzc9r\" (UID: \"4e284413-68bd-4e63-883b-59c5a0c9f13c\") " pod="ai-tenant-e2e-explicit-200c46/explicit-model-200c46-kserve-67d6bc85b5-hzc9r"
Oct 30 19:30:33.632946 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:33.627257    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4e284413-68bd-4e63-883b-59c5a0c9f13c-model-cache\") pod \"explicit-model-200c46-kserve-67d6bc85b5-hzc9r\" (UID: \"4e284413-68bd-4e63-883b-59c5a0c9f13c\") " pod="ai-tenant-e2e-explicit-200c46/explicit-model-200c46-kserve-67d6bc85b5-hzc9r"
Oct 30 19:30:33.632946 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:33.627316    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-82pn9\" (UniqueName: \"kubernetes.io/projected/4e284413-68bd-4e63-883b-59c5a0c9f13c-kube-api-access-82pn9\") pod \"explicit-model-200c46-kserve-67d6bc85b5-hzc9r\" (UID: \"4e284413-68bd-4e63-883b-59c5a0c9f13c\") " pod="ai-tenant-e2e-explicit-200c46/explicit-model-200c46-kserve-67d6bc85b5-hzc9r"
Oct 30 19:30:33.632946 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:33.627374    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4e284413-68bd-4e63-883b-59c5a0c9f13c-dshm\") pod \"explicit-model-200c46-kserve-67d6bc85b5-hzc9r\" (UID: \"4e284413-68bd-4e63-883b-59c5a0c9f13c\") " pod="ai-tenant-e2e-explicit-200c46/explicit-model-200c46-kserve-67d6bc85b5-hzc9r"
Oct 30 19:30:33.632946 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:33.631451    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-explicit-200c46\"/\"openshift-service-ca.crt\""
Oct 30 19:30:33.632946 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:33.631575    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-explicit-200c46\"/\"explicit-model-200c46-kserve-self-signed-certs\""
Oct 30 19:30:33.728907 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:33.728877    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4e284413-68bd-4e63-883b-59c5a0c9f13c-home\") pod \"explicit-model-200c46-kserve-67d6bc85b5-hzc9r\" (UID: \"4e284413-68bd-4e63-883b-59c5a0c9f13c\") " pod="ai-tenant-e2e-explicit-200c46/explicit-model-200c46-kserve-67d6bc85b5-hzc9r"
Oct 30 19:30:33.729073 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:33.728933    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4e284413-68bd-4e63-883b-59c5a0c9f13c-tls-certs\") pod \"explicit-model-200c46-kserve-67d6bc85b5-hzc9r\" (UID: \"4e284413-68bd-4e63-883b-59c5a0c9f13c\") " pod="ai-tenant-e2e-explicit-200c46/explicit-model-200c46-kserve-67d6bc85b5-hzc9r"
Oct 30 19:30:33.729073 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:33.728961    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4e284413-68bd-4e63-883b-59c5a0c9f13c-tmp-dir\") pod \"explicit-model-200c46-kserve-67d6bc85b5-hzc9r\" (UID: \"4e284413-68bd-4e63-883b-59c5a0c9f13c\") " pod="ai-tenant-e2e-explicit-200c46/explicit-model-200c46-kserve-67d6bc85b5-hzc9r"
Oct 30 19:30:33.729073 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:33.728983    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4e284413-68bd-4e63-883b-59c5a0c9f13c-model-cache\") pod \"explicit-model-200c46-kserve-67d6bc85b5-hzc9r\" (UID: \"4e284413-68bd-4e63-883b-59c5a0c9f13c\") " pod="ai-tenant-e2e-explicit-200c46/explicit-model-200c46-kserve-67d6bc85b5-hzc9r"
Oct 30 19:30:33.729073 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:33.729005    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-82pn9\" (UniqueName: \"kubernetes.io/projected/4e284413-68bd-4e63-883b-59c5a0c9f13c-kube-api-access-82pn9\") pod \"explicit-model-200c46-kserve-67d6bc85b5-hzc9r\" (UID: \"4e284413-68bd-4e63-883b-59c5a0c9f13c\") " pod="ai-tenant-e2e-explicit-200c46/explicit-model-200c46-kserve-67d6bc85b5-hzc9r"
Oct 30 19:30:33.729365 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:33.729125    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4e284413-68bd-4e63-883b-59c5a0c9f13c-dshm\") pod \"explicit-model-200c46-kserve-67d6bc85b5-hzc9r\" (UID: \"4e284413-68bd-4e63-883b-59c5a0c9f13c\") " pod="ai-tenant-e2e-explicit-200c46/explicit-model-200c46-kserve-67d6bc85b5-hzc9r"
Oct 30 19:30:33.729365 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:33.729351    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4e284413-68bd-4e63-883b-59c5a0c9f13c-home\") pod \"explicit-model-200c46-kserve-67d6bc85b5-hzc9r\" (UID: \"4e284413-68bd-4e63-883b-59c5a0c9f13c\") " pod="ai-tenant-e2e-explicit-200c46/explicit-model-200c46-kserve-67d6bc85b5-hzc9r"
Oct 30 19:30:33.729480 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:33.729413    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4e284413-68bd-4e63-883b-59c5a0c9f13c-model-cache\") pod \"explicit-model-200c46-kserve-67d6bc85b5-hzc9r\" (UID: \"4e284413-68bd-4e63-883b-59c5a0c9f13c\") " pod="ai-tenant-e2e-explicit-200c46/explicit-model-200c46-kserve-67d6bc85b5-hzc9r"
Oct 30 19:30:33.729480 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:33.729440    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4e284413-68bd-4e63-883b-59c5a0c9f13c-tmp-dir\") pod \"explicit-model-200c46-kserve-67d6bc85b5-hzc9r\" (UID: \"4e284413-68bd-4e63-883b-59c5a0c9f13c\") " pod="ai-tenant-e2e-explicit-200c46/explicit-model-200c46-kserve-67d6bc85b5-hzc9r"
Oct 30 19:30:33.731483 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:33.731460    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4e284413-68bd-4e63-883b-59c5a0c9f13c-dshm\") pod \"explicit-model-200c46-kserve-67d6bc85b5-hzc9r\" (UID: \"4e284413-68bd-4e63-883b-59c5a0c9f13c\") " pod="ai-tenant-e2e-explicit-200c46/explicit-model-200c46-kserve-67d6bc85b5-hzc9r"
Oct 30 19:30:33.731650 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:33.731630    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4e284413-68bd-4e63-883b-59c5a0c9f13c-tls-certs\") pod \"explicit-model-200c46-kserve-67d6bc85b5-hzc9r\" (UID: \"4e284413-68bd-4e63-883b-59c5a0c9f13c\") " pod="ai-tenant-e2e-explicit-200c46/explicit-model-200c46-kserve-67d6bc85b5-hzc9r"
Oct 30 19:30:33.793635 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:33.793601    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-82pn9\" (UniqueName: \"kubernetes.io/projected/4e284413-68bd-4e63-883b-59c5a0c9f13c-kube-api-access-82pn9\") pod \"explicit-model-200c46-kserve-67d6bc85b5-hzc9r\" (UID: \"4e284413-68bd-4e63-883b-59c5a0c9f13c\") " pod="ai-tenant-e2e-explicit-200c46/explicit-model-200c46-kserve-67d6bc85b5-hzc9r"
Oct 30 19:30:34.088141 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:34.088108    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-200c46/explicit-model-200c46-kserve-67d6bc85b5-hzc9r"
Oct 30 19:30:34.371015 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:34.370931    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w4-da1b8b-openshift-default-7876fcf7c5-p4wc5"
Oct 30 19:30:34.371965 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:34.371940    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w4-da1b8b-openshift-default-7876fcf7c5-p4wc5"
Oct 30 19:30:34.471314 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:34.471273    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-explicit-200c46/explicit-model-200c46-kserve-67d6bc85b5-hzc9r"]
Oct 30 19:30:34.472475 ip-10-0-134-46 kubenswrapper[4888]: W1030 19:30:34.472450    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4e284413_68bd_4e63_883b_59c5a0c9f13c.slice/crio-5f2b432b51d1ab583d0d9ec62bdb2053685d4a6a64f37a86e58abe0bc852f2c8 WatchSource:0}: Error finding container 5f2b432b51d1ab583d0d9ec62bdb2053685d4a6a64f37a86e58abe0bc852f2c8: Status 404 returned error can't find the container with id 5f2b432b51d1ab583d0d9ec62bdb2053685d4a6a64f37a86e58abe0bc852f2c8
Oct 30 19:30:35.393232 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:35.393198    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-200c46/explicit-model-200c46-kserve-67d6bc85b5-hzc9r" event={"ID":"4e284413-68bd-4e63-883b-59c5a0c9f13c","Type":"ContainerStarted","Data":"4cde0768c9259780d6aa1663d371ae44c0591d4ab22034037a9ec89a202b113b"}
Oct 30 19:30:35.393436 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:35.393246    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-200c46/explicit-model-200c46-kserve-67d6bc85b5-hzc9r" event={"ID":"4e284413-68bd-4e63-883b-59c5a0c9f13c","Type":"ContainerStarted","Data":"5f2b432b51d1ab583d0d9ec62bdb2053685d4a6a64f37a86e58abe0bc852f2c8"}
Oct 30 19:30:35.395396 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:35.395369    4888 generic.go:356] "Generic (PLEG): container finished" podID="b1598dab-80ac-408c-b65a-bc3cde3b01d5" containerID="9c0edb68221716f42b960b076aa36b86f73e97f779f65368ee89418000a387f3" exitCode=0
Oct 30 19:30:35.395512 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:35.395438    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-b-f5dfb8-openshift-default-d59df9d89-28vsl" event={"ID":"b1598dab-80ac-408c-b65a-bc3cde3b01d5","Type":"ContainerDied","Data":"9c0edb68221716f42b960b076aa36b86f73e97f779f65368ee89418000a387f3"}
Oct 30 19:30:35.395512 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:35.395496    4888 scope.go:122] "RemoveContainer" containerID="fb2a49e557e558552317c215f316850d9f2b0f5845a5f4702de557d71c99469d"
Oct 30 19:30:35.460102 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:35.460077    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-b-f5dfb8-openshift-default-d59df9d89-28vsl"
Oct 30 19:30:35.513956 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:35.513903    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-explicit-200c46/explicit-model-200c46-kserve-67d6bc85b5-hzc9r" podStartSLOduration=2.5138873 podStartE2EDuration="2.5138873s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:30:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:30:35.513444071 +0000 UTC m=+3399.760850109" watchObservedRunningTime="2026-10-30 19:30:35.5138873 +0000 UTC m=+3399.761293325"
Oct 30 19:30:35.622696 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:30:35.622664    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-jbsw6" containerName="extractor"
Oct 30 19:30:35.655417 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:35.655384    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b1598dab-80ac-408c-b65a-bc3cde3b01d5-kube-api-access-kspg2\" (UniqueName: \"kubernetes.io/projected/b1598dab-80ac-408c-b65a-bc3cde3b01d5-kube-api-access-kspg2\") pod \"b1598dab-80ac-408c-b65a-bc3cde3b01d5\" (UID: \"b1598dab-80ac-408c-b65a-bc3cde3b01d5\") "
Oct 30 19:30:35.655614 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:35.655445    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b1598dab-80ac-408c-b65a-bc3cde3b01d5-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b1598dab-80ac-408c-b65a-bc3cde3b01d5-istio-envoy\") pod \"b1598dab-80ac-408c-b65a-bc3cde3b01d5\" (UID: \"b1598dab-80ac-408c-b65a-bc3cde3b01d5\") "
Oct 30 19:30:35.655614 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:35.655562    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/b1598dab-80ac-408c-b65a-bc3cde3b01d5-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b1598dab-80ac-408c-b65a-bc3cde3b01d5-istio-podinfo\") pod \"b1598dab-80ac-408c-b65a-bc3cde3b01d5\" (UID: \"b1598dab-80ac-408c-b65a-bc3cde3b01d5\") "
Oct 30 19:30:35.655614 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:35.655607    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b1598dab-80ac-408c-b65a-bc3cde3b01d5-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1598dab-80ac-408c-b65a-bc3cde3b01d5-credential-socket\") pod \"b1598dab-80ac-408c-b65a-bc3cde3b01d5\" (UID: \"b1598dab-80ac-408c-b65a-bc3cde3b01d5\") "
Oct 30 19:30:35.655791 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:35.655643    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b1598dab-80ac-408c-b65a-bc3cde3b01d5-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1598dab-80ac-408c-b65a-bc3cde3b01d5-workload-socket\") pod \"b1598dab-80ac-408c-b65a-bc3cde3b01d5\" (UID: \"b1598dab-80ac-408c-b65a-bc3cde3b01d5\") "
Oct 30 19:30:35.655791 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:35.655755    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b1598dab-80ac-408c-b65a-bc3cde3b01d5-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b1598dab-80ac-408c-b65a-bc3cde3b01d5-istiod-ca-cert\") pod \"b1598dab-80ac-408c-b65a-bc3cde3b01d5\" (UID: \"b1598dab-80ac-408c-b65a-bc3cde3b01d5\") "
Oct 30 19:30:35.655903 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:35.655852    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b1598dab-80ac-408c-b65a-bc3cde3b01d5-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b1598dab-80ac-408c-b65a-bc3cde3b01d5-istio-data\") pod \"b1598dab-80ac-408c-b65a-bc3cde3b01d5\" (UID: \"b1598dab-80ac-408c-b65a-bc3cde3b01d5\") "
Oct 30 19:30:35.655903 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:35.655863    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b1598dab-80ac-408c-b65a-bc3cde3b01d5-credential-socket" pod "b1598dab-80ac-408c-b65a-bc3cde3b01d5" (UID: "b1598dab-80ac-408c-b65a-bc3cde3b01d5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:30:35.656004 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:35.655909    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b1598dab-80ac-408c-b65a-bc3cde3b01d5-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b1598dab-80ac-408c-b65a-bc3cde3b01d5-workload-certs\") pod \"b1598dab-80ac-408c-b65a-bc3cde3b01d5\" (UID: \"b1598dab-80ac-408c-b65a-bc3cde3b01d5\") "
Oct 30 19:30:35.656004 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:35.655944    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b1598dab-80ac-408c-b65a-bc3cde3b01d5-istio-token\" (UniqueName: \"kubernetes.io/projected/b1598dab-80ac-408c-b65a-bc3cde3b01d5-istio-token\") pod \"b1598dab-80ac-408c-b65a-bc3cde3b01d5\" (UID: \"b1598dab-80ac-408c-b65a-bc3cde3b01d5\") "
Oct 30 19:30:35.656108 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:35.656015    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b1598dab-80ac-408c-b65a-bc3cde3b01d5-workload-socket" pod "b1598dab-80ac-408c-b65a-bc3cde3b01d5" (UID: "b1598dab-80ac-408c-b65a-bc3cde3b01d5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:30:35.656503 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:35.656147    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b1598dab-80ac-408c-b65a-bc3cde3b01d5-istiod-ca-cert" pod "b1598dab-80ac-408c-b65a-bc3cde3b01d5" (UID: "b1598dab-80ac-408c-b65a-bc3cde3b01d5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:30:35.656503 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:35.656187    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b1598dab-80ac-408c-b65a-bc3cde3b01d5-istio-data" pod "b1598dab-80ac-408c-b65a-bc3cde3b01d5" (UID: "b1598dab-80ac-408c-b65a-bc3cde3b01d5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:30:35.656503 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:35.656473    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b1598dab-80ac-408c-b65a-bc3cde3b01d5-workload-certs" pod "b1598dab-80ac-408c-b65a-bc3cde3b01d5" (UID: "b1598dab-80ac-408c-b65a-bc3cde3b01d5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:30:35.656744 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:35.656609    4888 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1598dab-80ac-408c-b65a-bc3cde3b01d5-credential-socket\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:30:35.656744 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:35.656646    4888 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1598dab-80ac-408c-b65a-bc3cde3b01d5-workload-socket\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:30:35.656744 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:35.656660    4888 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b1598dab-80ac-408c-b65a-bc3cde3b01d5-istiod-ca-cert\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:30:35.656744 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:35.656676    4888 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b1598dab-80ac-408c-b65a-bc3cde3b01d5-istio-data\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:30:35.658468 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:35.658426    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b1598dab-80ac-408c-b65a-bc3cde3b01d5-istio-envoy" pod "b1598dab-80ac-408c-b65a-bc3cde3b01d5" (UID: "b1598dab-80ac-408c-b65a-bc3cde3b01d5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:30:35.659365 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:35.659343    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b1598dab-80ac-408c-b65a-bc3cde3b01d5-kube-api-access-kspg2" pod "b1598dab-80ac-408c-b65a-bc3cde3b01d5" (UID: "b1598dab-80ac-408c-b65a-bc3cde3b01d5"). InnerVolumeSpecName "kube-api-access-kspg2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:30:35.659477 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:35.659401    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b1598dab-80ac-408c-b65a-bc3cde3b01d5-istio-token" pod "b1598dab-80ac-408c-b65a-bc3cde3b01d5" (UID: "b1598dab-80ac-408c-b65a-bc3cde3b01d5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:30:35.660769 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:35.660745    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b1598dab-80ac-408c-b65a-bc3cde3b01d5-istio-podinfo" pod "b1598dab-80ac-408c-b65a-bc3cde3b01d5" (UID: "b1598dab-80ac-408c-b65a-bc3cde3b01d5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:30:35.757952 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:35.757888    4888 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b1598dab-80ac-408c-b65a-bc3cde3b01d5-workload-certs\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:30:35.757952 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:35.757914    4888 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b1598dab-80ac-408c-b65a-bc3cde3b01d5-istio-token\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:30:35.757952 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:35.757924    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kspg2\" (UniqueName: \"kubernetes.io/projected/b1598dab-80ac-408c-b65a-bc3cde3b01d5-kube-api-access-kspg2\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:30:35.757952 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:35.757933    4888 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b1598dab-80ac-408c-b65a-bc3cde3b01d5-istio-envoy\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:30:35.757952 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:35.757941    4888 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b1598dab-80ac-408c-b65a-bc3cde3b01d5-istio-podinfo\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:30:36.088773 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:36.088725    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-explicit-200c46/explicit-model-200c46-kserve-67d6bc85b5-hzc9r"
Oct 30 19:30:36.406024 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:36.405920    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-b-f5dfb8-openshift-default-d59df9d89-28vsl"
Oct 30 19:30:36.406024 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:36.405957    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-b-f5dfb8-openshift-default-d59df9d89-28vsl" event={"ID":"b1598dab-80ac-408c-b65a-bc3cde3b01d5","Type":"ContainerDied","Data":"f99dc4ffc737b109e772a2d6339755ee641489847ce862cfea10dab82b4e8ea8"}
Oct 30 19:30:36.406024 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:36.406016    4888 scope.go:122] "RemoveContainer" containerID="9c0edb68221716f42b960b076aa36b86f73e97f779f65368ee89418000a387f3"
Oct 30 19:30:36.579521 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:36.579483    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-praxis-b-f5dfb8-openshift-default-d59df9d89-28vsl"]
Oct 30 19:30:36.631873 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:36.631837    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-praxis-b-f5dfb8-openshift-default-d59df9d89-28vsl"]
Oct 30 19:30:37.891116 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:37.891081    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b1598dab-80ac-408c-b65a-bc3cde3b01d5" path="/var/lib/kubelet/pods/b1598dab-80ac-408c-b65a-bc3cde3b01d5/volumes"
Oct 30 19:30:38.779857 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:38.779802    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-extmod-a-7fdf6b-openshift-default-65b5b7b9b5-8q2gn"]
Oct 30 19:30:38.780707 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:38.780685    4888 cpu_manager.go:414] "containerMap: removing container" podUID="b1598dab-80ac-408c-b65a-bc3cde3b01d5" containerName="istio-proxy"
Oct 30 19:30:38.780707 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:38.780708    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b1598dab-80ac-408c-b65a-bc3cde3b01d5" containerName="istio-proxy"
Oct 30 19:30:38.780885 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:38.780742    4888 cpu_manager.go:414] "containerMap: removing container" podUID="b1598dab-80ac-408c-b65a-bc3cde3b01d5" containerName="istio-proxy"
Oct 30 19:30:38.780885 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:38.780750    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b1598dab-80ac-408c-b65a-bc3cde3b01d5" containerName="istio-proxy"
Oct 30 19:30:38.781007 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:38.780898    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="b1598dab-80ac-408c-b65a-bc3cde3b01d5" containerName="istio-proxy"
Oct 30 19:30:38.781007 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:38.780915    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="b1598dab-80ac-408c-b65a-bc3cde3b01d5" containerName="istio-proxy"
Oct 30 19:30:38.786122 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:38.786101    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-7fdf6b-openshift-default-65b5b7b9b5-8q2gn"
Oct 30 19:30:38.887412 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:38.887350    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-a-7fdf6b-openshift-default-65b5b7b9b5-8q2gn"]
Oct 30 19:30:38.889565 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:38.889541    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-extmod-a-7fdf6b-openshift-default-dockercfg-mk8hn\""
Oct 30 19:30:38.895939 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:38.895913    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6adbce7d-d243-4f53-b2f0-e1db8b942eb1-istiod-ca-cert\") pod \"e2e-extmod-a-7fdf6b-openshift-default-65b5b7b9b5-8q2gn\" (UID: \"6adbce7d-d243-4f53-b2f0-e1db8b942eb1\") " pod="openshift-ingress/e2e-extmod-a-7fdf6b-openshift-default-65b5b7b9b5-8q2gn"
Oct 30 19:30:38.897672 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:38.895977    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6adbce7d-d243-4f53-b2f0-e1db8b942eb1-credential-socket\") pod \"e2e-extmod-a-7fdf6b-openshift-default-65b5b7b9b5-8q2gn\" (UID: \"6adbce7d-d243-4f53-b2f0-e1db8b942eb1\") " pod="openshift-ingress/e2e-extmod-a-7fdf6b-openshift-default-65b5b7b9b5-8q2gn"
Oct 30 19:30:38.897945 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:38.897924    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xzdsr\" (UniqueName: \"kubernetes.io/projected/6adbce7d-d243-4f53-b2f0-e1db8b942eb1-kube-api-access-xzdsr\") pod \"e2e-extmod-a-7fdf6b-openshift-default-65b5b7b9b5-8q2gn\" (UID: \"6adbce7d-d243-4f53-b2f0-e1db8b942eb1\") " pod="openshift-ingress/e2e-extmod-a-7fdf6b-openshift-default-65b5b7b9b5-8q2gn"
Oct 30 19:30:38.898093 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:38.898077    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6adbce7d-d243-4f53-b2f0-e1db8b942eb1-istio-data\") pod \"e2e-extmod-a-7fdf6b-openshift-default-65b5b7b9b5-8q2gn\" (UID: \"6adbce7d-d243-4f53-b2f0-e1db8b942eb1\") " pod="openshift-ingress/e2e-extmod-a-7fdf6b-openshift-default-65b5b7b9b5-8q2gn"
Oct 30 19:30:38.898186 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:38.898174    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6adbce7d-d243-4f53-b2f0-e1db8b942eb1-istio-token\") pod \"e2e-extmod-a-7fdf6b-openshift-default-65b5b7b9b5-8q2gn\" (UID: \"6adbce7d-d243-4f53-b2f0-e1db8b942eb1\") " pod="openshift-ingress/e2e-extmod-a-7fdf6b-openshift-default-65b5b7b9b5-8q2gn"
Oct 30 19:30:38.898351 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:38.898333    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6adbce7d-d243-4f53-b2f0-e1db8b942eb1-istio-envoy\") pod \"e2e-extmod-a-7fdf6b-openshift-default-65b5b7b9b5-8q2gn\" (UID: \"6adbce7d-d243-4f53-b2f0-e1db8b942eb1\") " pod="openshift-ingress/e2e-extmod-a-7fdf6b-openshift-default-65b5b7b9b5-8q2gn"
Oct 30 19:30:38.898520 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:38.898506    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6adbce7d-d243-4f53-b2f0-e1db8b942eb1-workload-socket\") pod \"e2e-extmod-a-7fdf6b-openshift-default-65b5b7b9b5-8q2gn\" (UID: \"6adbce7d-d243-4f53-b2f0-e1db8b942eb1\") " pod="openshift-ingress/e2e-extmod-a-7fdf6b-openshift-default-65b5b7b9b5-8q2gn"
Oct 30 19:30:38.898700 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:38.898686    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6adbce7d-d243-4f53-b2f0-e1db8b942eb1-istio-podinfo\") pod \"e2e-extmod-a-7fdf6b-openshift-default-65b5b7b9b5-8q2gn\" (UID: \"6adbce7d-d243-4f53-b2f0-e1db8b942eb1\") " pod="openshift-ingress/e2e-extmod-a-7fdf6b-openshift-default-65b5b7b9b5-8q2gn"
Oct 30 19:30:38.898822 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:38.898809    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6adbce7d-d243-4f53-b2f0-e1db8b942eb1-workload-certs\") pod \"e2e-extmod-a-7fdf6b-openshift-default-65b5b7b9b5-8q2gn\" (UID: \"6adbce7d-d243-4f53-b2f0-e1db8b942eb1\") " pod="openshift-ingress/e2e-extmod-a-7fdf6b-openshift-default-65b5b7b9b5-8q2gn"
Oct 30 19:30:38.999620 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:38.999584    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6adbce7d-d243-4f53-b2f0-e1db8b942eb1-istio-podinfo\") pod \"e2e-extmod-a-7fdf6b-openshift-default-65b5b7b9b5-8q2gn\" (UID: \"6adbce7d-d243-4f53-b2f0-e1db8b942eb1\") " pod="openshift-ingress/e2e-extmod-a-7fdf6b-openshift-default-65b5b7b9b5-8q2gn"
Oct 30 19:30:38.999799 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:38.999628    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6adbce7d-d243-4f53-b2f0-e1db8b942eb1-workload-certs\") pod \"e2e-extmod-a-7fdf6b-openshift-default-65b5b7b9b5-8q2gn\" (UID: \"6adbce7d-d243-4f53-b2f0-e1db8b942eb1\") " pod="openshift-ingress/e2e-extmod-a-7fdf6b-openshift-default-65b5b7b9b5-8q2gn"
Oct 30 19:30:38.999799 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:38.999677    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6adbce7d-d243-4f53-b2f0-e1db8b942eb1-istiod-ca-cert\") pod \"e2e-extmod-a-7fdf6b-openshift-default-65b5b7b9b5-8q2gn\" (UID: \"6adbce7d-d243-4f53-b2f0-e1db8b942eb1\") " pod="openshift-ingress/e2e-extmod-a-7fdf6b-openshift-default-65b5b7b9b5-8q2gn"
Oct 30 19:30:38.999799 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:38.999710    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6adbce7d-d243-4f53-b2f0-e1db8b942eb1-credential-socket\") pod \"e2e-extmod-a-7fdf6b-openshift-default-65b5b7b9b5-8q2gn\" (UID: \"6adbce7d-d243-4f53-b2f0-e1db8b942eb1\") " pod="openshift-ingress/e2e-extmod-a-7fdf6b-openshift-default-65b5b7b9b5-8q2gn"
Oct 30 19:30:38.999986 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:38.999802    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xzdsr\" (UniqueName: \"kubernetes.io/projected/6adbce7d-d243-4f53-b2f0-e1db8b942eb1-kube-api-access-xzdsr\") pod \"e2e-extmod-a-7fdf6b-openshift-default-65b5b7b9b5-8q2gn\" (UID: \"6adbce7d-d243-4f53-b2f0-e1db8b942eb1\") " pod="openshift-ingress/e2e-extmod-a-7fdf6b-openshift-default-65b5b7b9b5-8q2gn"
Oct 30 19:30:38.999986 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:38.999837    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6adbce7d-d243-4f53-b2f0-e1db8b942eb1-istio-data\") pod \"e2e-extmod-a-7fdf6b-openshift-default-65b5b7b9b5-8q2gn\" (UID: \"6adbce7d-d243-4f53-b2f0-e1db8b942eb1\") " pod="openshift-ingress/e2e-extmod-a-7fdf6b-openshift-default-65b5b7b9b5-8q2gn"
Oct 30 19:30:38.999986 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:38.999862    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6adbce7d-d243-4f53-b2f0-e1db8b942eb1-istio-token\") pod \"e2e-extmod-a-7fdf6b-openshift-default-65b5b7b9b5-8q2gn\" (UID: \"6adbce7d-d243-4f53-b2f0-e1db8b942eb1\") " pod="openshift-ingress/e2e-extmod-a-7fdf6b-openshift-default-65b5b7b9b5-8q2gn"
Oct 30 19:30:38.999986 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:38.999941    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6adbce7d-d243-4f53-b2f0-e1db8b942eb1-istio-envoy\") pod \"e2e-extmod-a-7fdf6b-openshift-default-65b5b7b9b5-8q2gn\" (UID: \"6adbce7d-d243-4f53-b2f0-e1db8b942eb1\") " pod="openshift-ingress/e2e-extmod-a-7fdf6b-openshift-default-65b5b7b9b5-8q2gn"
Oct 30 19:30:39.000186 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:39.000035    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6adbce7d-d243-4f53-b2f0-e1db8b942eb1-workload-socket\") pod \"e2e-extmod-a-7fdf6b-openshift-default-65b5b7b9b5-8q2gn\" (UID: \"6adbce7d-d243-4f53-b2f0-e1db8b942eb1\") " pod="openshift-ingress/e2e-extmod-a-7fdf6b-openshift-default-65b5b7b9b5-8q2gn"
Oct 30 19:30:39.000604 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:39.000569    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6adbce7d-d243-4f53-b2f0-e1db8b942eb1-istio-data\") pod \"e2e-extmod-a-7fdf6b-openshift-default-65b5b7b9b5-8q2gn\" (UID: \"6adbce7d-d243-4f53-b2f0-e1db8b942eb1\") " pod="openshift-ingress/e2e-extmod-a-7fdf6b-openshift-default-65b5b7b9b5-8q2gn"
Oct 30 19:30:39.000604 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:39.000597    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6adbce7d-d243-4f53-b2f0-e1db8b942eb1-workload-socket\") pod \"e2e-extmod-a-7fdf6b-openshift-default-65b5b7b9b5-8q2gn\" (UID: \"6adbce7d-d243-4f53-b2f0-e1db8b942eb1\") " pod="openshift-ingress/e2e-extmod-a-7fdf6b-openshift-default-65b5b7b9b5-8q2gn"
Oct 30 19:30:39.001140 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:39.001120    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6adbce7d-d243-4f53-b2f0-e1db8b942eb1-istiod-ca-cert\") pod \"e2e-extmod-a-7fdf6b-openshift-default-65b5b7b9b5-8q2gn\" (UID: \"6adbce7d-d243-4f53-b2f0-e1db8b942eb1\") " pod="openshift-ingress/e2e-extmod-a-7fdf6b-openshift-default-65b5b7b9b5-8q2gn"
Oct 30 19:30:39.001668 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:39.001646    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6adbce7d-d243-4f53-b2f0-e1db8b942eb1-workload-certs\") pod \"e2e-extmod-a-7fdf6b-openshift-default-65b5b7b9b5-8q2gn\" (UID: \"6adbce7d-d243-4f53-b2f0-e1db8b942eb1\") " pod="openshift-ingress/e2e-extmod-a-7fdf6b-openshift-default-65b5b7b9b5-8q2gn"
Oct 30 19:30:39.001753 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:39.001725    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6adbce7d-d243-4f53-b2f0-e1db8b942eb1-credential-socket\") pod \"e2e-extmod-a-7fdf6b-openshift-default-65b5b7b9b5-8q2gn\" (UID: \"6adbce7d-d243-4f53-b2f0-e1db8b942eb1\") " pod="openshift-ingress/e2e-extmod-a-7fdf6b-openshift-default-65b5b7b9b5-8q2gn"
Oct 30 19:30:39.002856 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:39.002836    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6adbce7d-d243-4f53-b2f0-e1db8b942eb1-istio-podinfo\") pod \"e2e-extmod-a-7fdf6b-openshift-default-65b5b7b9b5-8q2gn\" (UID: \"6adbce7d-d243-4f53-b2f0-e1db8b942eb1\") " pod="openshift-ingress/e2e-extmod-a-7fdf6b-openshift-default-65b5b7b9b5-8q2gn"
Oct 30 19:30:39.003826 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:39.003807    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6adbce7d-d243-4f53-b2f0-e1db8b942eb1-istio-envoy\") pod \"e2e-extmod-a-7fdf6b-openshift-default-65b5b7b9b5-8q2gn\" (UID: \"6adbce7d-d243-4f53-b2f0-e1db8b942eb1\") " pod="openshift-ingress/e2e-extmod-a-7fdf6b-openshift-default-65b5b7b9b5-8q2gn"
Oct 30 19:30:39.065384 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:39.065330    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xzdsr\" (UniqueName: \"kubernetes.io/projected/6adbce7d-d243-4f53-b2f0-e1db8b942eb1-kube-api-access-xzdsr\") pod \"e2e-extmod-a-7fdf6b-openshift-default-65b5b7b9b5-8q2gn\" (UID: \"6adbce7d-d243-4f53-b2f0-e1db8b942eb1\") " pod="openshift-ingress/e2e-extmod-a-7fdf6b-openshift-default-65b5b7b9b5-8q2gn"
Oct 30 19:30:39.069142 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:39.069117    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6adbce7d-d243-4f53-b2f0-e1db8b942eb1-istio-token\") pod \"e2e-extmod-a-7fdf6b-openshift-default-65b5b7b9b5-8q2gn\" (UID: \"6adbce7d-d243-4f53-b2f0-e1db8b942eb1\") " pod="openshift-ingress/e2e-extmod-a-7fdf6b-openshift-default-65b5b7b9b5-8q2gn"
Oct 30 19:30:39.106637 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:39.101581    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-7fdf6b-openshift-default-65b5b7b9b5-8q2gn"
Oct 30 19:30:39.563106 ip-10-0-134-46 kubenswrapper[4888]: W1030 19:30:39.563079    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6adbce7d_d243_4f53_b2f0_e1db8b942eb1.slice/crio-269f6a4e5c8275b6fb8524153fbf98517109a46316a6b9d81ad5c30473a75d2d WatchSource:0}: Error finding container 269f6a4e5c8275b6fb8524153fbf98517109a46316a6b9d81ad5c30473a75d2d: Status 404 returned error can't find the container with id 269f6a4e5c8275b6fb8524153fbf98517109a46316a6b9d81ad5c30473a75d2d
Oct 30 19:30:39.563495 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:39.563472    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-a-7fdf6b-openshift-default-65b5b7b9b5-8q2gn"]
Oct 30 19:30:39.565754 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:39.565724    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:30:39.565841 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:39.565791    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:30:39.565841 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:39.565832    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:30:40.449743 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:40.449708    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-7fdf6b-openshift-default-65b5b7b9b5-8q2gn" event={"ID":"6adbce7d-d243-4f53-b2f0-e1db8b942eb1","Type":"ContainerStarted","Data":"fdb6c2b135cb669ebf140f23063ba340a8cc37075c9dd94958b33da87e797b4d"}
Oct 30 19:30:40.450145 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:40.449761    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-7fdf6b-openshift-default-65b5b7b9b5-8q2gn" event={"ID":"6adbce7d-d243-4f53-b2f0-e1db8b942eb1","Type":"ContainerStarted","Data":"269f6a4e5c8275b6fb8524153fbf98517109a46316a6b9d81ad5c30473a75d2d"}
Oct 30 19:30:40.642311 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:40.642244    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-extmod-a-7fdf6b-openshift-default-65b5b7b9b5-8q2gn" podStartSLOduration=2.6422286870000002 podStartE2EDuration="2.642228687s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:30:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:30:40.639006623 +0000 UTC m=+3404.886412651" watchObservedRunningTime="2026-10-30 19:30:40.642228687 +0000 UTC m=+3404.889634711"
Oct 30 19:30:41.104371 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:41.104338    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-extmod-a-7fdf6b-openshift-default-65b5b7b9b5-8q2gn"
Oct 30 19:30:41.110631 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:41.110608    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-extmod-a-7fdf6b-openshift-default-65b5b7b9b5-8q2gn"
Oct 30 19:30:41.456504 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:41.456427    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-extmod-a-7fdf6b-openshift-default-65b5b7b9b5-8q2gn"
Oct 30 19:30:41.457484 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:41.457461    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-extmod-a-7fdf6b-openshift-default-65b5b7b9b5-8q2gn"
Oct 30 19:30:44.089003 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:44.088975    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-explicit-200c46/explicit-model-200c46-kserve-67d6bc85b5-hzc9r"
Oct 30 19:30:44.107015 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:44.106989    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-explicit-200c46/explicit-model-200c46-kserve-67d6bc85b5-hzc9r"
Oct 30 19:30:44.490109 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:30:44.490023    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-explicit-200c46/explicit-model-200c46-kserve-67d6bc85b5-hzc9r"
Oct 30 19:31:00.631744 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:31:00.631713    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-rjmdz"]
Oct 30 19:31:05.619456 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:31:05.619421    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-jbsw6" containerName="extractor"
Oct 30 19:31:26.722123 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:31:26.722088    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-200c46-6fb66fd44d-blvbz"]
Oct 30 19:31:31.723367 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:31:31.723291    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-explicit-200c46-6fb66fd44d-blvbz" podUID="b8ad2219-bda0-4eb5-b2d8-4f922611b50a" containerName="payload-processing" containerID="cri-o://b8c9321d71fb3596406cf9fa63d3ede45e13aebe612155a002c648729115e0e9" gracePeriod=25
Oct 30 19:31:32.006075 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:31:32.006053    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-200c46-6fb66fd44d-blvbz"
Oct 30 19:31:32.071742 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:31:32.071718    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b8ad2219-bda0-4eb5-b2d8-4f922611b50a-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b8ad2219-bda0-4eb5-b2d8-4f922611b50a-plugins-config-volume\") pod \"b8ad2219-bda0-4eb5-b2d8-4f922611b50a\" (UID: \"b8ad2219-bda0-4eb5-b2d8-4f922611b50a\") "
Oct 30 19:31:32.071889 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:31:32.071786    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b8ad2219-bda0-4eb5-b2d8-4f922611b50a-kube-api-access-mcx54\" (UniqueName: \"kubernetes.io/projected/b8ad2219-bda0-4eb5-b2d8-4f922611b50a-kube-api-access-mcx54\") pod \"b8ad2219-bda0-4eb5-b2d8-4f922611b50a\" (UID: \"b8ad2219-bda0-4eb5-b2d8-4f922611b50a\") "
Oct 30 19:31:32.072154 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:31:32.072130    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b8ad2219-bda0-4eb5-b2d8-4f922611b50a-plugins-config-volume" pod "b8ad2219-bda0-4eb5-b2d8-4f922611b50a" (UID: "b8ad2219-bda0-4eb5-b2d8-4f922611b50a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:31:32.073914 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:31:32.073894    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b8ad2219-bda0-4eb5-b2d8-4f922611b50a-kube-api-access-mcx54" pod "b8ad2219-bda0-4eb5-b2d8-4f922611b50a" (UID: "b8ad2219-bda0-4eb5-b2d8-4f922611b50a"). InnerVolumeSpecName "kube-api-access-mcx54". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:31:32.172856 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:31:32.172814    4888 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b8ad2219-bda0-4eb5-b2d8-4f922611b50a-plugins-config-volume\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:31:32.172856 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:31:32.172849    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mcx54\" (UniqueName: \"kubernetes.io/projected/b8ad2219-bda0-4eb5-b2d8-4f922611b50a-kube-api-access-mcx54\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:31:32.824696 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:31:32.824628    4888 generic.go:356] "Generic (PLEG): container finished" podID="b8ad2219-bda0-4eb5-b2d8-4f922611b50a" containerID="b8c9321d71fb3596406cf9fa63d3ede45e13aebe612155a002c648729115e0e9" exitCode=0
Oct 30 19:31:32.824696 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:31:32.824676    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-200c46-6fb66fd44d-blvbz"
Oct 30 19:31:32.825234 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:31:32.824728    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-200c46-6fb66fd44d-blvbz" event={"ID":"b8ad2219-bda0-4eb5-b2d8-4f922611b50a","Type":"ContainerDied","Data":"b8c9321d71fb3596406cf9fa63d3ede45e13aebe612155a002c648729115e0e9"}
Oct 30 19:31:32.825234 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:31:32.824786    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-200c46-6fb66fd44d-blvbz" event={"ID":"b8ad2219-bda0-4eb5-b2d8-4f922611b50a","Type":"ContainerDied","Data":"d9d65448646b7f77e323b06d8c2cc11ae7dd9adafd688557498010d170793250"}
Oct 30 19:31:32.825234 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:31:32.824816    4888 scope.go:122] "RemoveContainer" containerID="b8c9321d71fb3596406cf9fa63d3ede45e13aebe612155a002c648729115e0e9"
Oct 30 19:31:32.837675 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:31:32.837654    4888 scope.go:122] "RemoveContainer" containerID="b8c9321d71fb3596406cf9fa63d3ede45e13aebe612155a002c648729115e0e9"
Oct 30 19:31:32.837973 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:31:32.837955    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b8c9321d71fb3596406cf9fa63d3ede45e13aebe612155a002c648729115e0e9\": container with ID starting with b8c9321d71fb3596406cf9fa63d3ede45e13aebe612155a002c648729115e0e9 not found: ID does not exist" containerID="b8c9321d71fb3596406cf9fa63d3ede45e13aebe612155a002c648729115e0e9"
Oct 30 19:31:32.838037 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:31:32.837981    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b8c9321d71fb3596406cf9fa63d3ede45e13aebe612155a002c648729115e0e9"} err="failed to get container status \"b8c9321d71fb3596406cf9fa63d3ede45e13aebe612155a002c648729115e0e9\": rpc error: code = NotFound desc = could not find container \"b8c9321d71fb3596406cf9fa63d3ede45e13aebe612155a002c648729115e0e9\": container with ID starting with b8c9321d71fb3596406cf9fa63d3ede45e13aebe612155a002c648729115e0e9 not found: ID does not exist"
Oct 30 19:31:32.996316 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:31:32.996270    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-200c46-6fb66fd44d-blvbz"]
Oct 30 19:31:33.044809 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:31:33.044782    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-200c46-6fb66fd44d-blvbz"]
Oct 30 19:31:33.192995 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:31:33.192913    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-explicit-200c46/explicit-model-200c46-kserve-67d6bc85b5-hzc9r"]
Oct 30 19:31:33.890707 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:31:33.890676    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b8ad2219-bda0-4eb5-b2d8-4f922611b50a" path="/var/lib/kubelet/pods/b8ad2219-bda0-4eb5-b2d8-4f922611b50a/volumes"
Oct 30 19:31:35.618070 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:31:35.617992    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-jbsw6" containerName="extractor"
Oct 30 19:31:37.113038 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:31:37.113009    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w1-d4f84d/auth-test-model-d4f84d-kserve-5b9745569c-h4pcq"]
Oct 30 19:31:41.548269 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:31:41.548237    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-da1b8b-55c47d44b6-mf594"]
Oct 30 19:31:41.548793 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:31:41.548778    4888 cpu_manager.go:414] "containerMap: removing container" podUID="b8ad2219-bda0-4eb5-b2d8-4f922611b50a" containerName="payload-processing"
Oct 30 19:31:41.548793 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:31:41.548793    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b8ad2219-bda0-4eb5-b2d8-4f922611b50a" containerName="payload-processing"
Oct 30 19:31:41.548923 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:31:41.548911    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="b8ad2219-bda0-4eb5-b2d8-4f922611b50a" containerName="payload-processing"
Oct 30 19:31:41.553619 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:31:41.553602    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-da1b8b-55c47d44b6-mf594"
Oct 30 19:31:41.607615 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:31:41.607588    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w4-da1b8b-serving-cert\""
Oct 30 19:31:41.648487 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:31:41.648462    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-44qj7\" (UniqueName: \"kubernetes.io/projected/53be2715-fb83-4227-9b79-cc1dcd732396-kube-api-access-44qj7\") pod \"maas-api-e2e-shared-a-w4-da1b8b-55c47d44b6-mf594\" (UID: \"53be2715-fb83-4227-9b79-cc1dcd732396\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-da1b8b-55c47d44b6-mf594"
Oct 30 19:31:41.648634 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:31:41.648501    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/53be2715-fb83-4227-9b79-cc1dcd732396-metrics-certs\") pod \"maas-api-e2e-shared-a-w4-da1b8b-55c47d44b6-mf594\" (UID: \"53be2715-fb83-4227-9b79-cc1dcd732396\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-da1b8b-55c47d44b6-mf594"
Oct 30 19:31:41.648634 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:31:41.648586    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/53be2715-fb83-4227-9b79-cc1dcd732396-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w4-da1b8b-55c47d44b6-mf594\" (UID: \"53be2715-fb83-4227-9b79-cc1dcd732396\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-da1b8b-55c47d44b6-mf594"
Oct 30 19:31:41.648709 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:31:41.648698    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/53be2715-fb83-4227-9b79-cc1dcd732396-maas-api-tls\") pod \"maas-api-e2e-shared-a-w4-da1b8b-55c47d44b6-mf594\" (UID: \"53be2715-fb83-4227-9b79-cc1dcd732396\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-da1b8b-55c47d44b6-mf594"
Oct 30 19:31:41.661596 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:31:41.661576    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-da1b8b-55c47d44b6-mf594"]
Oct 30 19:31:41.749451 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:31:41.749418    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-44qj7\" (UniqueName: \"kubernetes.io/projected/53be2715-fb83-4227-9b79-cc1dcd732396-kube-api-access-44qj7\") pod \"maas-api-e2e-shared-a-w4-da1b8b-55c47d44b6-mf594\" (UID: \"53be2715-fb83-4227-9b79-cc1dcd732396\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-da1b8b-55c47d44b6-mf594"
Oct 30 19:31:41.749631 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:31:41.749458    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/53be2715-fb83-4227-9b79-cc1dcd732396-metrics-certs\") pod \"maas-api-e2e-shared-a-w4-da1b8b-55c47d44b6-mf594\" (UID: \"53be2715-fb83-4227-9b79-cc1dcd732396\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-da1b8b-55c47d44b6-mf594"
Oct 30 19:31:41.749631 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:31:41.749498    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/53be2715-fb83-4227-9b79-cc1dcd732396-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w4-da1b8b-55c47d44b6-mf594\" (UID: \"53be2715-fb83-4227-9b79-cc1dcd732396\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-da1b8b-55c47d44b6-mf594"
Oct 30 19:31:41.749631 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:31:41.749527    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/53be2715-fb83-4227-9b79-cc1dcd732396-maas-api-tls\") pod \"maas-api-e2e-shared-a-w4-da1b8b-55c47d44b6-mf594\" (UID: \"53be2715-fb83-4227-9b79-cc1dcd732396\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-da1b8b-55c47d44b6-mf594"
Oct 30 19:31:41.749758 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:31:41.749642    4888 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-da1b8b-serving-cert: secret "maas-api-e2e-shared-a-w4-da1b8b-serving-cert" not found
Oct 30 19:31:41.749758 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:31:41.749705    4888 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/53be2715-fb83-4227-9b79-cc1dcd732396-maas-api-tls podName:53be2715-fb83-4227-9b79-cc1dcd732396 nodeName:}" failed. No retries permitted until 2026-10-30 19:31:42.249687735 +0000 UTC m=+3466.497093737 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/53be2715-fb83-4227-9b79-cc1dcd732396-maas-api-tls") pod "maas-api-e2e-shared-a-w4-da1b8b-55c47d44b6-mf594" (UID: "53be2715-fb83-4227-9b79-cc1dcd732396") : secret "maas-api-e2e-shared-a-w4-da1b8b-serving-cert" not found
Oct 30 19:31:41.750177 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:31:41.750158    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/53be2715-fb83-4227-9b79-cc1dcd732396-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w4-da1b8b-55c47d44b6-mf594\" (UID: \"53be2715-fb83-4227-9b79-cc1dcd732396\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-da1b8b-55c47d44b6-mf594"
Oct 30 19:31:41.752202 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:31:41.752180    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/53be2715-fb83-4227-9b79-cc1dcd732396-metrics-certs\") pod \"maas-api-e2e-shared-a-w4-da1b8b-55c47d44b6-mf594\" (UID: \"53be2715-fb83-4227-9b79-cc1dcd732396\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-da1b8b-55c47d44b6-mf594"
Oct 30 19:31:41.816148 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:31:41.816077    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-44qj7\" (UniqueName: \"kubernetes.io/projected/53be2715-fb83-4227-9b79-cc1dcd732396-kube-api-access-44qj7\") pod \"maas-api-e2e-shared-a-w4-da1b8b-55c47d44b6-mf594\" (UID: \"53be2715-fb83-4227-9b79-cc1dcd732396\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-da1b8b-55c47d44b6-mf594"
Oct 30 19:31:42.256330 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:31:42.256297    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/53be2715-fb83-4227-9b79-cc1dcd732396-maas-api-tls\") pod \"maas-api-e2e-shared-a-w4-da1b8b-55c47d44b6-mf594\" (UID: \"53be2715-fb83-4227-9b79-cc1dcd732396\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-da1b8b-55c47d44b6-mf594"
Oct 30 19:31:42.259040 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:31:42.259015    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/53be2715-fb83-4227-9b79-cc1dcd732396-maas-api-tls\") pod \"maas-api-e2e-shared-a-w4-da1b8b-55c47d44b6-mf594\" (UID: \"53be2715-fb83-4227-9b79-cc1dcd732396\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-da1b8b-55c47d44b6-mf594"
Oct 30 19:31:42.467493 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:31:42.467451    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-da1b8b-55c47d44b6-mf594"
Oct 30 19:31:42.777326 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:31:42.777277    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-rjmdz"]
Oct 30 19:31:42.975604 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:31:42.957774    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-da1b8b-55c47d44b6-mf594"]
Oct 30 19:31:42.987398 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:31:42.986569    4888 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 19:31:43.913565 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:31:43.913535    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-da1b8b-55c47d44b6-mf594" event={"ID":"53be2715-fb83-4227-9b79-cc1dcd732396","Type":"ContainerStarted","Data":"d70105ca1de645aa7021907cd643e486c6db8d5adcf87488f98ccb7b87919a99"}
Oct 30 19:31:44.933491 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:31:44.933453    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-da1b8b-55c47d44b6-mf594" event={"ID":"53be2715-fb83-4227-9b79-cc1dcd732396","Type":"ContainerStarted","Data":"26a1b2512fd03daffd228bfceeaec2befb18eedac023380aaf360f0d2cc89380"}
Oct 30 19:31:44.934001 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:31:44.933556    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-da1b8b-55c47d44b6-mf594"
Oct 30 19:31:45.053543 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:31:45.053488    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-da1b8b-55c47d44b6-mf594" podStartSLOduration=3.04021609 podStartE2EDuration="4.053475527s" totalImagesPullingTime="1.013259437s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:31:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:31:45.052806549 +0000 UTC m=+3469.300212586" watchObservedRunningTime="2026-10-30 19:31:45.053475527 +0000 UTC m=+3469.300881552"
Oct 30 19:31:48.214383 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:31:48.214332    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-explicit-200c46/explicit-model-200c46-kserve-67d6bc85b5-hzc9r" podUID="4e284413-68bd-4e63-883b-59c5a0c9f13c" containerName="main" containerID="cri-o://4cde0768c9259780d6aa1663d371ae44c0591d4ab22034037a9ec89a202b113b" gracePeriod=45
Oct 30 19:31:48.496064 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:31:48.496025    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-200c46/explicit-model-200c46-kserve-67d6bc85b5-hzc9r"
Oct 30 19:31:48.544023 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:31:48.543991    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4e284413-68bd-4e63-883b-59c5a0c9f13c-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4e284413-68bd-4e63-883b-59c5a0c9f13c-model-cache\") pod \"4e284413-68bd-4e63-883b-59c5a0c9f13c\" (UID: \"4e284413-68bd-4e63-883b-59c5a0c9f13c\") "
Oct 30 19:31:48.544228 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:31:48.544072    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4e284413-68bd-4e63-883b-59c5a0c9f13c-dshm\" (UniqueName: \"kubernetes.io/empty-dir/4e284413-68bd-4e63-883b-59c5a0c9f13c-dshm\") pod \"4e284413-68bd-4e63-883b-59c5a0c9f13c\" (UID: \"4e284413-68bd-4e63-883b-59c5a0c9f13c\") "
Oct 30 19:31:48.544228 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:31:48.544127    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4e284413-68bd-4e63-883b-59c5a0c9f13c-home\" (UniqueName: \"kubernetes.io/empty-dir/4e284413-68bd-4e63-883b-59c5a0c9f13c-home\") pod \"4e284413-68bd-4e63-883b-59c5a0c9f13c\" (UID: \"4e284413-68bd-4e63-883b-59c5a0c9f13c\") "
Oct 30 19:31:48.544228 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:31:48.544175    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4e284413-68bd-4e63-883b-59c5a0c9f13c-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4e284413-68bd-4e63-883b-59c5a0c9f13c-tmp-dir\") pod \"4e284413-68bd-4e63-883b-59c5a0c9f13c\" (UID: \"4e284413-68bd-4e63-883b-59c5a0c9f13c\") "
Oct 30 19:31:48.544415 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:31:48.544256    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4e284413-68bd-4e63-883b-59c5a0c9f13c-tls-certs\" (UniqueName: \"kubernetes.io/secret/4e284413-68bd-4e63-883b-59c5a0c9f13c-tls-certs\") pod \"4e284413-68bd-4e63-883b-59c5a0c9f13c\" (UID: \"4e284413-68bd-4e63-883b-59c5a0c9f13c\") "
Oct 30 19:31:48.544415 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:31:48.544407    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4e284413-68bd-4e63-883b-59c5a0c9f13c-kube-api-access-82pn9\" (UniqueName: \"kubernetes.io/projected/4e284413-68bd-4e63-883b-59c5a0c9f13c-kube-api-access-82pn9\") pod \"4e284413-68bd-4e63-883b-59c5a0c9f13c\" (UID: \"4e284413-68bd-4e63-883b-59c5a0c9f13c\") "
Oct 30 19:31:48.544520 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:31:48.544254    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e284413-68bd-4e63-883b-59c5a0c9f13c-model-cache" pod "4e284413-68bd-4e63-883b-59c5a0c9f13c" (UID: "4e284413-68bd-4e63-883b-59c5a0c9f13c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:31:48.544520 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:31:48.544454    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e284413-68bd-4e63-883b-59c5a0c9f13c-home" pod "4e284413-68bd-4e63-883b-59c5a0c9f13c" (UID: "4e284413-68bd-4e63-883b-59c5a0c9f13c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:31:48.544800 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:31:48.544770    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e284413-68bd-4e63-883b-59c5a0c9f13c-tmp-dir" pod "4e284413-68bd-4e63-883b-59c5a0c9f13c" (UID: "4e284413-68bd-4e63-883b-59c5a0c9f13c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:31:48.544940 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:31:48.544834    4888 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4e284413-68bd-4e63-883b-59c5a0c9f13c-home\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:31:48.544940 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:31:48.544851    4888 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4e284413-68bd-4e63-883b-59c5a0c9f13c-model-cache\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:31:48.546881 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:31:48.546858    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e284413-68bd-4e63-883b-59c5a0c9f13c-dshm" pod "4e284413-68bd-4e63-883b-59c5a0c9f13c" (UID: "4e284413-68bd-4e63-883b-59c5a0c9f13c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:31:48.547138 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:31:48.547114    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4e284413-68bd-4e63-883b-59c5a0c9f13c-kube-api-access-82pn9" pod "4e284413-68bd-4e63-883b-59c5a0c9f13c" (UID: "4e284413-68bd-4e63-883b-59c5a0c9f13c"). InnerVolumeSpecName "kube-api-access-82pn9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:31:48.547323 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:31:48.547171    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4e284413-68bd-4e63-883b-59c5a0c9f13c-tls-certs" pod "4e284413-68bd-4e63-883b-59c5a0c9f13c" (UID: "4e284413-68bd-4e63-883b-59c5a0c9f13c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:31:48.646241 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:31:48.646200    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-82pn9\" (UniqueName: \"kubernetes.io/projected/4e284413-68bd-4e63-883b-59c5a0c9f13c-kube-api-access-82pn9\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:31:48.646241 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:31:48.646244    4888 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4e284413-68bd-4e63-883b-59c5a0c9f13c-dshm\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:31:48.646471 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:31:48.646297    4888 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4e284413-68bd-4e63-883b-59c5a0c9f13c-tmp-dir\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:31:48.646471 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:31:48.646312    4888 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4e284413-68bd-4e63-883b-59c5a0c9f13c-tls-certs\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:31:48.959583 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:31:48.959551    4888 generic.go:356] "Generic (PLEG): container finished" podID="4e284413-68bd-4e63-883b-59c5a0c9f13c" containerID="4cde0768c9259780d6aa1663d371ae44c0591d4ab22034037a9ec89a202b113b" exitCode=0
Oct 30 19:31:48.959773 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:31:48.959595    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-200c46/explicit-model-200c46-kserve-67d6bc85b5-hzc9r"
Oct 30 19:31:48.959773 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:31:48.959623    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-200c46/explicit-model-200c46-kserve-67d6bc85b5-hzc9r" event={"ID":"4e284413-68bd-4e63-883b-59c5a0c9f13c","Type":"ContainerDied","Data":"4cde0768c9259780d6aa1663d371ae44c0591d4ab22034037a9ec89a202b113b"}
Oct 30 19:31:48.959773 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:31:48.959666    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-200c46/explicit-model-200c46-kserve-67d6bc85b5-hzc9r" event={"ID":"4e284413-68bd-4e63-883b-59c5a0c9f13c","Type":"ContainerDied","Data":"5f2b432b51d1ab583d0d9ec62bdb2053685d4a6a64f37a86e58abe0bc852f2c8"}
Oct 30 19:31:48.959773 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:31:48.959683    4888 scope.go:122] "RemoveContainer" containerID="4cde0768c9259780d6aa1663d371ae44c0591d4ab22034037a9ec89a202b113b"
Oct 30 19:31:48.970364 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:31:48.970346    4888 scope.go:122] "RemoveContainer" containerID="4cde0768c9259780d6aa1663d371ae44c0591d4ab22034037a9ec89a202b113b"
Oct 30 19:31:48.970632 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:31:48.970606    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4cde0768c9259780d6aa1663d371ae44c0591d4ab22034037a9ec89a202b113b\": container with ID starting with 4cde0768c9259780d6aa1663d371ae44c0591d4ab22034037a9ec89a202b113b not found: ID does not exist" containerID="4cde0768c9259780d6aa1663d371ae44c0591d4ab22034037a9ec89a202b113b"
Oct 30 19:31:48.970676 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:31:48.970639    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4cde0768c9259780d6aa1663d371ae44c0591d4ab22034037a9ec89a202b113b"} err="failed to get container status \"4cde0768c9259780d6aa1663d371ae44c0591d4ab22034037a9ec89a202b113b\": rpc error: code = NotFound desc = could not find container \"4cde0768c9259780d6aa1663d371ae44c0591d4ab22034037a9ec89a202b113b\": container with ID starting with 4cde0768c9259780d6aa1663d371ae44c0591d4ab22034037a9ec89a202b113b not found: ID does not exist"
Oct 30 19:31:49.133330 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:31:49.133278    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-explicit-200c46/explicit-model-200c46-kserve-67d6bc85b5-hzc9r"]
Oct 30 19:31:49.179380 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:31:49.179345    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-explicit-200c46/explicit-model-200c46-kserve-67d6bc85b5-hzc9r"]
Oct 30 19:31:49.479636 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:31:49.479594    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-explicit-200c46/explicit-model-200c46-kserve-67d6bc85b5-hzc9r" podUID="4e284413-68bd-4e63-883b-59c5a0c9f13c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.162:8000/ready\": context deadline exceeded"
Oct 30 19:31:49.893046 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:31:49.893016    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4e284413-68bd-4e63-883b-59c5a0c9f13c" path="/var/lib/kubelet/pods/4e284413-68bd-4e63-883b-59c5a0c9f13c/volumes"
Oct 30 19:31:50.945064 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:31:50.945040    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-da1b8b-55c47d44b6-mf594"
Oct 30 19:31:52.134446 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:31:52.134391    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w1-d4f84d/auth-test-model-d4f84d-kserve-5b9745569c-h4pcq" podUID="362aa1c7-3163-49ee-9ed9-6a965a4a9a56" containerName="main" containerID="cri-o://fdb3523ce12e2652f8b3b046ec55606c9635ae42f413b35556fb7c444b5828f1" gracePeriod=45
Oct 30 19:31:52.439450 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:31:52.439425    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w1-d4f84d/auth-test-model-d4f84d-kserve-5b9745569c-h4pcq"
Oct 30 19:31:52.495358 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:31:52.495331    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/362aa1c7-3163-49ee-9ed9-6a965a4a9a56-kube-api-access-fkz9n\" (UniqueName: \"kubernetes.io/projected/362aa1c7-3163-49ee-9ed9-6a965a4a9a56-kube-api-access-fkz9n\") pod \"362aa1c7-3163-49ee-9ed9-6a965a4a9a56\" (UID: \"362aa1c7-3163-49ee-9ed9-6a965a4a9a56\") "
Oct 30 19:31:52.495533 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:31:52.495369    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/362aa1c7-3163-49ee-9ed9-6a965a4a9a56-home\" (UniqueName: \"kubernetes.io/empty-dir/362aa1c7-3163-49ee-9ed9-6a965a4a9a56-home\") pod \"362aa1c7-3163-49ee-9ed9-6a965a4a9a56\" (UID: \"362aa1c7-3163-49ee-9ed9-6a965a4a9a56\") "
Oct 30 19:31:52.495533 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:31:52.495422    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/362aa1c7-3163-49ee-9ed9-6a965a4a9a56-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/362aa1c7-3163-49ee-9ed9-6a965a4a9a56-model-cache\") pod \"362aa1c7-3163-49ee-9ed9-6a965a4a9a56\" (UID: \"362aa1c7-3163-49ee-9ed9-6a965a4a9a56\") "
Oct 30 19:31:52.495533 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:31:52.495457    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/362aa1c7-3163-49ee-9ed9-6a965a4a9a56-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/362aa1c7-3163-49ee-9ed9-6a965a4a9a56-tmp-dir\") pod \"362aa1c7-3163-49ee-9ed9-6a965a4a9a56\" (UID: \"362aa1c7-3163-49ee-9ed9-6a965a4a9a56\") "
Oct 30 19:31:52.495533 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:31:52.495525    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/362aa1c7-3163-49ee-9ed9-6a965a4a9a56-tls-certs\" (UniqueName: \"kubernetes.io/secret/362aa1c7-3163-49ee-9ed9-6a965a4a9a56-tls-certs\") pod \"362aa1c7-3163-49ee-9ed9-6a965a4a9a56\" (UID: \"362aa1c7-3163-49ee-9ed9-6a965a4a9a56\") "
Oct 30 19:31:52.495751 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:31:52.495579    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/362aa1c7-3163-49ee-9ed9-6a965a4a9a56-dshm\" (UniqueName: \"kubernetes.io/empty-dir/362aa1c7-3163-49ee-9ed9-6a965a4a9a56-dshm\") pod \"362aa1c7-3163-49ee-9ed9-6a965a4a9a56\" (UID: \"362aa1c7-3163-49ee-9ed9-6a965a4a9a56\") "
Oct 30 19:31:52.495751 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:31:52.495734    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/362aa1c7-3163-49ee-9ed9-6a965a4a9a56-home" pod "362aa1c7-3163-49ee-9ed9-6a965a4a9a56" (UID: "362aa1c7-3163-49ee-9ed9-6a965a4a9a56"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:31:52.495861 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:31:52.495750    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/362aa1c7-3163-49ee-9ed9-6a965a4a9a56-model-cache" pod "362aa1c7-3163-49ee-9ed9-6a965a4a9a56" (UID: "362aa1c7-3163-49ee-9ed9-6a965a4a9a56"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:31:52.496151 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:31:52.496128    4888 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/362aa1c7-3163-49ee-9ed9-6a965a4a9a56-home\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:31:52.496315 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:31:52.496156    4888 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/362aa1c7-3163-49ee-9ed9-6a965a4a9a56-model-cache\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:31:52.496315 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:31:52.496179    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/362aa1c7-3163-49ee-9ed9-6a965a4a9a56-tmp-dir" pod "362aa1c7-3163-49ee-9ed9-6a965a4a9a56" (UID: "362aa1c7-3163-49ee-9ed9-6a965a4a9a56"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:31:52.498075 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:31:52.498049    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/362aa1c7-3163-49ee-9ed9-6a965a4a9a56-dshm" pod "362aa1c7-3163-49ee-9ed9-6a965a4a9a56" (UID: "362aa1c7-3163-49ee-9ed9-6a965a4a9a56"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:31:52.498189 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:31:52.498084    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/362aa1c7-3163-49ee-9ed9-6a965a4a9a56-tls-certs" pod "362aa1c7-3163-49ee-9ed9-6a965a4a9a56" (UID: "362aa1c7-3163-49ee-9ed9-6a965a4a9a56"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:31:52.498248 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:31:52.498201    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/362aa1c7-3163-49ee-9ed9-6a965a4a9a56-kube-api-access-fkz9n" pod "362aa1c7-3163-49ee-9ed9-6a965a4a9a56" (UID: "362aa1c7-3163-49ee-9ed9-6a965a4a9a56"). InnerVolumeSpecName "kube-api-access-fkz9n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:31:52.597902 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:31:52.597865    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fkz9n\" (UniqueName: \"kubernetes.io/projected/362aa1c7-3163-49ee-9ed9-6a965a4a9a56-kube-api-access-fkz9n\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:31:52.597902 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:31:52.597903    4888 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/362aa1c7-3163-49ee-9ed9-6a965a4a9a56-tmp-dir\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:31:52.598118 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:31:52.597918    4888 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/362aa1c7-3163-49ee-9ed9-6a965a4a9a56-tls-certs\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:31:52.598118 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:31:52.597931    4888 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/362aa1c7-3163-49ee-9ed9-6a965a4a9a56-dshm\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:31:52.994073 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:31:52.994036    4888 generic.go:356] "Generic (PLEG): container finished" podID="362aa1c7-3163-49ee-9ed9-6a965a4a9a56" containerID="fdb3523ce12e2652f8b3b046ec55606c9635ae42f413b35556fb7c444b5828f1" exitCode=0
Oct 30 19:31:52.994319 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:31:52.994093    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w1-d4f84d/auth-test-model-d4f84d-kserve-5b9745569c-h4pcq"
Oct 30 19:31:52.994319 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:31:52.994096    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w1-d4f84d/auth-test-model-d4f84d-kserve-5b9745569c-h4pcq" event={"ID":"362aa1c7-3163-49ee-9ed9-6a965a4a9a56","Type":"ContainerDied","Data":"fdb3523ce12e2652f8b3b046ec55606c9635ae42f413b35556fb7c444b5828f1"}
Oct 30 19:31:52.994319 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:31:52.994241    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w1-d4f84d/auth-test-model-d4f84d-kserve-5b9745569c-h4pcq" event={"ID":"362aa1c7-3163-49ee-9ed9-6a965a4a9a56","Type":"ContainerDied","Data":"dc64a3d96c8ec344959f0970d24682c56fa04c1d96760fa4cf44605078d9dda8"}
Oct 30 19:31:52.994319 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:31:52.994268    4888 scope.go:122] "RemoveContainer" containerID="fdb3523ce12e2652f8b3b046ec55606c9635ae42f413b35556fb7c444b5828f1"
Oct 30 19:31:53.008133 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:31:53.008114    4888 scope.go:122] "RemoveContainer" containerID="fdb3523ce12e2652f8b3b046ec55606c9635ae42f413b35556fb7c444b5828f1"
Oct 30 19:31:53.008463 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:31:53.008438    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fdb3523ce12e2652f8b3b046ec55606c9635ae42f413b35556fb7c444b5828f1\": container with ID starting with fdb3523ce12e2652f8b3b046ec55606c9635ae42f413b35556fb7c444b5828f1 not found: ID does not exist" containerID="fdb3523ce12e2652f8b3b046ec55606c9635ae42f413b35556fb7c444b5828f1"
Oct 30 19:31:53.008547 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:31:53.008475    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fdb3523ce12e2652f8b3b046ec55606c9635ae42f413b35556fb7c444b5828f1"} err="failed to get container status \"fdb3523ce12e2652f8b3b046ec55606c9635ae42f413b35556fb7c444b5828f1\": rpc error: code = NotFound desc = could not find container \"fdb3523ce12e2652f8b3b046ec55606c9635ae42f413b35556fb7c444b5828f1\": container with ID starting with fdb3523ce12e2652f8b3b046ec55606c9635ae42f413b35556fb7c444b5828f1 not found: ID does not exist"
Oct 30 19:31:53.164426 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:31:53.164395    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w1-d4f84d/auth-test-model-d4f84d-kserve-5b9745569c-h4pcq"]
Oct 30 19:31:53.217332 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:31:53.217264    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w1-d4f84d/auth-test-model-d4f84d-kserve-5b9745569c-h4pcq"]
Oct 30 19:31:53.893703 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:31:53.893657    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="362aa1c7-3163-49ee-9ed9-6a965a4a9a56" path="/var/lib/kubelet/pods/362aa1c7-3163-49ee-9ed9-6a965a4a9a56/volumes"
Oct 30 19:32:05.621734 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:32:05.621702    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-jbsw6" containerName="extractor"
Oct 30 19:32:05.746584 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:05.746551    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w0-19f6ea/auth-test-model-19f6ea-kserve-b646899bf-7bmlq"]
Oct 30 19:32:20.772304 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:20.771499    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w0-19f6ea/auth-test-model-19f6ea-kserve-b646899bf-7bmlq" podUID="53796da4-5dc9-40e5-8ce1-42a20807b8f0" containerName="main" containerID="cri-o://78b2a9c9da7d201419a810890cfe3442f4de5291a6ef886058889ac812abeb4c" gracePeriod=45
Oct 30 19:32:21.065546 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:21.065521    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w0-19f6ea/auth-test-model-19f6ea-kserve-b646899bf-7bmlq"
Oct 30 19:32:21.164154 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:21.164120    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/53796da4-5dc9-40e5-8ce1-42a20807b8f0-kube-api-access-j6lrv\" (UniqueName: \"kubernetes.io/projected/53796da4-5dc9-40e5-8ce1-42a20807b8f0-kube-api-access-j6lrv\") pod \"53796da4-5dc9-40e5-8ce1-42a20807b8f0\" (UID: \"53796da4-5dc9-40e5-8ce1-42a20807b8f0\") "
Oct 30 19:32:21.164396 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:21.164232    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/53796da4-5dc9-40e5-8ce1-42a20807b8f0-home\" (UniqueName: \"kubernetes.io/empty-dir/53796da4-5dc9-40e5-8ce1-42a20807b8f0-home\") pod \"53796da4-5dc9-40e5-8ce1-42a20807b8f0\" (UID: \"53796da4-5dc9-40e5-8ce1-42a20807b8f0\") "
Oct 30 19:32:21.164481 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:21.164401    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/53796da4-5dc9-40e5-8ce1-42a20807b8f0-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/53796da4-5dc9-40e5-8ce1-42a20807b8f0-model-cache\") pod \"53796da4-5dc9-40e5-8ce1-42a20807b8f0\" (UID: \"53796da4-5dc9-40e5-8ce1-42a20807b8f0\") "
Oct 30 19:32:21.164481 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:21.164460    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/53796da4-5dc9-40e5-8ce1-42a20807b8f0-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/53796da4-5dc9-40e5-8ce1-42a20807b8f0-tmp-dir\") pod \"53796da4-5dc9-40e5-8ce1-42a20807b8f0\" (UID: \"53796da4-5dc9-40e5-8ce1-42a20807b8f0\") "
Oct 30 19:32:21.164587 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:21.164508    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/53796da4-5dc9-40e5-8ce1-42a20807b8f0-dshm\" (UniqueName: \"kubernetes.io/empty-dir/53796da4-5dc9-40e5-8ce1-42a20807b8f0-dshm\") pod \"53796da4-5dc9-40e5-8ce1-42a20807b8f0\" (UID: \"53796da4-5dc9-40e5-8ce1-42a20807b8f0\") "
Oct 30 19:32:21.164587 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:21.164522    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/53796da4-5dc9-40e5-8ce1-42a20807b8f0-home" pod "53796da4-5dc9-40e5-8ce1-42a20807b8f0" (UID: "53796da4-5dc9-40e5-8ce1-42a20807b8f0"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:32:21.164587 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:21.164558    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/53796da4-5dc9-40e5-8ce1-42a20807b8f0-tls-certs\" (UniqueName: \"kubernetes.io/secret/53796da4-5dc9-40e5-8ce1-42a20807b8f0-tls-certs\") pod \"53796da4-5dc9-40e5-8ce1-42a20807b8f0\" (UID: \"53796da4-5dc9-40e5-8ce1-42a20807b8f0\") "
Oct 30 19:32:21.164740 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:21.164652    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/53796da4-5dc9-40e5-8ce1-42a20807b8f0-model-cache" pod "53796da4-5dc9-40e5-8ce1-42a20807b8f0" (UID: "53796da4-5dc9-40e5-8ce1-42a20807b8f0"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:32:21.164791 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:21.164759    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/53796da4-5dc9-40e5-8ce1-42a20807b8f0-tmp-dir" pod "53796da4-5dc9-40e5-8ce1-42a20807b8f0" (UID: "53796da4-5dc9-40e5-8ce1-42a20807b8f0"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:32:21.165059 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:21.165039    4888 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/53796da4-5dc9-40e5-8ce1-42a20807b8f0-home\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:32:21.165059 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:21.165060    4888 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/53796da4-5dc9-40e5-8ce1-42a20807b8f0-model-cache\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:32:21.165059 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:21.165070    4888 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/53796da4-5dc9-40e5-8ce1-42a20807b8f0-tmp-dir\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:32:21.166758 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:21.166735    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/53796da4-5dc9-40e5-8ce1-42a20807b8f0-dshm" pod "53796da4-5dc9-40e5-8ce1-42a20807b8f0" (UID: "53796da4-5dc9-40e5-8ce1-42a20807b8f0"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:32:21.166858 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:21.166771    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/53796da4-5dc9-40e5-8ce1-42a20807b8f0-kube-api-access-j6lrv" pod "53796da4-5dc9-40e5-8ce1-42a20807b8f0" (UID: "53796da4-5dc9-40e5-8ce1-42a20807b8f0"). InnerVolumeSpecName "kube-api-access-j6lrv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:32:21.166900 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:21.166852    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/53796da4-5dc9-40e5-8ce1-42a20807b8f0-tls-certs" pod "53796da4-5dc9-40e5-8ce1-42a20807b8f0" (UID: "53796da4-5dc9-40e5-8ce1-42a20807b8f0"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:32:21.206503 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:21.206466    4888 generic.go:356] "Generic (PLEG): container finished" podID="53796da4-5dc9-40e5-8ce1-42a20807b8f0" containerID="78b2a9c9da7d201419a810890cfe3442f4de5291a6ef886058889ac812abeb4c" exitCode=0
Oct 30 19:32:21.206661 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:21.206520    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w0-19f6ea/auth-test-model-19f6ea-kserve-b646899bf-7bmlq" event={"ID":"53796da4-5dc9-40e5-8ce1-42a20807b8f0","Type":"ContainerDied","Data":"78b2a9c9da7d201419a810890cfe3442f4de5291a6ef886058889ac812abeb4c"}
Oct 30 19:32:21.206661 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:21.206560    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w0-19f6ea/auth-test-model-19f6ea-kserve-b646899bf-7bmlq" event={"ID":"53796da4-5dc9-40e5-8ce1-42a20807b8f0","Type":"ContainerDied","Data":"ffba741576506023fc51deba9960e0bc6af170d90b316ff66470c697702b8ef9"}
Oct 30 19:32:21.206661 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:21.206581    4888 scope.go:122] "RemoveContainer" containerID="78b2a9c9da7d201419a810890cfe3442f4de5291a6ef886058889ac812abeb4c"
Oct 30 19:32:21.206782 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:21.206746    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w0-19f6ea/auth-test-model-19f6ea-kserve-b646899bf-7bmlq"
Oct 30 19:32:21.220456 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:21.220433    4888 scope.go:122] "RemoveContainer" containerID="78b2a9c9da7d201419a810890cfe3442f4de5291a6ef886058889ac812abeb4c"
Oct 30 19:32:21.220726 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:32:21.220699    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"78b2a9c9da7d201419a810890cfe3442f4de5291a6ef886058889ac812abeb4c\": container with ID starting with 78b2a9c9da7d201419a810890cfe3442f4de5291a6ef886058889ac812abeb4c not found: ID does not exist" containerID="78b2a9c9da7d201419a810890cfe3442f4de5291a6ef886058889ac812abeb4c"
Oct 30 19:32:21.220802 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:21.220738    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"78b2a9c9da7d201419a810890cfe3442f4de5291a6ef886058889ac812abeb4c"} err="failed to get container status \"78b2a9c9da7d201419a810890cfe3442f4de5291a6ef886058889ac812abeb4c\": rpc error: code = NotFound desc = could not find container \"78b2a9c9da7d201419a810890cfe3442f4de5291a6ef886058889ac812abeb4c\": container with ID starting with 78b2a9c9da7d201419a810890cfe3442f4de5291a6ef886058889ac812abeb4c not found: ID does not exist"
Oct 30 19:32:21.266576 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:21.266538    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j6lrv\" (UniqueName: \"kubernetes.io/projected/53796da4-5dc9-40e5-8ce1-42a20807b8f0-kube-api-access-j6lrv\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:32:21.266576 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:21.266569    4888 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/53796da4-5dc9-40e5-8ce1-42a20807b8f0-dshm\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:32:21.266576 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:21.266585    4888 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/53796da4-5dc9-40e5-8ce1-42a20807b8f0-tls-certs\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:32:21.378962 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:21.378930    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w0-19f6ea/auth-test-model-19f6ea-kserve-b646899bf-7bmlq"]
Oct 30 19:32:21.381936 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:21.381913    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w0-19f6ea/auth-test-model-19f6ea-kserve-b646899bf-7bmlq"]
Oct 30 19:32:21.899857 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:21.899822    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="53796da4-5dc9-40e5-8ce1-42a20807b8f0" path="/var/lib/kubelet/pods/53796da4-5dc9-40e5-8ce1-42a20807b8f0/volumes"
Oct 30 19:32:30.252070 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:30.252031    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-explicit-200c46-openshift-default-5f4d5cd597-t47sj"]
Oct 30 19:32:30.252603 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:30.252363    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-explicit-200c46-openshift-default-5f4d5cd597-t47sj" podUID="f7045fa9-0689-4317-95fc-d3bf87806ce1" containerName="istio-proxy" containerID="cri-o://603a8e613de33663f6326a93227cb1e07514511f0e0c20a124f3a33c63e0b0d1" gracePeriod=30
Oct 30 19:32:31.549866 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:31.524066    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-noten-3edc13-openshift-default-8487df5f79-bmkwd"]
Oct 30 19:32:31.549866 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:31.524932    4888 cpu_manager.go:414] "containerMap: removing container" podUID="53796da4-5dc9-40e5-8ce1-42a20807b8f0" containerName="main"
Oct 30 19:32:31.549866 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:31.524950    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="53796da4-5dc9-40e5-8ce1-42a20807b8f0" containerName="main"
Oct 30 19:32:31.549866 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:31.524992    4888 cpu_manager.go:414] "containerMap: removing container" podUID="362aa1c7-3163-49ee-9ed9-6a965a4a9a56" containerName="main"
Oct 30 19:32:31.549866 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:31.525000    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="362aa1c7-3163-49ee-9ed9-6a965a4a9a56" containerName="main"
Oct 30 19:32:31.549866 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:31.525016    4888 cpu_manager.go:414] "containerMap: removing container" podUID="4e284413-68bd-4e63-883b-59c5a0c9f13c" containerName="main"
Oct 30 19:32:31.549866 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:31.525024    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4e284413-68bd-4e63-883b-59c5a0c9f13c" containerName="main"
Oct 30 19:32:31.549866 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:31.525173    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="362aa1c7-3163-49ee-9ed9-6a965a4a9a56" containerName="main"
Oct 30 19:32:31.549866 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:31.525185    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="4e284413-68bd-4e63-883b-59c5a0c9f13c" containerName="main"
Oct 30 19:32:31.549866 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:31.525206    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="53796da4-5dc9-40e5-8ce1-42a20807b8f0" containerName="main"
Oct 30 19:32:31.549866 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:31.540903    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-3edc13-openshift-default-8487df5f79-bmkwd"
Oct 30 19:32:31.604825 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:31.604794    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-noten-3edc13-openshift-default-dockercfg-vthw9\""
Oct 30 19:32:31.660572 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:31.660496    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-noten-3edc13-openshift-default-8487df5f79-bmkwd"]
Oct 30 19:32:31.679726 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:31.679697    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p7q8k\" (UniqueName: \"kubernetes.io/projected/82782e69-02e1-44ef-b7f4-72f847f5a913-kube-api-access-p7q8k\") pod \"e2e-noten-3edc13-openshift-default-8487df5f79-bmkwd\" (UID: \"82782e69-02e1-44ef-b7f4-72f847f5a913\") " pod="openshift-ingress/e2e-noten-3edc13-openshift-default-8487df5f79-bmkwd"
Oct 30 19:32:31.679936 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:31.679738    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/82782e69-02e1-44ef-b7f4-72f847f5a913-istio-token\") pod \"e2e-noten-3edc13-openshift-default-8487df5f79-bmkwd\" (UID: \"82782e69-02e1-44ef-b7f4-72f847f5a913\") " pod="openshift-ingress/e2e-noten-3edc13-openshift-default-8487df5f79-bmkwd"
Oct 30 19:32:31.679936 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:31.679792    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/82782e69-02e1-44ef-b7f4-72f847f5a913-workload-certs\") pod \"e2e-noten-3edc13-openshift-default-8487df5f79-bmkwd\" (UID: \"82782e69-02e1-44ef-b7f4-72f847f5a913\") " pod="openshift-ingress/e2e-noten-3edc13-openshift-default-8487df5f79-bmkwd"
Oct 30 19:32:31.679936 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:31.679845    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/82782e69-02e1-44ef-b7f4-72f847f5a913-credential-socket\") pod \"e2e-noten-3edc13-openshift-default-8487df5f79-bmkwd\" (UID: \"82782e69-02e1-44ef-b7f4-72f847f5a913\") " pod="openshift-ingress/e2e-noten-3edc13-openshift-default-8487df5f79-bmkwd"
Oct 30 19:32:31.680232 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:31.680087    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/82782e69-02e1-44ef-b7f4-72f847f5a913-workload-socket\") pod \"e2e-noten-3edc13-openshift-default-8487df5f79-bmkwd\" (UID: \"82782e69-02e1-44ef-b7f4-72f847f5a913\") " pod="openshift-ingress/e2e-noten-3edc13-openshift-default-8487df5f79-bmkwd"
Oct 30 19:32:31.680232 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:31.680133    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/82782e69-02e1-44ef-b7f4-72f847f5a913-istio-podinfo\") pod \"e2e-noten-3edc13-openshift-default-8487df5f79-bmkwd\" (UID: \"82782e69-02e1-44ef-b7f4-72f847f5a913\") " pod="openshift-ingress/e2e-noten-3edc13-openshift-default-8487df5f79-bmkwd"
Oct 30 19:32:31.680232 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:31.680165    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/82782e69-02e1-44ef-b7f4-72f847f5a913-istio-data\") pod \"e2e-noten-3edc13-openshift-default-8487df5f79-bmkwd\" (UID: \"82782e69-02e1-44ef-b7f4-72f847f5a913\") " pod="openshift-ingress/e2e-noten-3edc13-openshift-default-8487df5f79-bmkwd"
Oct 30 19:32:31.680232 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:31.680191    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/82782e69-02e1-44ef-b7f4-72f847f5a913-istio-envoy\") pod \"e2e-noten-3edc13-openshift-default-8487df5f79-bmkwd\" (UID: \"82782e69-02e1-44ef-b7f4-72f847f5a913\") " pod="openshift-ingress/e2e-noten-3edc13-openshift-default-8487df5f79-bmkwd"
Oct 30 19:32:31.680232 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:31.680223    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/82782e69-02e1-44ef-b7f4-72f847f5a913-istiod-ca-cert\") pod \"e2e-noten-3edc13-openshift-default-8487df5f79-bmkwd\" (UID: \"82782e69-02e1-44ef-b7f4-72f847f5a913\") " pod="openshift-ingress/e2e-noten-3edc13-openshift-default-8487df5f79-bmkwd"
Oct 30 19:32:31.780996 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:31.780953    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/82782e69-02e1-44ef-b7f4-72f847f5a913-workload-socket\") pod \"e2e-noten-3edc13-openshift-default-8487df5f79-bmkwd\" (UID: \"82782e69-02e1-44ef-b7f4-72f847f5a913\") " pod="openshift-ingress/e2e-noten-3edc13-openshift-default-8487df5f79-bmkwd"
Oct 30 19:32:31.781208 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:31.781001    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/82782e69-02e1-44ef-b7f4-72f847f5a913-istio-podinfo\") pod \"e2e-noten-3edc13-openshift-default-8487df5f79-bmkwd\" (UID: \"82782e69-02e1-44ef-b7f4-72f847f5a913\") " pod="openshift-ingress/e2e-noten-3edc13-openshift-default-8487df5f79-bmkwd"
Oct 30 19:32:31.781208 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:31.781033    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/82782e69-02e1-44ef-b7f4-72f847f5a913-istio-data\") pod \"e2e-noten-3edc13-openshift-default-8487df5f79-bmkwd\" (UID: \"82782e69-02e1-44ef-b7f4-72f847f5a913\") " pod="openshift-ingress/e2e-noten-3edc13-openshift-default-8487df5f79-bmkwd"
Oct 30 19:32:31.781208 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:31.781072    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/82782e69-02e1-44ef-b7f4-72f847f5a913-istio-envoy\") pod \"e2e-noten-3edc13-openshift-default-8487df5f79-bmkwd\" (UID: \"82782e69-02e1-44ef-b7f4-72f847f5a913\") " pod="openshift-ingress/e2e-noten-3edc13-openshift-default-8487df5f79-bmkwd"
Oct 30 19:32:31.781208 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:31.781103    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/82782e69-02e1-44ef-b7f4-72f847f5a913-istiod-ca-cert\") pod \"e2e-noten-3edc13-openshift-default-8487df5f79-bmkwd\" (UID: \"82782e69-02e1-44ef-b7f4-72f847f5a913\") " pod="openshift-ingress/e2e-noten-3edc13-openshift-default-8487df5f79-bmkwd"
Oct 30 19:32:31.781208 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:31.781158    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p7q8k\" (UniqueName: \"kubernetes.io/projected/82782e69-02e1-44ef-b7f4-72f847f5a913-kube-api-access-p7q8k\") pod \"e2e-noten-3edc13-openshift-default-8487df5f79-bmkwd\" (UID: \"82782e69-02e1-44ef-b7f4-72f847f5a913\") " pod="openshift-ingress/e2e-noten-3edc13-openshift-default-8487df5f79-bmkwd"
Oct 30 19:32:31.781208 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:31.781180    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/82782e69-02e1-44ef-b7f4-72f847f5a913-istio-token\") pod \"e2e-noten-3edc13-openshift-default-8487df5f79-bmkwd\" (UID: \"82782e69-02e1-44ef-b7f4-72f847f5a913\") " pod="openshift-ingress/e2e-noten-3edc13-openshift-default-8487df5f79-bmkwd"
Oct 30 19:32:31.781717 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:31.781231    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/82782e69-02e1-44ef-b7f4-72f847f5a913-workload-certs\") pod \"e2e-noten-3edc13-openshift-default-8487df5f79-bmkwd\" (UID: \"82782e69-02e1-44ef-b7f4-72f847f5a913\") " pod="openshift-ingress/e2e-noten-3edc13-openshift-default-8487df5f79-bmkwd"
Oct 30 19:32:31.781717 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:31.781329    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/82782e69-02e1-44ef-b7f4-72f847f5a913-credential-socket\") pod \"e2e-noten-3edc13-openshift-default-8487df5f79-bmkwd\" (UID: \"82782e69-02e1-44ef-b7f4-72f847f5a913\") " pod="openshift-ingress/e2e-noten-3edc13-openshift-default-8487df5f79-bmkwd"
Oct 30 19:32:31.781717 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:31.781517    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/82782e69-02e1-44ef-b7f4-72f847f5a913-workload-socket\") pod \"e2e-noten-3edc13-openshift-default-8487df5f79-bmkwd\" (UID: \"82782e69-02e1-44ef-b7f4-72f847f5a913\") " pod="openshift-ingress/e2e-noten-3edc13-openshift-default-8487df5f79-bmkwd"
Oct 30 19:32:31.781717 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:31.781687    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/82782e69-02e1-44ef-b7f4-72f847f5a913-credential-socket\") pod \"e2e-noten-3edc13-openshift-default-8487df5f79-bmkwd\" (UID: \"82782e69-02e1-44ef-b7f4-72f847f5a913\") " pod="openshift-ingress/e2e-noten-3edc13-openshift-default-8487df5f79-bmkwd"
Oct 30 19:32:31.781977 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:31.781943    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/82782e69-02e1-44ef-b7f4-72f847f5a913-workload-certs\") pod \"e2e-noten-3edc13-openshift-default-8487df5f79-bmkwd\" (UID: \"82782e69-02e1-44ef-b7f4-72f847f5a913\") " pod="openshift-ingress/e2e-noten-3edc13-openshift-default-8487df5f79-bmkwd"
Oct 30 19:32:31.782041 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:31.781975    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/82782e69-02e1-44ef-b7f4-72f847f5a913-istio-data\") pod \"e2e-noten-3edc13-openshift-default-8487df5f79-bmkwd\" (UID: \"82782e69-02e1-44ef-b7f4-72f847f5a913\") " pod="openshift-ingress/e2e-noten-3edc13-openshift-default-8487df5f79-bmkwd"
Oct 30 19:32:31.782658 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:31.782632    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/82782e69-02e1-44ef-b7f4-72f847f5a913-istiod-ca-cert\") pod \"e2e-noten-3edc13-openshift-default-8487df5f79-bmkwd\" (UID: \"82782e69-02e1-44ef-b7f4-72f847f5a913\") " pod="openshift-ingress/e2e-noten-3edc13-openshift-default-8487df5f79-bmkwd"
Oct 30 19:32:31.784372 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:31.784346    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/82782e69-02e1-44ef-b7f4-72f847f5a913-istio-envoy\") pod \"e2e-noten-3edc13-openshift-default-8487df5f79-bmkwd\" (UID: \"82782e69-02e1-44ef-b7f4-72f847f5a913\") " pod="openshift-ingress/e2e-noten-3edc13-openshift-default-8487df5f79-bmkwd"
Oct 30 19:32:31.784604 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:31.784585    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/82782e69-02e1-44ef-b7f4-72f847f5a913-istio-podinfo\") pod \"e2e-noten-3edc13-openshift-default-8487df5f79-bmkwd\" (UID: \"82782e69-02e1-44ef-b7f4-72f847f5a913\") " pod="openshift-ingress/e2e-noten-3edc13-openshift-default-8487df5f79-bmkwd"
Oct 30 19:32:31.848702 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:31.848671    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/82782e69-02e1-44ef-b7f4-72f847f5a913-istio-token\") pod \"e2e-noten-3edc13-openshift-default-8487df5f79-bmkwd\" (UID: \"82782e69-02e1-44ef-b7f4-72f847f5a913\") " pod="openshift-ingress/e2e-noten-3edc13-openshift-default-8487df5f79-bmkwd"
Oct 30 19:32:31.848855 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:31.848802    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p7q8k\" (UniqueName: \"kubernetes.io/projected/82782e69-02e1-44ef-b7f4-72f847f5a913-kube-api-access-p7q8k\") pod \"e2e-noten-3edc13-openshift-default-8487df5f79-bmkwd\" (UID: \"82782e69-02e1-44ef-b7f4-72f847f5a913\") " pod="openshift-ingress/e2e-noten-3edc13-openshift-default-8487df5f79-bmkwd"
Oct 30 19:32:31.870573 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:31.870549    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-3edc13-openshift-default-8487df5f79-bmkwd"
Oct 30 19:32:32.317402 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:32.316785    4888 generic.go:356] "Generic (PLEG): container finished" podID="6adbce7d-d243-4f53-b2f0-e1db8b942eb1" containerID="fdb6c2b135cb669ebf140f23063ba340a8cc37075c9dd94958b33da87e797b4d" exitCode=0
Oct 30 19:32:32.317402 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:32.316919    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-7fdf6b-openshift-default-65b5b7b9b5-8q2gn" event={"ID":"6adbce7d-d243-4f53-b2f0-e1db8b942eb1","Type":"ContainerDied","Data":"fdb6c2b135cb669ebf140f23063ba340a8cc37075c9dd94958b33da87e797b4d"}
Oct 30 19:32:32.318017 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:32.317994    4888 scope.go:122] "RemoveContainer" containerID="fdb6c2b135cb669ebf140f23063ba340a8cc37075c9dd94958b33da87e797b4d"
Oct 30 19:32:32.328507 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:32.328473    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:32:32.328588 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:32.328573    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:32:32.328636 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:32.328620    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:32:32.332142 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:32.332119    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-noten-3edc13-openshift-default-8487df5f79-bmkwd"]
Oct 30 19:32:32.363773 ip-10-0-134-46 kubenswrapper[4888]: W1030 19:32:32.363751    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82782e69_02e1_44ef_b7f4_72f847f5a913.slice/crio-79ba62a286bc9ba57dff76dc2c8cb9fadaee3c68813135727e369cc9eb538593 WatchSource:0}: Error finding container 79ba62a286bc9ba57dff76dc2c8cb9fadaee3c68813135727e369cc9eb538593: Status 404 returned error can't find the container with id 79ba62a286bc9ba57dff76dc2c8cb9fadaee3c68813135727e369cc9eb538593
Oct 30 19:32:32.366150 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:32.366122    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:32:32.366237 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:32.366181    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:32:32.366237 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:32.366208    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:32:33.325968 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:33.325940    4888 generic.go:356] "Generic (PLEG): container finished" podID="73a3358c-ef67-4b35-a010-46c168a95f1e" containerID="0cd319ad64495916f49bc6f0c96bfb426c5f4908cc00f00238416829cd194448" exitCode=0
Oct 30 19:32:33.326420 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:33.326012    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w4-da1b8b-openshift-default-7876fcf7c5-p4wc5" event={"ID":"73a3358c-ef67-4b35-a010-46c168a95f1e","Type":"ContainerDied","Data":"0cd319ad64495916f49bc6f0c96bfb426c5f4908cc00f00238416829cd194448"}
Oct 30 19:32:33.326496 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:33.326460    4888 scope.go:122] "RemoveContainer" containerID="0cd319ad64495916f49bc6f0c96bfb426c5f4908cc00f00238416829cd194448"
Oct 30 19:32:33.328137 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:33.328110    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:32:33.328255 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:33.328173    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-7fdf6b-openshift-default-65b5b7b9b5-8q2gn" event={"ID":"6adbce7d-d243-4f53-b2f0-e1db8b942eb1","Type":"ContainerStarted","Data":"634d90cecd836dd461e5e997d9b9f40a1b832e04238b11c6d484e26dc1e040e8"}
Oct 30 19:32:33.328255 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:33.328178    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:32:33.328410 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:33.328335    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:32:33.330253 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:33.330231    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-3edc13-openshift-default-8487df5f79-bmkwd" event={"ID":"82782e69-02e1-44ef-b7f4-72f847f5a913","Type":"ContainerStarted","Data":"6f132904e2b31099412a5543adb7d98898535b5a5ecbda6141ccc01dcda67ee7"}
Oct 30 19:32:33.330363 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:33.330272    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-3edc13-openshift-default-8487df5f79-bmkwd" event={"ID":"82782e69-02e1-44ef-b7f4-72f847f5a913","Type":"ContainerStarted","Data":"79ba62a286bc9ba57dff76dc2c8cb9fadaee3c68813135727e369cc9eb538593"}
Oct 30 19:32:33.717706 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:33.717600    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-noten-3edc13-openshift-default-8487df5f79-bmkwd" podStartSLOduration=2.71758063 podStartE2EDuration="2.71758063s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:32:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:32:33.710904224 +0000 UTC m=+3517.958310253" watchObservedRunningTime="2026-10-30 19:32:33.71758063 +0000 UTC m=+3517.964986659"
Oct 30 19:32:33.871670 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:33.871590    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-noten-3edc13-openshift-default-8487df5f79-bmkwd"
Oct 30 19:32:33.908417 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:33.908384    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-noten-3edc13-openshift-default-8487df5f79-bmkwd"
Oct 30 19:32:34.107743 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:34.107665    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-extmod-a-7fdf6b-openshift-default-65b5b7b9b5-8q2gn"
Oct 30 19:32:34.107955 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:34.107801    4888 patch_prober.go:29] interesting pod/e2e-extmod-a-7fdf6b-openshift-default-65b5b7b9b5-8q2gn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.163:15021/healthz/ready\": dial tcp 10.132.0.163:15021: connect: connection refused" start-of-body=
Oct 30 19:32:34.107955 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:34.107846    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-7fdf6b-openshift-default-65b5b7b9b5-8q2gn" podUID="6adbce7d-d243-4f53-b2f0-e1db8b942eb1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.163:15021/healthz/ready\": dial tcp 10.132.0.163:15021: connect: connection refused"
Oct 30 19:32:34.340337 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:34.340303    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w4-da1b8b-openshift-default-7876fcf7c5-p4wc5" event={"ID":"73a3358c-ef67-4b35-a010-46c168a95f1e","Type":"ContainerStarted","Data":"e935d77e8b1bc9d4a13053d6547fb4793ed00bc8827a743a610ebfda6db1a8c5"}
Oct 30 19:32:34.340913 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:34.340889    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-noten-3edc13-openshift-default-8487df5f79-bmkwd"
Oct 30 19:32:34.341793 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:34.341778    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-noten-3edc13-openshift-default-8487df5f79-bmkwd"
Oct 30 19:32:34.388485 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:34.388391    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w4-da1b8b-openshift-default-7876fcf7c5-p4wc5"
Oct 30 19:32:34.457227 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:34.457199    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w4-da1b8b-openshift-default-7876fcf7c5-p4wc5"
Oct 30 19:32:34.458504 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:34.458014    4888 patch_prober.go:29] interesting pod/e2e-shared-a-w4-da1b8b-openshift-default-7876fcf7c5-p4wc5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.161:15021/healthz/ready\": dial tcp 10.132.0.161:15021: connect: connection refused" start-of-body=
Oct 30 19:32:34.458504 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:34.458052    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w4-da1b8b-openshift-default-7876fcf7c5-p4wc5" podUID="73a3358c-ef67-4b35-a010-46c168a95f1e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.161:15021/healthz/ready\": dial tcp 10.132.0.161:15021: connect: connection refused"
Oct 30 19:32:35.108728 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:35.108687    4888 patch_prober.go:29] interesting pod/e2e-extmod-a-7fdf6b-openshift-default-65b5b7b9b5-8q2gn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.163:15021/healthz/ready\": dial tcp 10.132.0.163:15021: connect: connection refused" start-of-body=
Oct 30 19:32:35.109020 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:35.108986    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-7fdf6b-openshift-default-65b5b7b9b5-8q2gn" podUID="6adbce7d-d243-4f53-b2f0-e1db8b942eb1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.163:15021/healthz/ready\": dial tcp 10.132.0.163:15021: connect: connection refused"
Oct 30 19:32:35.456666 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:35.456636    4888 patch_prober.go:29] interesting pod/e2e-shared-a-w4-da1b8b-openshift-default-7876fcf7c5-p4wc5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.161:15021/healthz/ready\": dial tcp 10.132.0.161:15021: connect: connection refused" start-of-body=
Oct 30 19:32:35.457045 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:35.456685    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w4-da1b8b-openshift-default-7876fcf7c5-p4wc5" podUID="73a3358c-ef67-4b35-a010-46c168a95f1e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.161:15021/healthz/ready\": dial tcp 10.132.0.161:15021: connect: connection refused"
Oct 30 19:32:35.652975 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:32:35.652918    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-jbsw6" containerName="extractor"
Oct 30 19:32:35.659085 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:35.659066    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-200c46-openshift-default-5f4d5cd597-t47sj"
Oct 30 19:32:35.757934 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:35.757906    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/f7045fa9-0689-4317-95fc-d3bf87806ce1-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f7045fa9-0689-4317-95fc-d3bf87806ce1-istio-podinfo\") pod \"f7045fa9-0689-4317-95fc-d3bf87806ce1\" (UID: \"f7045fa9-0689-4317-95fc-d3bf87806ce1\") "
Oct 30 19:32:35.758111 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:35.757946    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f7045fa9-0689-4317-95fc-d3bf87806ce1-istio-token\" (UniqueName: \"kubernetes.io/projected/f7045fa9-0689-4317-95fc-d3bf87806ce1-istio-token\") pod \"f7045fa9-0689-4317-95fc-d3bf87806ce1\" (UID: \"f7045fa9-0689-4317-95fc-d3bf87806ce1\") "
Oct 30 19:32:35.758111 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:35.758010    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f7045fa9-0689-4317-95fc-d3bf87806ce1-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f7045fa9-0689-4317-95fc-d3bf87806ce1-istio-envoy\") pod \"f7045fa9-0689-4317-95fc-d3bf87806ce1\" (UID: \"f7045fa9-0689-4317-95fc-d3bf87806ce1\") "
Oct 30 19:32:35.758111 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:35.758053    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f7045fa9-0689-4317-95fc-d3bf87806ce1-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7045fa9-0689-4317-95fc-d3bf87806ce1-credential-socket\") pod \"f7045fa9-0689-4317-95fc-d3bf87806ce1\" (UID: \"f7045fa9-0689-4317-95fc-d3bf87806ce1\") "
Oct 30 19:32:35.758111 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:35.758093    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f7045fa9-0689-4317-95fc-d3bf87806ce1-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f7045fa9-0689-4317-95fc-d3bf87806ce1-workload-certs\") pod \"f7045fa9-0689-4317-95fc-d3bf87806ce1\" (UID: \"f7045fa9-0689-4317-95fc-d3bf87806ce1\") "
Oct 30 19:32:35.758386 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:35.758128    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f7045fa9-0689-4317-95fc-d3bf87806ce1-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f7045fa9-0689-4317-95fc-d3bf87806ce1-istio-data\") pod \"f7045fa9-0689-4317-95fc-d3bf87806ce1\" (UID: \"f7045fa9-0689-4317-95fc-d3bf87806ce1\") "
Oct 30 19:32:35.758386 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:35.758151    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f7045fa9-0689-4317-95fc-d3bf87806ce1-kube-api-access-hgxhd\" (UniqueName: \"kubernetes.io/projected/f7045fa9-0689-4317-95fc-d3bf87806ce1-kube-api-access-hgxhd\") pod \"f7045fa9-0689-4317-95fc-d3bf87806ce1\" (UID: \"f7045fa9-0689-4317-95fc-d3bf87806ce1\") "
Oct 30 19:32:35.758386 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:35.758209    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f7045fa9-0689-4317-95fc-d3bf87806ce1-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7045fa9-0689-4317-95fc-d3bf87806ce1-workload-socket\") pod \"f7045fa9-0689-4317-95fc-d3bf87806ce1\" (UID: \"f7045fa9-0689-4317-95fc-d3bf87806ce1\") "
Oct 30 19:32:35.758386 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:35.758256    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f7045fa9-0689-4317-95fc-d3bf87806ce1-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f7045fa9-0689-4317-95fc-d3bf87806ce1-istiod-ca-cert\") pod \"f7045fa9-0689-4317-95fc-d3bf87806ce1\" (UID: \"f7045fa9-0689-4317-95fc-d3bf87806ce1\") "
Oct 30 19:32:35.758620 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:35.758590    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7045fa9-0689-4317-95fc-d3bf87806ce1-workload-certs" pod "f7045fa9-0689-4317-95fc-d3bf87806ce1" (UID: "f7045fa9-0689-4317-95fc-d3bf87806ce1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:32:35.758676 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:35.758605    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7045fa9-0689-4317-95fc-d3bf87806ce1-workload-socket" pod "f7045fa9-0689-4317-95fc-d3bf87806ce1" (UID: "f7045fa9-0689-4317-95fc-d3bf87806ce1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:32:35.758676 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:35.758617    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7045fa9-0689-4317-95fc-d3bf87806ce1-credential-socket" pod "f7045fa9-0689-4317-95fc-d3bf87806ce1" (UID: "f7045fa9-0689-4317-95fc-d3bf87806ce1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:32:35.758842 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:35.758818    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7045fa9-0689-4317-95fc-d3bf87806ce1-istio-data" pod "f7045fa9-0689-4317-95fc-d3bf87806ce1" (UID: "f7045fa9-0689-4317-95fc-d3bf87806ce1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:32:35.758906 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:35.758847    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f7045fa9-0689-4317-95fc-d3bf87806ce1-istiod-ca-cert" pod "f7045fa9-0689-4317-95fc-d3bf87806ce1" (UID: "f7045fa9-0689-4317-95fc-d3bf87806ce1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:32:35.758963 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:35.758899    4888 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f7045fa9-0689-4317-95fc-d3bf87806ce1-workload-certs\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:32:35.758963 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:35.758922    4888 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7045fa9-0689-4317-95fc-d3bf87806ce1-workload-socket\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:32:35.760562 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:35.760534    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f7045fa9-0689-4317-95fc-d3bf87806ce1-istio-podinfo" pod "f7045fa9-0689-4317-95fc-d3bf87806ce1" (UID: "f7045fa9-0689-4317-95fc-d3bf87806ce1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:32:35.761683 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:35.761499    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f7045fa9-0689-4317-95fc-d3bf87806ce1-istio-token" pod "f7045fa9-0689-4317-95fc-d3bf87806ce1" (UID: "f7045fa9-0689-4317-95fc-d3bf87806ce1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:32:35.761803 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:35.761492    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7045fa9-0689-4317-95fc-d3bf87806ce1-istio-envoy" pod "f7045fa9-0689-4317-95fc-d3bf87806ce1" (UID: "f7045fa9-0689-4317-95fc-d3bf87806ce1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:32:35.761803 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:35.761524    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f7045fa9-0689-4317-95fc-d3bf87806ce1-kube-api-access-hgxhd" pod "f7045fa9-0689-4317-95fc-d3bf87806ce1" (UID: "f7045fa9-0689-4317-95fc-d3bf87806ce1"). InnerVolumeSpecName "kube-api-access-hgxhd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:32:35.859716 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:35.859686    4888 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f7045fa9-0689-4317-95fc-d3bf87806ce1-istio-token\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:32:35.859716 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:35.859718    4888 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f7045fa9-0689-4317-95fc-d3bf87806ce1-istio-envoy\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:32:35.859911 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:35.859733    4888 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f7045fa9-0689-4317-95fc-d3bf87806ce1-credential-socket\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:32:35.859911 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:35.859742    4888 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f7045fa9-0689-4317-95fc-d3bf87806ce1-istio-data\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:32:35.859911 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:35.859753    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hgxhd\" (UniqueName: \"kubernetes.io/projected/f7045fa9-0689-4317-95fc-d3bf87806ce1-kube-api-access-hgxhd\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:32:35.859911 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:35.859806    4888 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f7045fa9-0689-4317-95fc-d3bf87806ce1-istiod-ca-cert\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:32:35.859911 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:35.859824    4888 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f7045fa9-0689-4317-95fc-d3bf87806ce1-istio-podinfo\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:32:36.110529 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:36.110496    4888 patch_prober.go:29] interesting pod/e2e-extmod-a-7fdf6b-openshift-default-65b5b7b9b5-8q2gn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.163:15021/healthz/ready\": dial tcp 10.132.0.163:15021: connect: connection refused" start-of-body=
Oct 30 19:32:36.110829 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:36.110800    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-7fdf6b-openshift-default-65b5b7b9b5-8q2gn" podUID="6adbce7d-d243-4f53-b2f0-e1db8b942eb1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.163:15021/healthz/ready\": dial tcp 10.132.0.163:15021: connect: connection refused"
Oct 30 19:32:36.356118 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:36.356088    4888 generic.go:356] "Generic (PLEG): container finished" podID="f7045fa9-0689-4317-95fc-d3bf87806ce1" containerID="603a8e613de33663f6326a93227cb1e07514511f0e0c20a124f3a33c63e0b0d1" exitCode=0
Oct 30 19:32:36.356365 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:36.356348    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-200c46-openshift-default-5f4d5cd597-t47sj"
Oct 30 19:32:36.356473 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:36.356249    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-200c46-openshift-default-5f4d5cd597-t47sj" event={"ID":"f7045fa9-0689-4317-95fc-d3bf87806ce1","Type":"ContainerDied","Data":"603a8e613de33663f6326a93227cb1e07514511f0e0c20a124f3a33c63e0b0d1"}
Oct 30 19:32:36.356533 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:36.356479    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-200c46-openshift-default-5f4d5cd597-t47sj" event={"ID":"f7045fa9-0689-4317-95fc-d3bf87806ce1","Type":"ContainerDied","Data":"55a65a0e2c5db80a34ab8cc5ef7e461cdda4d33bd0e562f79fb0580dfb20ac6d"}
Oct 30 19:32:36.356533 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:36.356522    4888 scope.go:122] "RemoveContainer" containerID="603a8e613de33663f6326a93227cb1e07514511f0e0c20a124f3a33c63e0b0d1"
Oct 30 19:32:36.393526 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:36.392458    4888 scope.go:122] "RemoveContainer" containerID="011d00ec587977bd8622e89c6434b2e80ccb712763ece18f9533bf46fed0c95e"
Oct 30 19:32:36.444269 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:36.432715    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-extmod-a-9e4e63-openshift-default-5785d6dfd9-nbk9s"]
Oct 30 19:32:36.444269 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:36.444093    4888 scope.go:122] "RemoveContainer" containerID="603a8e613de33663f6326a93227cb1e07514511f0e0c20a124f3a33c63e0b0d1"
Oct 30 19:32:36.445008 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:32:36.444653    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"603a8e613de33663f6326a93227cb1e07514511f0e0c20a124f3a33c63e0b0d1\": container with ID starting with 603a8e613de33663f6326a93227cb1e07514511f0e0c20a124f3a33c63e0b0d1 not found: ID does not exist" containerID="603a8e613de33663f6326a93227cb1e07514511f0e0c20a124f3a33c63e0b0d1"
Oct 30 19:32:36.445008 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:36.444698    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"603a8e613de33663f6326a93227cb1e07514511f0e0c20a124f3a33c63e0b0d1"} err="failed to get container status \"603a8e613de33663f6326a93227cb1e07514511f0e0c20a124f3a33c63e0b0d1\": rpc error: code = NotFound desc = could not find container \"603a8e613de33663f6326a93227cb1e07514511f0e0c20a124f3a33c63e0b0d1\": container with ID starting with 603a8e613de33663f6326a93227cb1e07514511f0e0c20a124f3a33c63e0b0d1 not found: ID does not exist"
Oct 30 19:32:36.445008 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:36.444727    4888 scope.go:122] "RemoveContainer" containerID="011d00ec587977bd8622e89c6434b2e80ccb712763ece18f9533bf46fed0c95e"
Oct 30 19:32:36.445211 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:32:36.445072    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"011d00ec587977bd8622e89c6434b2e80ccb712763ece18f9533bf46fed0c95e\": container with ID starting with 011d00ec587977bd8622e89c6434b2e80ccb712763ece18f9533bf46fed0c95e not found: ID does not exist" containerID="011d00ec587977bd8622e89c6434b2e80ccb712763ece18f9533bf46fed0c95e"
Oct 30 19:32:36.445211 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:36.445100    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"011d00ec587977bd8622e89c6434b2e80ccb712763ece18f9533bf46fed0c95e"} err="failed to get container status \"011d00ec587977bd8622e89c6434b2e80ccb712763ece18f9533bf46fed0c95e\": rpc error: code = NotFound desc = could not find container \"011d00ec587977bd8622e89c6434b2e80ccb712763ece18f9533bf46fed0c95e\": container with ID starting with 011d00ec587977bd8622e89c6434b2e80ccb712763ece18f9533bf46fed0c95e not found: ID does not exist"
Oct 30 19:32:36.470312 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:36.455536    4888 cpu_manager.go:414] "containerMap: removing container" podUID="f7045fa9-0689-4317-95fc-d3bf87806ce1" containerName="istio-proxy"
Oct 30 19:32:36.470312 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:36.455560    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f7045fa9-0689-4317-95fc-d3bf87806ce1" containerName="istio-proxy"
Oct 30 19:32:36.470312 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:36.455618    4888 cpu_manager.go:414] "containerMap: removing container" podUID="f7045fa9-0689-4317-95fc-d3bf87806ce1" containerName="istio-proxy"
Oct 30 19:32:36.470312 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:36.455625    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f7045fa9-0689-4317-95fc-d3bf87806ce1" containerName="istio-proxy"
Oct 30 19:32:36.470312 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:36.455773    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="f7045fa9-0689-4317-95fc-d3bf87806ce1" containerName="istio-proxy"
Oct 30 19:32:36.532319 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:36.508999    4888 patch_prober.go:29] interesting pod/e2e-shared-a-w4-da1b8b-openshift-default-7876fcf7c5-p4wc5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.161:15021/healthz/ready\": dial tcp 10.132.0.161:15021: connect: connection refused" start-of-body=
Oct 30 19:32:36.532319 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:36.509082    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w4-da1b8b-openshift-default-7876fcf7c5-p4wc5" podUID="73a3358c-ef67-4b35-a010-46c168a95f1e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.161:15021/healthz/ready\": dial tcp 10.132.0.161:15021: connect: connection refused"
Oct 30 19:32:36.643193 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:36.643112    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-a-9e4e63-openshift-default-5785d6dfd9-nbk9s"]
Oct 30 19:32:36.643480 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:36.643314    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-9e4e63-openshift-default-5785d6dfd9-nbk9s"
Oct 30 19:32:36.643586 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:36.643570    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-explicit-200c46-openshift-default-5f4d5cd597-t47sj"]
Oct 30 19:32:36.645486 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:36.645462    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-explicit-200c46-openshift-default-5f4d5cd597-t47sj"]
Oct 30 19:32:36.708310 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:36.698255    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-extmod-a-9e4e63-openshift-default-dockercfg-npm7w\""
Oct 30 19:32:36.744543 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:36.744500    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5cc67731-e338-42cc-8076-5806861336d9-credential-socket\") pod \"e2e-extmod-a-9e4e63-openshift-default-5785d6dfd9-nbk9s\" (UID: \"5cc67731-e338-42cc-8076-5806861336d9\") " pod="openshift-ingress/e2e-extmod-a-9e4e63-openshift-default-5785d6dfd9-nbk9s"
Oct 30 19:32:36.744712 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:36.744557    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2vghx\" (UniqueName: \"kubernetes.io/projected/5cc67731-e338-42cc-8076-5806861336d9-kube-api-access-2vghx\") pod \"e2e-extmod-a-9e4e63-openshift-default-5785d6dfd9-nbk9s\" (UID: \"5cc67731-e338-42cc-8076-5806861336d9\") " pod="openshift-ingress/e2e-extmod-a-9e4e63-openshift-default-5785d6dfd9-nbk9s"
Oct 30 19:32:36.744712 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:36.744645    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5cc67731-e338-42cc-8076-5806861336d9-istio-podinfo\") pod \"e2e-extmod-a-9e4e63-openshift-default-5785d6dfd9-nbk9s\" (UID: \"5cc67731-e338-42cc-8076-5806861336d9\") " pod="openshift-ingress/e2e-extmod-a-9e4e63-openshift-default-5785d6dfd9-nbk9s"
Oct 30 19:32:36.744712 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:36.744686    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5cc67731-e338-42cc-8076-5806861336d9-istio-data\") pod \"e2e-extmod-a-9e4e63-openshift-default-5785d6dfd9-nbk9s\" (UID: \"5cc67731-e338-42cc-8076-5806861336d9\") " pod="openshift-ingress/e2e-extmod-a-9e4e63-openshift-default-5785d6dfd9-nbk9s"
Oct 30 19:32:36.744712 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:36.744708    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5cc67731-e338-42cc-8076-5806861336d9-istiod-ca-cert\") pod \"e2e-extmod-a-9e4e63-openshift-default-5785d6dfd9-nbk9s\" (UID: \"5cc67731-e338-42cc-8076-5806861336d9\") " pod="openshift-ingress/e2e-extmod-a-9e4e63-openshift-default-5785d6dfd9-nbk9s"
Oct 30 19:32:36.744859 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:36.744793    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5cc67731-e338-42cc-8076-5806861336d9-workload-socket\") pod \"e2e-extmod-a-9e4e63-openshift-default-5785d6dfd9-nbk9s\" (UID: \"5cc67731-e338-42cc-8076-5806861336d9\") " pod="openshift-ingress/e2e-extmod-a-9e4e63-openshift-default-5785d6dfd9-nbk9s"
Oct 30 19:32:36.744859 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:36.744811    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5cc67731-e338-42cc-8076-5806861336d9-istio-envoy\") pod \"e2e-extmod-a-9e4e63-openshift-default-5785d6dfd9-nbk9s\" (UID: \"5cc67731-e338-42cc-8076-5806861336d9\") " pod="openshift-ingress/e2e-extmod-a-9e4e63-openshift-default-5785d6dfd9-nbk9s"
Oct 30 19:32:36.744925 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:36.744869    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5cc67731-e338-42cc-8076-5806861336d9-istio-token\") pod \"e2e-extmod-a-9e4e63-openshift-default-5785d6dfd9-nbk9s\" (UID: \"5cc67731-e338-42cc-8076-5806861336d9\") " pod="openshift-ingress/e2e-extmod-a-9e4e63-openshift-default-5785d6dfd9-nbk9s"
Oct 30 19:32:36.744925 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:36.744908    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5cc67731-e338-42cc-8076-5806861336d9-workload-certs\") pod \"e2e-extmod-a-9e4e63-openshift-default-5785d6dfd9-nbk9s\" (UID: \"5cc67731-e338-42cc-8076-5806861336d9\") " pod="openshift-ingress/e2e-extmod-a-9e4e63-openshift-default-5785d6dfd9-nbk9s"
Oct 30 19:32:36.845828 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:36.845794    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5cc67731-e338-42cc-8076-5806861336d9-istiod-ca-cert\") pod \"e2e-extmod-a-9e4e63-openshift-default-5785d6dfd9-nbk9s\" (UID: \"5cc67731-e338-42cc-8076-5806861336d9\") " pod="openshift-ingress/e2e-extmod-a-9e4e63-openshift-default-5785d6dfd9-nbk9s"
Oct 30 19:32:36.846028 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:36.845886    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5cc67731-e338-42cc-8076-5806861336d9-workload-socket\") pod \"e2e-extmod-a-9e4e63-openshift-default-5785d6dfd9-nbk9s\" (UID: \"5cc67731-e338-42cc-8076-5806861336d9\") " pod="openshift-ingress/e2e-extmod-a-9e4e63-openshift-default-5785d6dfd9-nbk9s"
Oct 30 19:32:36.846028 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:36.845918    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5cc67731-e338-42cc-8076-5806861336d9-istio-envoy\") pod \"e2e-extmod-a-9e4e63-openshift-default-5785d6dfd9-nbk9s\" (UID: \"5cc67731-e338-42cc-8076-5806861336d9\") " pod="openshift-ingress/e2e-extmod-a-9e4e63-openshift-default-5785d6dfd9-nbk9s"
Oct 30 19:32:36.846167 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:36.846024    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5cc67731-e338-42cc-8076-5806861336d9-istio-token\") pod \"e2e-extmod-a-9e4e63-openshift-default-5785d6dfd9-nbk9s\" (UID: \"5cc67731-e338-42cc-8076-5806861336d9\") " pod="openshift-ingress/e2e-extmod-a-9e4e63-openshift-default-5785d6dfd9-nbk9s"
Oct 30 19:32:36.846167 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:36.846091    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5cc67731-e338-42cc-8076-5806861336d9-workload-certs\") pod \"e2e-extmod-a-9e4e63-openshift-default-5785d6dfd9-nbk9s\" (UID: \"5cc67731-e338-42cc-8076-5806861336d9\") " pod="openshift-ingress/e2e-extmod-a-9e4e63-openshift-default-5785d6dfd9-nbk9s"
Oct 30 19:32:36.846295 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:36.846168    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5cc67731-e338-42cc-8076-5806861336d9-credential-socket\") pod \"e2e-extmod-a-9e4e63-openshift-default-5785d6dfd9-nbk9s\" (UID: \"5cc67731-e338-42cc-8076-5806861336d9\") " pod="openshift-ingress/e2e-extmod-a-9e4e63-openshift-default-5785d6dfd9-nbk9s"
Oct 30 19:32:36.846295 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:36.846201    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2vghx\" (UniqueName: \"kubernetes.io/projected/5cc67731-e338-42cc-8076-5806861336d9-kube-api-access-2vghx\") pod \"e2e-extmod-a-9e4e63-openshift-default-5785d6dfd9-nbk9s\" (UID: \"5cc67731-e338-42cc-8076-5806861336d9\") " pod="openshift-ingress/e2e-extmod-a-9e4e63-openshift-default-5785d6dfd9-nbk9s"
Oct 30 19:32:36.846295 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:36.846237    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5cc67731-e338-42cc-8076-5806861336d9-istio-podinfo\") pod \"e2e-extmod-a-9e4e63-openshift-default-5785d6dfd9-nbk9s\" (UID: \"5cc67731-e338-42cc-8076-5806861336d9\") " pod="openshift-ingress/e2e-extmod-a-9e4e63-openshift-default-5785d6dfd9-nbk9s"
Oct 30 19:32:36.846295 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:36.846274    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5cc67731-e338-42cc-8076-5806861336d9-istio-data\") pod \"e2e-extmod-a-9e4e63-openshift-default-5785d6dfd9-nbk9s\" (UID: \"5cc67731-e338-42cc-8076-5806861336d9\") " pod="openshift-ingress/e2e-extmod-a-9e4e63-openshift-default-5785d6dfd9-nbk9s"
Oct 30 19:32:36.846688 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:36.846666    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5cc67731-e338-42cc-8076-5806861336d9-workload-socket\") pod \"e2e-extmod-a-9e4e63-openshift-default-5785d6dfd9-nbk9s\" (UID: \"5cc67731-e338-42cc-8076-5806861336d9\") " pod="openshift-ingress/e2e-extmod-a-9e4e63-openshift-default-5785d6dfd9-nbk9s"
Oct 30 19:32:36.846791 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:36.846732    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5cc67731-e338-42cc-8076-5806861336d9-workload-certs\") pod \"e2e-extmod-a-9e4e63-openshift-default-5785d6dfd9-nbk9s\" (UID: \"5cc67731-e338-42cc-8076-5806861336d9\") " pod="openshift-ingress/e2e-extmod-a-9e4e63-openshift-default-5785d6dfd9-nbk9s"
Oct 30 19:32:36.846791 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:36.846755    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5cc67731-e338-42cc-8076-5806861336d9-istio-data\") pod \"e2e-extmod-a-9e4e63-openshift-default-5785d6dfd9-nbk9s\" (UID: \"5cc67731-e338-42cc-8076-5806861336d9\") " pod="openshift-ingress/e2e-extmod-a-9e4e63-openshift-default-5785d6dfd9-nbk9s"
Oct 30 19:32:36.846791 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:36.846780    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5cc67731-e338-42cc-8076-5806861336d9-istiod-ca-cert\") pod \"e2e-extmod-a-9e4e63-openshift-default-5785d6dfd9-nbk9s\" (UID: \"5cc67731-e338-42cc-8076-5806861336d9\") " pod="openshift-ingress/e2e-extmod-a-9e4e63-openshift-default-5785d6dfd9-nbk9s"
Oct 30 19:32:36.846953 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:36.846898    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5cc67731-e338-42cc-8076-5806861336d9-credential-socket\") pod \"e2e-extmod-a-9e4e63-openshift-default-5785d6dfd9-nbk9s\" (UID: \"5cc67731-e338-42cc-8076-5806861336d9\") " pod="openshift-ingress/e2e-extmod-a-9e4e63-openshift-default-5785d6dfd9-nbk9s"
Oct 30 19:32:36.848935 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:36.848912    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5cc67731-e338-42cc-8076-5806861336d9-istio-envoy\") pod \"e2e-extmod-a-9e4e63-openshift-default-5785d6dfd9-nbk9s\" (UID: \"5cc67731-e338-42cc-8076-5806861336d9\") " pod="openshift-ingress/e2e-extmod-a-9e4e63-openshift-default-5785d6dfd9-nbk9s"
Oct 30 19:32:36.849044 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:36.848988    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5cc67731-e338-42cc-8076-5806861336d9-istio-podinfo\") pod \"e2e-extmod-a-9e4e63-openshift-default-5785d6dfd9-nbk9s\" (UID: \"5cc67731-e338-42cc-8076-5806861336d9\") " pod="openshift-ingress/e2e-extmod-a-9e4e63-openshift-default-5785d6dfd9-nbk9s"
Oct 30 19:32:36.921929 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:36.921833    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5cc67731-e338-42cc-8076-5806861336d9-istio-token\") pod \"e2e-extmod-a-9e4e63-openshift-default-5785d6dfd9-nbk9s\" (UID: \"5cc67731-e338-42cc-8076-5806861336d9\") " pod="openshift-ingress/e2e-extmod-a-9e4e63-openshift-default-5785d6dfd9-nbk9s"
Oct 30 19:32:36.924585 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:36.924558    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2vghx\" (UniqueName: \"kubernetes.io/projected/5cc67731-e338-42cc-8076-5806861336d9-kube-api-access-2vghx\") pod \"e2e-extmod-a-9e4e63-openshift-default-5785d6dfd9-nbk9s\" (UID: \"5cc67731-e338-42cc-8076-5806861336d9\") " pod="openshift-ingress/e2e-extmod-a-9e4e63-openshift-default-5785d6dfd9-nbk9s"
Oct 30 19:32:36.961760 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:36.961733    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-9e4e63-openshift-default-5785d6dfd9-nbk9s"
Oct 30 19:32:37.113782 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:37.109813    4888 patch_prober.go:29] interesting pod/e2e-extmod-a-7fdf6b-openshift-default-65b5b7b9b5-8q2gn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.163:15021/healthz/ready\": dial tcp 10.132.0.163:15021: connect: connection refused" start-of-body=
Oct 30 19:32:37.113782 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:37.109870    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-7fdf6b-openshift-default-65b5b7b9b5-8q2gn" podUID="6adbce7d-d243-4f53-b2f0-e1db8b942eb1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.163:15021/healthz/ready\": dial tcp 10.132.0.163:15021: connect: connection refused"
Oct 30 19:32:37.456629 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:37.456591    4888 patch_prober.go:29] interesting pod/e2e-shared-a-w4-da1b8b-openshift-default-7876fcf7c5-p4wc5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.161:15021/healthz/ready\": dial tcp 10.132.0.161:15021: connect: connection refused" start-of-body=
Oct 30 19:32:37.456844 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:37.456640    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w4-da1b8b-openshift-default-7876fcf7c5-p4wc5" podUID="73a3358c-ef67-4b35-a010-46c168a95f1e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.161:15021/healthz/ready\": dial tcp 10.132.0.161:15021: connect: connection refused"
Oct 30 19:32:37.518307 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:37.517391    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-a-9e4e63-openshift-default-5785d6dfd9-nbk9s"]
Oct 30 19:32:37.533857 ip-10-0-134-46 kubenswrapper[4888]: W1030 19:32:37.533822    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5cc67731_e338_42cc_8076_5806861336d9.slice/crio-e454f1305396fb56b63bdcfaa72cce98a5b53d630e9ce97f03d828dda8966b93 WatchSource:0}: Error finding container e454f1305396fb56b63bdcfaa72cce98a5b53d630e9ce97f03d828dda8966b93: Status 404 returned error can't find the container with id e454f1305396fb56b63bdcfaa72cce98a5b53d630e9ce97f03d828dda8966b93
Oct 30 19:32:37.536589 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:37.536561    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:32:37.536660 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:37.536625    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:32:37.536660 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:37.536657    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:32:37.931310 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:37.931168    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f7045fa9-0689-4317-95fc-d3bf87806ce1" path="/var/lib/kubelet/pods/f7045fa9-0689-4317-95fc-d3bf87806ce1/volumes"
Oct 30 19:32:38.108310 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:38.107410    4888 patch_prober.go:29] interesting pod/e2e-extmod-a-7fdf6b-openshift-default-65b5b7b9b5-8q2gn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.163:15021/healthz/ready\": dial tcp 10.132.0.163:15021: connect: connection refused" start-of-body=
Oct 30 19:32:38.108310 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:38.107482    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-7fdf6b-openshift-default-65b5b7b9b5-8q2gn" podUID="6adbce7d-d243-4f53-b2f0-e1db8b942eb1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.163:15021/healthz/ready\": dial tcp 10.132.0.163:15021: connect: connection refused"
Oct 30 19:32:38.424369 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:38.424337    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-9e4e63-openshift-default-5785d6dfd9-nbk9s" event={"ID":"5cc67731-e338-42cc-8076-5806861336d9","Type":"ContainerStarted","Data":"48dd6a1cb16f3c9c13e8f9359ee07c0da725624613312a0452443c8bc3383aec"}
Oct 30 19:32:38.424623 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:38.424608    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-9e4e63-openshift-default-5785d6dfd9-nbk9s" event={"ID":"5cc67731-e338-42cc-8076-5806861336d9","Type":"ContainerStarted","Data":"e454f1305396fb56b63bdcfaa72cce98a5b53d630e9ce97f03d828dda8966b93"}
Oct 30 19:32:38.456354 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:38.456327    4888 patch_prober.go:29] interesting pod/e2e-shared-a-w4-da1b8b-openshift-default-7876fcf7c5-p4wc5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.161:15021/healthz/ready\": dial tcp 10.132.0.161:15021: connect: connection refused" start-of-body=
Oct 30 19:32:38.456516 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:38.456372    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w4-da1b8b-openshift-default-7876fcf7c5-p4wc5" podUID="73a3358c-ef67-4b35-a010-46c168a95f1e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.161:15021/healthz/ready\": dial tcp 10.132.0.161:15021: connect: connection refused"
Oct 30 19:32:38.584310 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:38.580112    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-extmod-a-9e4e63-openshift-default-5785d6dfd9-nbk9s" podStartSLOduration=2.580092383 podStartE2EDuration="2.580092383s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:32:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:32:38.566832114 +0000 UTC m=+3522.814238142" watchObservedRunningTime="2026-10-30 19:32:38.580092383 +0000 UTC m=+3522.827498410"
Oct 30 19:32:38.962014 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:38.961977    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-extmod-a-9e4e63-openshift-default-5785d6dfd9-nbk9s"
Oct 30 19:32:38.985312 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:38.977585    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-extmod-a-9e4e63-openshift-default-5785d6dfd9-nbk9s"
Oct 30 19:32:39.119265 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:39.119237    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-extmod-a-7fdf6b-openshift-default-65b5b7b9b5-8q2gn"
Oct 30 19:32:39.119756 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:39.119737    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-extmod-a-7fdf6b-openshift-default-65b5b7b9b5-8q2gn"
Oct 30 19:32:39.120565 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:39.120543    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-extmod-a-7fdf6b-openshift-default-65b5b7b9b5-8q2gn"
Oct 30 19:32:39.435270 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:39.435245    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-extmod-a-9e4e63-openshift-default-5785d6dfd9-nbk9s"
Oct 30 19:32:39.436622 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:39.436600    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-extmod-a-9e4e63-openshift-default-5785d6dfd9-nbk9s"
Oct 30 19:32:39.456398 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:39.456373    4888 patch_prober.go:29] interesting pod/e2e-shared-a-w4-da1b8b-openshift-default-7876fcf7c5-p4wc5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.161:15021/healthz/ready\": dial tcp 10.132.0.161:15021: connect: connection refused" start-of-body=
Oct 30 19:32:39.456551 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:39.456418    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w4-da1b8b-openshift-default-7876fcf7c5-p4wc5" podUID="73a3358c-ef67-4b35-a010-46c168a95f1e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.161:15021/healthz/ready\": dial tcp 10.132.0.161:15021: connect: connection refused"
Oct 30 19:32:40.460947 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:40.460921    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w4-da1b8b-openshift-default-7876fcf7c5-p4wc5"
Oct 30 19:32:40.461914 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:40.461894    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w4-da1b8b-openshift-default-7876fcf7c5-p4wc5"
Oct 30 19:32:45.180982 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:45.180948    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-9e4e63-6bb4cfdc5-5mxz7"]
Oct 30 19:32:45.181653 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:45.181638    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="f7045fa9-0689-4317-95fc-d3bf87806ce1" containerName="istio-proxy"
Oct 30 19:32:45.184501 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:45.184472    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-9e4e63-6bb4cfdc5-5mxz7"
Oct 30 19:32:45.315308 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:45.311067    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-extmod-a-9e4e63-dockercfg-nsbgf\""
Oct 30 19:32:45.316069 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:45.316013    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-extmod-a-9e4e63\""
Oct 30 19:32:45.317325 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:45.294385    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/63b61d8b-43d3-439f-bc73-639e205a90d1-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-a-9e4e63-6bb4cfdc5-5mxz7\" (UID: \"63b61d8b-43d3-439f-bc73-639e205a90d1\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-9e4e63-6bb4cfdc5-5mxz7"
Oct 30 19:32:45.317542 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:45.317520    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8d7q2\" (UniqueName: \"kubernetes.io/projected/63b61d8b-43d3-439f-bc73-639e205a90d1-kube-api-access-8d7q2\") pod \"payload-pre-processing-e2e-extmod-a-9e4e63-6bb4cfdc5-5mxz7\" (UID: \"63b61d8b-43d3-439f-bc73-639e205a90d1\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-9e4e63-6bb4cfdc5-5mxz7"
Oct 30 19:32:45.322482 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:45.322445    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-9e4e63-6bb4cfdc5-5mxz7"]
Oct 30 19:32:45.348965 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:45.348930    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-9e4e63-fbff868b4-hx2k4"]
Oct 30 19:32:45.355570 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:45.355543    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-a-9e4e63-fbff868b4-hx2k4"
Oct 30 19:32:45.394226 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:45.394187    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-9e4e63-fbff868b4-hx2k4"]
Oct 30 19:32:45.418634 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:45.418604    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cppjd\" (UniqueName: \"kubernetes.io/projected/0b08bd02-8702-44a6-819c-c7f9d044a826-kube-api-access-cppjd\") pod \"payload-processing-e2e-extmod-a-9e4e63-fbff868b4-hx2k4\" (UID: \"0b08bd02-8702-44a6-819c-c7f9d044a826\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-9e4e63-fbff868b4-hx2k4"
Oct 30 19:32:45.418822 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:45.418684    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/63b61d8b-43d3-439f-bc73-639e205a90d1-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-a-9e4e63-6bb4cfdc5-5mxz7\" (UID: \"63b61d8b-43d3-439f-bc73-639e205a90d1\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-9e4e63-6bb4cfdc5-5mxz7"
Oct 30 19:32:45.418822 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:45.418748    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8d7q2\" (UniqueName: \"kubernetes.io/projected/63b61d8b-43d3-439f-bc73-639e205a90d1-kube-api-access-8d7q2\") pod \"payload-pre-processing-e2e-extmod-a-9e4e63-6bb4cfdc5-5mxz7\" (UID: \"63b61d8b-43d3-439f-bc73-639e205a90d1\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-9e4e63-6bb4cfdc5-5mxz7"
Oct 30 19:32:45.418910 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:45.418861    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0b08bd02-8702-44a6-819c-c7f9d044a826-plugins-config-volume\") pod \"payload-processing-e2e-extmod-a-9e4e63-fbff868b4-hx2k4\" (UID: \"0b08bd02-8702-44a6-819c-c7f9d044a826\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-9e4e63-fbff868b4-hx2k4"
Oct 30 19:32:45.419490 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:45.419474    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/63b61d8b-43d3-439f-bc73-639e205a90d1-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-a-9e4e63-6bb4cfdc5-5mxz7\" (UID: \"63b61d8b-43d3-439f-bc73-639e205a90d1\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-9e4e63-6bb4cfdc5-5mxz7"
Oct 30 19:32:45.481861 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:45.481795    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8d7q2\" (UniqueName: \"kubernetes.io/projected/63b61d8b-43d3-439f-bc73-639e205a90d1-kube-api-access-8d7q2\") pod \"payload-pre-processing-e2e-extmod-a-9e4e63-6bb4cfdc5-5mxz7\" (UID: \"63b61d8b-43d3-439f-bc73-639e205a90d1\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-9e4e63-6bb4cfdc5-5mxz7"
Oct 30 19:32:45.497907 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:45.497884    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-9e4e63-6bb4cfdc5-5mxz7"
Oct 30 19:32:45.519715 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:45.519688    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0b08bd02-8702-44a6-819c-c7f9d044a826-plugins-config-volume\") pod \"payload-processing-e2e-extmod-a-9e4e63-fbff868b4-hx2k4\" (UID: \"0b08bd02-8702-44a6-819c-c7f9d044a826\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-9e4e63-fbff868b4-hx2k4"
Oct 30 19:32:45.519845 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:45.519778    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cppjd\" (UniqueName: \"kubernetes.io/projected/0b08bd02-8702-44a6-819c-c7f9d044a826-kube-api-access-cppjd\") pod \"payload-processing-e2e-extmod-a-9e4e63-fbff868b4-hx2k4\" (UID: \"0b08bd02-8702-44a6-819c-c7f9d044a826\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-9e4e63-fbff868b4-hx2k4"
Oct 30 19:32:45.520593 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:45.520570    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0b08bd02-8702-44a6-819c-c7f9d044a826-plugins-config-volume\") pod \"payload-processing-e2e-extmod-a-9e4e63-fbff868b4-hx2k4\" (UID: \"0b08bd02-8702-44a6-819c-c7f9d044a826\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-9e4e63-fbff868b4-hx2k4"
Oct 30 19:32:45.599070 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:45.592177    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cppjd\" (UniqueName: \"kubernetes.io/projected/0b08bd02-8702-44a6-819c-c7f9d044a826-kube-api-access-cppjd\") pod \"payload-processing-e2e-extmod-a-9e4e63-fbff868b4-hx2k4\" (UID: \"0b08bd02-8702-44a6-819c-c7f9d044a826\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-9e4e63-fbff868b4-hx2k4"
Oct 30 19:32:45.677995 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:45.677967    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-a-9e4e63-fbff868b4-hx2k4"
Oct 30 19:32:45.895009 ip-10-0-134-46 kubenswrapper[4888]: W1030 19:32:45.894977    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod63b61d8b_43d3_439f_bc73_639e205a90d1.slice/crio-26e92411bf8a6d216712acd8717705bc2ff82e104def7f1a545200c4eafc4ed8 WatchSource:0}: Error finding container 26e92411bf8a6d216712acd8717705bc2ff82e104def7f1a545200c4eafc4ed8: Status 404 returned error can't find the container with id 26e92411bf8a6d216712acd8717705bc2ff82e104def7f1a545200c4eafc4ed8
Oct 30 19:32:45.896195 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:45.896171    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-9e4e63-6bb4cfdc5-5mxz7"]
Oct 30 19:32:46.056804 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:46.056767    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-9e4e63-fbff868b4-hx2k4"]
Oct 30 19:32:46.057463 ip-10-0-134-46 kubenswrapper[4888]: W1030 19:32:46.057438    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b08bd02_8702_44a6_819c_c7f9d044a826.slice/crio-f871eeea688140f48c7580d507130d55790aa598e266a91ff65b4f958da5e574 WatchSource:0}: Error finding container f871eeea688140f48c7580d507130d55790aa598e266a91ff65b4f958da5e574: Status 404 returned error can't find the container with id f871eeea688140f48c7580d507130d55790aa598e266a91ff65b4f958da5e574
Oct 30 19:32:46.484170 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:46.484134    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-9e4e63-6bb4cfdc5-5mxz7" event={"ID":"63b61d8b-43d3-439f-bc73-639e205a90d1","Type":"ContainerStarted","Data":"a6706f7b4e48f4af4ede9d48f5649040ac4f75defcf15530a018f83054e9b66f"}
Oct 30 19:32:46.484681 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:46.484200    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-9e4e63-6bb4cfdc5-5mxz7" event={"ID":"63b61d8b-43d3-439f-bc73-639e205a90d1","Type":"ContainerStarted","Data":"26e92411bf8a6d216712acd8717705bc2ff82e104def7f1a545200c4eafc4ed8"}
Oct 30 19:32:46.484681 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:46.484233    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-9e4e63-6bb4cfdc5-5mxz7"
Oct 30 19:32:46.486464 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:46.486421    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-a-9e4e63-fbff868b4-hx2k4" event={"ID":"0b08bd02-8702-44a6-819c-c7f9d044a826","Type":"ContainerStarted","Data":"c543c4eaacb0d508d6be0d746f4ceba53c12292c2b6a6644899b229ffe2d2706"}
Oct 30 19:32:46.486591 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:46.486468    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-a-9e4e63-fbff868b4-hx2k4" event={"ID":"0b08bd02-8702-44a6-819c-c7f9d044a826","Type":"ContainerStarted","Data":"f871eeea688140f48c7580d507130d55790aa598e266a91ff65b4f958da5e574"}
Oct 30 19:32:46.486591 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:46.486548    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-extmod-a-9e4e63-fbff868b4-hx2k4"
Oct 30 19:32:46.605089 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:46.605039    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-9e4e63-6bb4cfdc5-5mxz7" podStartSLOduration=1.605026045 podStartE2EDuration="1.605026045s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:32:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:32:46.603027225 +0000 UTC m=+3530.850433249" watchObservedRunningTime="2026-10-30 19:32:46.605026045 +0000 UTC m=+3530.852432070"
Oct 30 19:32:46.726585 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:46.726526    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-extmod-a-9e4e63-fbff868b4-hx2k4" podStartSLOduration=1.726505967 podStartE2EDuration="1.726505967s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:32:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:32:46.725841144 +0000 UTC m=+3530.973247176" watchObservedRunningTime="2026-10-30 19:32:46.726505967 +0000 UTC m=+3530.973911993"
Oct 30 19:32:52.814596 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:52.814563    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-rjmdz"]
Oct 30 19:32:57.512794 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:57.512768    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-extmod-a-9e4e63-fbff868b4-hx2k4"
Oct 30 19:32:57.513222 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:32:57.512837    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-9e4e63-6bb4cfdc5-5mxz7"
Oct 30 19:33:05.618549 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:33:05.618462    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-jbsw6" containerName="extractor"
Oct 30 19:33:14.511911 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:33:14.511881    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w4-0f83f5-55dd878599-fk8l7"]
Oct 30 19:33:14.515802 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:33:14.515787    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w4-0f83f5-55dd878599-fk8l7"
Oct 30 19:33:14.574693 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:33:14.574662    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w4-0f83f5-55dd878599-fk8l7"]
Oct 30 19:33:14.606663 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:33:14.606569    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cdvkd\" (UniqueName: \"kubernetes.io/projected/3f801255-a7eb-4a5f-b80f-82c3c0a6a8a6-kube-api-access-cdvkd\") pod \"payload-pre-processing-e2e-shared-b-w4-0f83f5-55dd878599-fk8l7\" (UID: \"3f801255-a7eb-4a5f-b80f-82c3c0a6a8a6\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w4-0f83f5-55dd878599-fk8l7"
Oct 30 19:33:14.606850 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:33:14.606822    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3f801255-a7eb-4a5f-b80f-82c3c0a6a8a6-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w4-0f83f5-55dd878599-fk8l7\" (UID: \"3f801255-a7eb-4a5f-b80f-82c3c0a6a8a6\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w4-0f83f5-55dd878599-fk8l7"
Oct 30 19:33:14.620988 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:33:14.620964    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w4-0f83f5\""
Oct 30 19:33:14.712651 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:33:14.712614    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3f801255-a7eb-4a5f-b80f-82c3c0a6a8a6-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w4-0f83f5-55dd878599-fk8l7\" (UID: \"3f801255-a7eb-4a5f-b80f-82c3c0a6a8a6\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w4-0f83f5-55dd878599-fk8l7"
Oct 30 19:33:14.712843 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:33:14.712797    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cdvkd\" (UniqueName: \"kubernetes.io/projected/3f801255-a7eb-4a5f-b80f-82c3c0a6a8a6-kube-api-access-cdvkd\") pod \"payload-pre-processing-e2e-shared-b-w4-0f83f5-55dd878599-fk8l7\" (UID: \"3f801255-a7eb-4a5f-b80f-82c3c0a6a8a6\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w4-0f83f5-55dd878599-fk8l7"
Oct 30 19:33:14.714085 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:33:14.714060    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3f801255-a7eb-4a5f-b80f-82c3c0a6a8a6-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w4-0f83f5-55dd878599-fk8l7\" (UID: \"3f801255-a7eb-4a5f-b80f-82c3c0a6a8a6\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w4-0f83f5-55dd878599-fk8l7"
Oct 30 19:33:14.778084 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:33:14.778011    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cdvkd\" (UniqueName: \"kubernetes.io/projected/3f801255-a7eb-4a5f-b80f-82c3c0a6a8a6-kube-api-access-cdvkd\") pod \"payload-pre-processing-e2e-shared-b-w4-0f83f5-55dd878599-fk8l7\" (UID: \"3f801255-a7eb-4a5f-b80f-82c3c0a6a8a6\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w4-0f83f5-55dd878599-fk8l7"
Oct 30 19:33:14.829394 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:33:14.829367    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w4-0f83f5-55dd878599-fk8l7"
Oct 30 19:33:15.224782 ip-10-0-134-46 kubenswrapper[4888]: W1030 19:33:15.224751    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f801255_a7eb_4a5f_b80f_82c3c0a6a8a6.slice/crio-fd347c74b1b874afb380a2b3cf3787d8d80dc5645e78d5bf193eac0fa16742d8 WatchSource:0}: Error finding container fd347c74b1b874afb380a2b3cf3787d8d80dc5645e78d5bf193eac0fa16742d8: Status 404 returned error can't find the container with id fd347c74b1b874afb380a2b3cf3787d8d80dc5645e78d5bf193eac0fa16742d8
Oct 30 19:33:15.225653 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:33:15.225624    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w4-0f83f5-55dd878599-fk8l7"]
Oct 30 19:33:15.723883 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:33:15.723825    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w4-0f83f5-55dd878599-fk8l7" event={"ID":"3f801255-a7eb-4a5f-b80f-82c3c0a6a8a6","Type":"ContainerStarted","Data":"9d491dd29d68775c97dcbc17ecc25e2230b972be1d98680c14c2baf555b0402e"}
Oct 30 19:33:15.724365 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:33:15.723899    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w4-0f83f5-55dd878599-fk8l7"
Oct 30 19:33:15.724365 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:33:15.723923    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w4-0f83f5-55dd878599-fk8l7" event={"ID":"3f801255-a7eb-4a5f-b80f-82c3c0a6a8a6","Type":"ContainerStarted","Data":"fd347c74b1b874afb380a2b3cf3787d8d80dc5645e78d5bf193eac0fa16742d8"}
Oct 30 19:33:15.849495 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:33:15.849438    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w4-0f83f5-55dd878599-fk8l7" podStartSLOduration=1.8494198590000002 podStartE2EDuration="1.849419859s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:33:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:33:15.846787964 +0000 UTC m=+3560.094193990" watchObservedRunningTime="2026-10-30 19:33:15.849419859 +0000 UTC m=+3560.096825888"
Oct 30 19:33:19.098319 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:33:19.097914    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-extmod-b-185e8b-openshift-default-59ff49bb5c-tflbd"]
Oct 30 19:33:19.111130 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:33:19.111102    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-185e8b-openshift-default-59ff49bb5c-tflbd"
Oct 30 19:33:19.168111 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:33:19.168070    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6381d3a5-bf3a-454a-b888-3aaea17c86f4-istio-token\") pod \"e2e-extmod-b-185e8b-openshift-default-59ff49bb5c-tflbd\" (UID: \"6381d3a5-bf3a-454a-b888-3aaea17c86f4\") " pod="openshift-ingress/e2e-extmod-b-185e8b-openshift-default-59ff49bb5c-tflbd"
Oct 30 19:33:19.168482 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:33:19.168124    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6381d3a5-bf3a-454a-b888-3aaea17c86f4-istiod-ca-cert\") pod \"e2e-extmod-b-185e8b-openshift-default-59ff49bb5c-tflbd\" (UID: \"6381d3a5-bf3a-454a-b888-3aaea17c86f4\") " pod="openshift-ingress/e2e-extmod-b-185e8b-openshift-default-59ff49bb5c-tflbd"
Oct 30 19:33:19.168482 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:33:19.168154    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6381d3a5-bf3a-454a-b888-3aaea17c86f4-workload-certs\") pod \"e2e-extmod-b-185e8b-openshift-default-59ff49bb5c-tflbd\" (UID: \"6381d3a5-bf3a-454a-b888-3aaea17c86f4\") " pod="openshift-ingress/e2e-extmod-b-185e8b-openshift-default-59ff49bb5c-tflbd"
Oct 30 19:33:19.168482 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:33:19.168223    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6381d3a5-bf3a-454a-b888-3aaea17c86f4-credential-socket\") pod \"e2e-extmod-b-185e8b-openshift-default-59ff49bb5c-tflbd\" (UID: \"6381d3a5-bf3a-454a-b888-3aaea17c86f4\") " pod="openshift-ingress/e2e-extmod-b-185e8b-openshift-default-59ff49bb5c-tflbd"
Oct 30 19:33:19.168482 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:33:19.168312    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6381d3a5-bf3a-454a-b888-3aaea17c86f4-istio-data\") pod \"e2e-extmod-b-185e8b-openshift-default-59ff49bb5c-tflbd\" (UID: \"6381d3a5-bf3a-454a-b888-3aaea17c86f4\") " pod="openshift-ingress/e2e-extmod-b-185e8b-openshift-default-59ff49bb5c-tflbd"
Oct 30 19:33:19.168482 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:33:19.168352    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6381d3a5-bf3a-454a-b888-3aaea17c86f4-workload-socket\") pod \"e2e-extmod-b-185e8b-openshift-default-59ff49bb5c-tflbd\" (UID: \"6381d3a5-bf3a-454a-b888-3aaea17c86f4\") " pod="openshift-ingress/e2e-extmod-b-185e8b-openshift-default-59ff49bb5c-tflbd"
Oct 30 19:33:19.168482 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:33:19.168374    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6381d3a5-bf3a-454a-b888-3aaea17c86f4-istio-envoy\") pod \"e2e-extmod-b-185e8b-openshift-default-59ff49bb5c-tflbd\" (UID: \"6381d3a5-bf3a-454a-b888-3aaea17c86f4\") " pod="openshift-ingress/e2e-extmod-b-185e8b-openshift-default-59ff49bb5c-tflbd"
Oct 30 19:33:19.168482 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:33:19.168416    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xz9g9\" (UniqueName: \"kubernetes.io/projected/6381d3a5-bf3a-454a-b888-3aaea17c86f4-kube-api-access-xz9g9\") pod \"e2e-extmod-b-185e8b-openshift-default-59ff49bb5c-tflbd\" (UID: \"6381d3a5-bf3a-454a-b888-3aaea17c86f4\") " pod="openshift-ingress/e2e-extmod-b-185e8b-openshift-default-59ff49bb5c-tflbd"
Oct 30 19:33:19.168936 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:33:19.168488    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6381d3a5-bf3a-454a-b888-3aaea17c86f4-istio-podinfo\") pod \"e2e-extmod-b-185e8b-openshift-default-59ff49bb5c-tflbd\" (UID: \"6381d3a5-bf3a-454a-b888-3aaea17c86f4\") " pod="openshift-ingress/e2e-extmod-b-185e8b-openshift-default-59ff49bb5c-tflbd"
Oct 30 19:33:19.174819 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:33:19.174783    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-b-185e8b-openshift-default-59ff49bb5c-tflbd"]
Oct 30 19:33:19.223573 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:33:19.220800    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-extmod-b-185e8b-openshift-default-dockercfg-hzsdb\""
Oct 30 19:33:19.270077 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:33:19.270047    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6381d3a5-bf3a-454a-b888-3aaea17c86f4-istio-data\") pod \"e2e-extmod-b-185e8b-openshift-default-59ff49bb5c-tflbd\" (UID: \"6381d3a5-bf3a-454a-b888-3aaea17c86f4\") " pod="openshift-ingress/e2e-extmod-b-185e8b-openshift-default-59ff49bb5c-tflbd"
Oct 30 19:33:19.270311 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:33:19.270098    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6381d3a5-bf3a-454a-b888-3aaea17c86f4-workload-socket\") pod \"e2e-extmod-b-185e8b-openshift-default-59ff49bb5c-tflbd\" (UID: \"6381d3a5-bf3a-454a-b888-3aaea17c86f4\") " pod="openshift-ingress/e2e-extmod-b-185e8b-openshift-default-59ff49bb5c-tflbd"
Oct 30 19:33:19.270311 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:33:19.270133    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6381d3a5-bf3a-454a-b888-3aaea17c86f4-istio-envoy\") pod \"e2e-extmod-b-185e8b-openshift-default-59ff49bb5c-tflbd\" (UID: \"6381d3a5-bf3a-454a-b888-3aaea17c86f4\") " pod="openshift-ingress/e2e-extmod-b-185e8b-openshift-default-59ff49bb5c-tflbd"
Oct 30 19:33:19.270311 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:33:19.270169    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xz9g9\" (UniqueName: \"kubernetes.io/projected/6381d3a5-bf3a-454a-b888-3aaea17c86f4-kube-api-access-xz9g9\") pod \"e2e-extmod-b-185e8b-openshift-default-59ff49bb5c-tflbd\" (UID: \"6381d3a5-bf3a-454a-b888-3aaea17c86f4\") " pod="openshift-ingress/e2e-extmod-b-185e8b-openshift-default-59ff49bb5c-tflbd"
Oct 30 19:33:19.270311 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:33:19.270235    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6381d3a5-bf3a-454a-b888-3aaea17c86f4-istio-podinfo\") pod \"e2e-extmod-b-185e8b-openshift-default-59ff49bb5c-tflbd\" (UID: \"6381d3a5-bf3a-454a-b888-3aaea17c86f4\") " pod="openshift-ingress/e2e-extmod-b-185e8b-openshift-default-59ff49bb5c-tflbd"
Oct 30 19:33:19.270589 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:33:19.270547    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6381d3a5-bf3a-454a-b888-3aaea17c86f4-workload-socket\") pod \"e2e-extmod-b-185e8b-openshift-default-59ff49bb5c-tflbd\" (UID: \"6381d3a5-bf3a-454a-b888-3aaea17c86f4\") " pod="openshift-ingress/e2e-extmod-b-185e8b-openshift-default-59ff49bb5c-tflbd"
Oct 30 19:33:19.270589 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:33:19.270579    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6381d3a5-bf3a-454a-b888-3aaea17c86f4-istio-data\") pod \"e2e-extmod-b-185e8b-openshift-default-59ff49bb5c-tflbd\" (UID: \"6381d3a5-bf3a-454a-b888-3aaea17c86f4\") " pod="openshift-ingress/e2e-extmod-b-185e8b-openshift-default-59ff49bb5c-tflbd"
Oct 30 19:33:19.270828 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:33:19.270806    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6381d3a5-bf3a-454a-b888-3aaea17c86f4-istio-token\") pod \"e2e-extmod-b-185e8b-openshift-default-59ff49bb5c-tflbd\" (UID: \"6381d3a5-bf3a-454a-b888-3aaea17c86f4\") " pod="openshift-ingress/e2e-extmod-b-185e8b-openshift-default-59ff49bb5c-tflbd"
Oct 30 19:33:19.270906 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:33:19.270843    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6381d3a5-bf3a-454a-b888-3aaea17c86f4-istiod-ca-cert\") pod \"e2e-extmod-b-185e8b-openshift-default-59ff49bb5c-tflbd\" (UID: \"6381d3a5-bf3a-454a-b888-3aaea17c86f4\") " pod="openshift-ingress/e2e-extmod-b-185e8b-openshift-default-59ff49bb5c-tflbd"
Oct 30 19:33:19.270906 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:33:19.270872    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6381d3a5-bf3a-454a-b888-3aaea17c86f4-workload-certs\") pod \"e2e-extmod-b-185e8b-openshift-default-59ff49bb5c-tflbd\" (UID: \"6381d3a5-bf3a-454a-b888-3aaea17c86f4\") " pod="openshift-ingress/e2e-extmod-b-185e8b-openshift-default-59ff49bb5c-tflbd"
Oct 30 19:33:19.271011 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:33:19.270945    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6381d3a5-bf3a-454a-b888-3aaea17c86f4-credential-socket\") pod \"e2e-extmod-b-185e8b-openshift-default-59ff49bb5c-tflbd\" (UID: \"6381d3a5-bf3a-454a-b888-3aaea17c86f4\") " pod="openshift-ingress/e2e-extmod-b-185e8b-openshift-default-59ff49bb5c-tflbd"
Oct 30 19:33:19.271319 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:33:19.271260    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6381d3a5-bf3a-454a-b888-3aaea17c86f4-credential-socket\") pod \"e2e-extmod-b-185e8b-openshift-default-59ff49bb5c-tflbd\" (UID: \"6381d3a5-bf3a-454a-b888-3aaea17c86f4\") " pod="openshift-ingress/e2e-extmod-b-185e8b-openshift-default-59ff49bb5c-tflbd"
Oct 30 19:33:19.271475 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:33:19.271451    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6381d3a5-bf3a-454a-b888-3aaea17c86f4-workload-certs\") pod \"e2e-extmod-b-185e8b-openshift-default-59ff49bb5c-tflbd\" (UID: \"6381d3a5-bf3a-454a-b888-3aaea17c86f4\") " pod="openshift-ingress/e2e-extmod-b-185e8b-openshift-default-59ff49bb5c-tflbd"
Oct 30 19:33:19.271769 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:33:19.271747    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6381d3a5-bf3a-454a-b888-3aaea17c86f4-istiod-ca-cert\") pod \"e2e-extmod-b-185e8b-openshift-default-59ff49bb5c-tflbd\" (UID: \"6381d3a5-bf3a-454a-b888-3aaea17c86f4\") " pod="openshift-ingress/e2e-extmod-b-185e8b-openshift-default-59ff49bb5c-tflbd"
Oct 30 19:33:19.273401 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:33:19.273380    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6381d3a5-bf3a-454a-b888-3aaea17c86f4-istio-envoy\") pod \"e2e-extmod-b-185e8b-openshift-default-59ff49bb5c-tflbd\" (UID: \"6381d3a5-bf3a-454a-b888-3aaea17c86f4\") " pod="openshift-ingress/e2e-extmod-b-185e8b-openshift-default-59ff49bb5c-tflbd"
Oct 30 19:33:19.273605 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:33:19.273584    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6381d3a5-bf3a-454a-b888-3aaea17c86f4-istio-podinfo\") pod \"e2e-extmod-b-185e8b-openshift-default-59ff49bb5c-tflbd\" (UID: \"6381d3a5-bf3a-454a-b888-3aaea17c86f4\") " pod="openshift-ingress/e2e-extmod-b-185e8b-openshift-default-59ff49bb5c-tflbd"
Oct 30 19:33:19.344947 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:33:19.344917    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6381d3a5-bf3a-454a-b888-3aaea17c86f4-istio-token\") pod \"e2e-extmod-b-185e8b-openshift-default-59ff49bb5c-tflbd\" (UID: \"6381d3a5-bf3a-454a-b888-3aaea17c86f4\") " pod="openshift-ingress/e2e-extmod-b-185e8b-openshift-default-59ff49bb5c-tflbd"
Oct 30 19:33:19.345658 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:33:19.345639    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xz9g9\" (UniqueName: \"kubernetes.io/projected/6381d3a5-bf3a-454a-b888-3aaea17c86f4-kube-api-access-xz9g9\") pod \"e2e-extmod-b-185e8b-openshift-default-59ff49bb5c-tflbd\" (UID: \"6381d3a5-bf3a-454a-b888-3aaea17c86f4\") " pod="openshift-ingress/e2e-extmod-b-185e8b-openshift-default-59ff49bb5c-tflbd"
Oct 30 19:33:19.445610 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:33:19.445519    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-185e8b-openshift-default-59ff49bb5c-tflbd"
Oct 30 19:33:19.892352 ip-10-0-134-46 kubenswrapper[4888]: W1030 19:33:19.892325    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6381d3a5_bf3a_454a_b888_3aaea17c86f4.slice/crio-bbeddbfa3f43f67071e7f0148010beda68d53cf101dbe41b7d494e5bbf748400 WatchSource:0}: Error finding container bbeddbfa3f43f67071e7f0148010beda68d53cf101dbe41b7d494e5bbf748400: Status 404 returned error can't find the container with id bbeddbfa3f43f67071e7f0148010beda68d53cf101dbe41b7d494e5bbf748400
Oct 30 19:33:19.894989 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:33:19.894955    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:33:19.895092 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:33:19.895019    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:33:19.895092 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:33:19.895048    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:33:19.896138 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:33:19.896116    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-b-185e8b-openshift-default-59ff49bb5c-tflbd"]
Oct 30 19:33:20.758712 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:33:20.758677    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-185e8b-openshift-default-59ff49bb5c-tflbd" event={"ID":"6381d3a5-bf3a-454a-b888-3aaea17c86f4","Type":"ContainerStarted","Data":"fc7ae7314397c53e7e758a8ca85c7cdea891a75a12d8f00e1610b36d04b6cc28"}
Oct 30 19:33:20.759204 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:33:20.758741    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-185e8b-openshift-default-59ff49bb5c-tflbd" event={"ID":"6381d3a5-bf3a-454a-b888-3aaea17c86f4","Type":"ContainerStarted","Data":"bbeddbfa3f43f67071e7f0148010beda68d53cf101dbe41b7d494e5bbf748400"}
Oct 30 19:33:20.932740 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:33:20.932694    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-extmod-b-185e8b-openshift-default-59ff49bb5c-tflbd" podStartSLOduration=2.932680559 podStartE2EDuration="2.932680559s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:33:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:33:20.932021242 +0000 UTC m=+3565.179427266" watchObservedRunningTime="2026-10-30 19:33:20.932680559 +0000 UTC m=+3565.180086584"
Oct 30 19:33:21.446338 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:33:21.446303    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-extmod-b-185e8b-openshift-default-59ff49bb5c-tflbd"
Oct 30 19:33:21.452161 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:33:21.452138    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-extmod-b-185e8b-openshift-default-59ff49bb5c-tflbd"
Oct 30 19:33:21.766960 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:33:21.766886    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-extmod-b-185e8b-openshift-default-59ff49bb5c-tflbd"
Oct 30 19:33:21.767913 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:33:21.767891    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-extmod-b-185e8b-openshift-default-59ff49bb5c-tflbd"
Oct 30 19:33:26.733754 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:33:26.733726    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w4-0f83f5-55dd878599-fk8l7"
Oct 30 19:33:27.798898 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:33:27.798869    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-185e8b-98bb8ccf5-lcntb"]
Oct 30 19:33:27.803507 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:33:27.803478    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-185e8b-98bb8ccf5-lcntb"
Oct 30 19:33:27.860547 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:33:27.860519    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-extmod-b-185e8b-dockercfg-6xj6c\""
Oct 30 19:33:27.861297 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:33:27.861265    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-extmod-b-185e8b\""
Oct 30 19:33:27.866227 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:33:27.866202    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-185e8b-98bb8ccf5-lcntb"]
Oct 30 19:33:27.993227 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:33:27.993194    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/56a9a3f1-852b-458f-979f-1b815f8175d7-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-b-185e8b-98bb8ccf5-lcntb\" (UID: \"56a9a3f1-852b-458f-979f-1b815f8175d7\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-185e8b-98bb8ccf5-lcntb"
Oct 30 19:33:27.993532 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:33:27.993496    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jqrp2\" (UniqueName: \"kubernetes.io/projected/56a9a3f1-852b-458f-979f-1b815f8175d7-kube-api-access-jqrp2\") pod \"payload-pre-processing-e2e-extmod-b-185e8b-98bb8ccf5-lcntb\" (UID: \"56a9a3f1-852b-458f-979f-1b815f8175d7\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-185e8b-98bb8ccf5-lcntb"
Oct 30 19:33:28.094646 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:33:28.094614    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jqrp2\" (UniqueName: \"kubernetes.io/projected/56a9a3f1-852b-458f-979f-1b815f8175d7-kube-api-access-jqrp2\") pod \"payload-pre-processing-e2e-extmod-b-185e8b-98bb8ccf5-lcntb\" (UID: \"56a9a3f1-852b-458f-979f-1b815f8175d7\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-185e8b-98bb8ccf5-lcntb"
Oct 30 19:33:28.094823 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:33:28.094681    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/56a9a3f1-852b-458f-979f-1b815f8175d7-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-b-185e8b-98bb8ccf5-lcntb\" (UID: \"56a9a3f1-852b-458f-979f-1b815f8175d7\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-185e8b-98bb8ccf5-lcntb"
Oct 30 19:33:28.095570 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:33:28.095544    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/56a9a3f1-852b-458f-979f-1b815f8175d7-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-b-185e8b-98bb8ccf5-lcntb\" (UID: \"56a9a3f1-852b-458f-979f-1b815f8175d7\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-185e8b-98bb8ccf5-lcntb"
Oct 30 19:33:28.157834 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:33:28.157805    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jqrp2\" (UniqueName: \"kubernetes.io/projected/56a9a3f1-852b-458f-979f-1b815f8175d7-kube-api-access-jqrp2\") pod \"payload-pre-processing-e2e-extmod-b-185e8b-98bb8ccf5-lcntb\" (UID: \"56a9a3f1-852b-458f-979f-1b815f8175d7\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-185e8b-98bb8ccf5-lcntb"
Oct 30 19:33:28.417755 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:33:28.417660    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-185e8b-98bb8ccf5-lcntb"
Oct 30 19:33:28.799856 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:33:28.799826    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-185e8b-98bb8ccf5-lcntb"]
Oct 30 19:33:28.801099 ip-10-0-134-46 kubenswrapper[4888]: W1030 19:33:28.801068    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56a9a3f1_852b_458f_979f_1b815f8175d7.slice/crio-80603a992eb834c3774ab316c4fdd9c82058e335089f604aca3fc9c256260278 WatchSource:0}: Error finding container 80603a992eb834c3774ab316c4fdd9c82058e335089f604aca3fc9c256260278: Status 404 returned error can't find the container with id 80603a992eb834c3774ab316c4fdd9c82058e335089f604aca3fc9c256260278
Oct 30 19:33:28.821128 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:33:28.821096    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-185e8b-98bb8ccf5-lcntb" event={"ID":"56a9a3f1-852b-458f-979f-1b815f8175d7","Type":"ContainerStarted","Data":"80603a992eb834c3774ab316c4fdd9c82058e335089f604aca3fc9c256260278"}
Oct 30 19:33:29.831140 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:33:29.831102    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-185e8b-98bb8ccf5-lcntb" event={"ID":"56a9a3f1-852b-458f-979f-1b815f8175d7","Type":"ContainerStarted","Data":"1573430f9974d1b1c497479a9bbef5c737901a3ea9c6bb4b21dfc3cfd71c2250"}
Oct 30 19:33:29.831699 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:33:29.831169    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-185e8b-98bb8ccf5-lcntb"
Oct 30 19:33:29.950927 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:33:29.950873    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-185e8b-98bb8ccf5-lcntb" podStartSLOduration=2.950859336 podStartE2EDuration="2.950859336s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:33:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:33:29.94905626 +0000 UTC m=+3574.196462284" watchObservedRunningTime="2026-10-30 19:33:29.950859336 +0000 UTC m=+3574.198265361"
Oct 30 19:33:35.617676 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:33:35.617636    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-jbsw6" containerName="extractor"
Oct 30 19:33:40.841805 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:33:40.841780    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-185e8b-98bb8ccf5-lcntb"
Oct 30 19:33:44.653887 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:33:44.653850    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-rjmdz"]
Oct 30 19:33:44.938034 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:33:44.937943    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-noten-3edc13-openshift-default-8487df5f79-bmkwd"]
Oct 30 19:33:44.938208 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:33:44.938164    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-noten-3edc13-openshift-default-8487df5f79-bmkwd" podUID="82782e69-02e1-44ef-b7f4-72f847f5a913" containerName="istio-proxy" containerID="cri-o://6f132904e2b31099412a5543adb7d98898535b5a5ecbda6141ccc01dcda67ee7" gracePeriod=30
Oct 30 19:33:49.342804 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:33:49.342767    4888 patch_prober.go:29] interesting pod/e2e-noten-3edc13-openshift-default-8487df5f79-bmkwd container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 19:33:49.343278 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:33:49.342844    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-noten-3edc13-openshift-default-8487df5f79-bmkwd" podUID="82782e69-02e1-44ef-b7f4-72f847f5a913" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 19:33:50.220238 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:33:50.220213    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-3edc13-openshift-default-8487df5f79-bmkwd"
Oct 30 19:33:50.356979 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:33:50.356944    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/82782e69-02e1-44ef-b7f4-72f847f5a913-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/82782e69-02e1-44ef-b7f4-72f847f5a913-istiod-ca-cert\") pod \"82782e69-02e1-44ef-b7f4-72f847f5a913\" (UID: \"82782e69-02e1-44ef-b7f4-72f847f5a913\") "
Oct 30 19:33:50.357558 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:33:50.357037    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/82782e69-02e1-44ef-b7f4-72f847f5a913-kube-api-access-p7q8k\" (UniqueName: \"kubernetes.io/projected/82782e69-02e1-44ef-b7f4-72f847f5a913-kube-api-access-p7q8k\") pod \"82782e69-02e1-44ef-b7f4-72f847f5a913\" (UID: \"82782e69-02e1-44ef-b7f4-72f847f5a913\") "
Oct 30 19:33:50.357558 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:33:50.357102    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/82782e69-02e1-44ef-b7f4-72f847f5a913-istio-token\" (UniqueName: \"kubernetes.io/projected/82782e69-02e1-44ef-b7f4-72f847f5a913-istio-token\") pod \"82782e69-02e1-44ef-b7f4-72f847f5a913\" (UID: \"82782e69-02e1-44ef-b7f4-72f847f5a913\") "
Oct 30 19:33:50.357558 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:33:50.357243    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/82782e69-02e1-44ef-b7f4-72f847f5a913-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/82782e69-02e1-44ef-b7f4-72f847f5a913-workload-socket\") pod \"82782e69-02e1-44ef-b7f4-72f847f5a913\" (UID: \"82782e69-02e1-44ef-b7f4-72f847f5a913\") "
Oct 30 19:33:50.357558 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:33:50.357278    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/82782e69-02e1-44ef-b7f4-72f847f5a913-istiod-ca-cert" pod "82782e69-02e1-44ef-b7f4-72f847f5a913" (UID: "82782e69-02e1-44ef-b7f4-72f847f5a913"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:33:50.357558 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:33:50.357348    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/82782e69-02e1-44ef-b7f4-72f847f5a913-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/82782e69-02e1-44ef-b7f4-72f847f5a913-istio-data\") pod \"82782e69-02e1-44ef-b7f4-72f847f5a913\" (UID: \"82782e69-02e1-44ef-b7f4-72f847f5a913\") "
Oct 30 19:33:50.357558 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:33:50.357389    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/82782e69-02e1-44ef-b7f4-72f847f5a913-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/82782e69-02e1-44ef-b7f4-72f847f5a913-workload-certs\") pod \"82782e69-02e1-44ef-b7f4-72f847f5a913\" (UID: \"82782e69-02e1-44ef-b7f4-72f847f5a913\") "
Oct 30 19:33:50.357558 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:33:50.357424    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/82782e69-02e1-44ef-b7f4-72f847f5a913-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/82782e69-02e1-44ef-b7f4-72f847f5a913-istio-envoy\") pod \"82782e69-02e1-44ef-b7f4-72f847f5a913\" (UID: \"82782e69-02e1-44ef-b7f4-72f847f5a913\") "
Oct 30 19:33:50.357558 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:33:50.357455    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/82782e69-02e1-44ef-b7f4-72f847f5a913-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/82782e69-02e1-44ef-b7f4-72f847f5a913-istio-podinfo\") pod \"82782e69-02e1-44ef-b7f4-72f847f5a913\" (UID: \"82782e69-02e1-44ef-b7f4-72f847f5a913\") "
Oct 30 19:33:50.357558 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:33:50.357487    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/82782e69-02e1-44ef-b7f4-72f847f5a913-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/82782e69-02e1-44ef-b7f4-72f847f5a913-credential-socket\") pod \"82782e69-02e1-44ef-b7f4-72f847f5a913\" (UID: \"82782e69-02e1-44ef-b7f4-72f847f5a913\") "
Oct 30 19:33:50.358019 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:33:50.357533    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/82782e69-02e1-44ef-b7f4-72f847f5a913-workload-socket" pod "82782e69-02e1-44ef-b7f4-72f847f5a913" (UID: "82782e69-02e1-44ef-b7f4-72f847f5a913"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:33:50.358019 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:33:50.357587    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/82782e69-02e1-44ef-b7f4-72f847f5a913-istio-data" pod "82782e69-02e1-44ef-b7f4-72f847f5a913" (UID: "82782e69-02e1-44ef-b7f4-72f847f5a913"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:33:50.358019 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:33:50.357607    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/82782e69-02e1-44ef-b7f4-72f847f5a913-workload-certs" pod "82782e69-02e1-44ef-b7f4-72f847f5a913" (UID: "82782e69-02e1-44ef-b7f4-72f847f5a913"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:33:50.358019 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:33:50.357892    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/82782e69-02e1-44ef-b7f4-72f847f5a913-credential-socket" pod "82782e69-02e1-44ef-b7f4-72f847f5a913" (UID: "82782e69-02e1-44ef-b7f4-72f847f5a913"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:33:50.358184 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:33:50.358033    4888 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/82782e69-02e1-44ef-b7f4-72f847f5a913-workload-socket\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:33:50.358184 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:33:50.358052    4888 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/82782e69-02e1-44ef-b7f4-72f847f5a913-istio-data\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:33:50.358184 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:33:50.358067    4888 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/82782e69-02e1-44ef-b7f4-72f847f5a913-workload-certs\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:33:50.358184 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:33:50.358081    4888 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/82782e69-02e1-44ef-b7f4-72f847f5a913-credential-socket\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:33:50.358184 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:33:50.358095    4888 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/82782e69-02e1-44ef-b7f4-72f847f5a913-istiod-ca-cert\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:33:50.359773 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:33:50.359745    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/82782e69-02e1-44ef-b7f4-72f847f5a913-istio-podinfo" pod "82782e69-02e1-44ef-b7f4-72f847f5a913" (UID: "82782e69-02e1-44ef-b7f4-72f847f5a913"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:33:50.359884 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:33:50.359864    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/82782e69-02e1-44ef-b7f4-72f847f5a913-istio-envoy" pod "82782e69-02e1-44ef-b7f4-72f847f5a913" (UID: "82782e69-02e1-44ef-b7f4-72f847f5a913"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:33:50.360081 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:33:50.360062    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/82782e69-02e1-44ef-b7f4-72f847f5a913-kube-api-access-p7q8k" pod "82782e69-02e1-44ef-b7f4-72f847f5a913" (UID: "82782e69-02e1-44ef-b7f4-72f847f5a913"). InnerVolumeSpecName "kube-api-access-p7q8k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:33:50.360124 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:33:50.360076    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/82782e69-02e1-44ef-b7f4-72f847f5a913-istio-token" pod "82782e69-02e1-44ef-b7f4-72f847f5a913" (UID: "82782e69-02e1-44ef-b7f4-72f847f5a913"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:33:50.459017 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:33:50.458948    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p7q8k\" (UniqueName: \"kubernetes.io/projected/82782e69-02e1-44ef-b7f4-72f847f5a913-kube-api-access-p7q8k\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:33:50.459017 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:33:50.458973    4888 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/82782e69-02e1-44ef-b7f4-72f847f5a913-istio-token\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:33:50.459017 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:33:50.458983    4888 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/82782e69-02e1-44ef-b7f4-72f847f5a913-istio-envoy\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:33:50.459017 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:33:50.458991    4888 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/82782e69-02e1-44ef-b7f4-72f847f5a913-istio-podinfo\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:33:50.996352 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:33:50.996318    4888 generic.go:356] "Generic (PLEG): container finished" podID="82782e69-02e1-44ef-b7f4-72f847f5a913" containerID="6f132904e2b31099412a5543adb7d98898535b5a5ecbda6141ccc01dcda67ee7" exitCode=0
Oct 30 19:33:50.996535 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:33:50.996371    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-3edc13-openshift-default-8487df5f79-bmkwd"
Oct 30 19:33:50.996535 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:33:50.996387    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-3edc13-openshift-default-8487df5f79-bmkwd" event={"ID":"82782e69-02e1-44ef-b7f4-72f847f5a913","Type":"ContainerDied","Data":"6f132904e2b31099412a5543adb7d98898535b5a5ecbda6141ccc01dcda67ee7"}
Oct 30 19:33:50.996535 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:33:50.996456    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-3edc13-openshift-default-8487df5f79-bmkwd" event={"ID":"82782e69-02e1-44ef-b7f4-72f847f5a913","Type":"ContainerDied","Data":"79ba62a286bc9ba57dff76dc2c8cb9fadaee3c68813135727e369cc9eb538593"}
Oct 30 19:33:50.996535 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:33:50.996485    4888 scope.go:122] "RemoveContainer" containerID="6f132904e2b31099412a5543adb7d98898535b5a5ecbda6141ccc01dcda67ee7"
Oct 30 19:33:51.008592 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:33:51.008569    4888 scope.go:122] "RemoveContainer" containerID="6f132904e2b31099412a5543adb7d98898535b5a5ecbda6141ccc01dcda67ee7"
Oct 30 19:33:51.008906 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:33:51.008886    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6f132904e2b31099412a5543adb7d98898535b5a5ecbda6141ccc01dcda67ee7\": container with ID starting with 6f132904e2b31099412a5543adb7d98898535b5a5ecbda6141ccc01dcda67ee7 not found: ID does not exist" containerID="6f132904e2b31099412a5543adb7d98898535b5a5ecbda6141ccc01dcda67ee7"
Oct 30 19:33:51.008986 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:33:51.008913    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f132904e2b31099412a5543adb7d98898535b5a5ecbda6141ccc01dcda67ee7"} err="failed to get container status \"6f132904e2b31099412a5543adb7d98898535b5a5ecbda6141ccc01dcda67ee7\": rpc error: code = NotFound desc = could not find container \"6f132904e2b31099412a5543adb7d98898535b5a5ecbda6141ccc01dcda67ee7\": container with ID starting with 6f132904e2b31099412a5543adb7d98898535b5a5ecbda6141ccc01dcda67ee7 not found: ID does not exist"
Oct 30 19:33:51.167100 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:33:51.167067    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-noten-3edc13-openshift-default-8487df5f79-bmkwd"]
Oct 30 19:33:51.218690 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:33:51.218652    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-noten-3edc13-openshift-default-8487df5f79-bmkwd"]
Oct 30 19:33:51.891799 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:33:51.891769    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="82782e69-02e1-44ef-b7f4-72f847f5a913" path="/var/lib/kubelet/pods/82782e69-02e1-44ef-b7f4-72f847f5a913/volumes"
Oct 30 19:33:56.807749 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:33:56.807722    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-7x4rb_382e8b31-a7fa-43f3-9f15-4c61627b0c04/console-operator/2.log"
Oct 30 19:33:56.816605 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:33:56.816578    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xfcdl_3f3fe764-2652-46c9-8e54-7882c0e6bcf1/ovn-acl-logging/0.log"
Oct 30 19:33:56.818707 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:33:56.818684    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-7x4rb_382e8b31-a7fa-43f3-9f15-4c61627b0c04/console-operator/2.log"
Oct 30 19:33:56.825705 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:33:56.825688    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xfcdl_3f3fe764-2652-46c9-8e54-7882c0e6bcf1/ovn-acl-logging/0.log"
Oct 30 19:34:00.096477 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:34:00.096444    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-9e4e63-b858ffbf4-z7whl"]
Oct 30 19:34:00.097378 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:34:00.097355    4888 cpu_manager.go:414] "containerMap: removing container" podUID="82782e69-02e1-44ef-b7f4-72f847f5a913" containerName="istio-proxy"
Oct 30 19:34:00.097378 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:34:00.097377    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="82782e69-02e1-44ef-b7f4-72f847f5a913" containerName="istio-proxy"
Oct 30 19:34:00.097578 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:34:00.097566    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="82782e69-02e1-44ef-b7f4-72f847f5a913" containerName="istio-proxy"
Oct 30 19:34:00.101631 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:34:00.101613    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-9e4e63-b858ffbf4-z7whl"
Oct 30 19:34:00.209609 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:34:00.208818    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-extmod-a-9e4e63-serving-cert\""
Oct 30 19:34:00.214222 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:34:00.214185    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-9e4e63-b858ffbf4-z7whl"]
Oct 30 19:34:00.275750 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:34:00.275720    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d8169ec1-dc9b-44a4-a5ba-09ee2ccf7a66-metrics-certs\") pod \"maas-api-e2e-extmod-a-9e4e63-b858ffbf4-z7whl\" (UID: \"d8169ec1-dc9b-44a4-a5ba-09ee2ccf7a66\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-9e4e63-b858ffbf4-z7whl"
Oct 30 19:34:00.275959 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:34:00.275846    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d8169ec1-dc9b-44a4-a5ba-09ee2ccf7a66-openshift-service-ca\") pod \"maas-api-e2e-extmod-a-9e4e63-b858ffbf4-z7whl\" (UID: \"d8169ec1-dc9b-44a4-a5ba-09ee2ccf7a66\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-9e4e63-b858ffbf4-z7whl"
Oct 30 19:34:00.275959 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:34:00.275927    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hpsr4\" (UniqueName: \"kubernetes.io/projected/d8169ec1-dc9b-44a4-a5ba-09ee2ccf7a66-kube-api-access-hpsr4\") pod \"maas-api-e2e-extmod-a-9e4e63-b858ffbf4-z7whl\" (UID: \"d8169ec1-dc9b-44a4-a5ba-09ee2ccf7a66\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-9e4e63-b858ffbf4-z7whl"
Oct 30 19:34:00.276145 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:34:00.276120    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d8169ec1-dc9b-44a4-a5ba-09ee2ccf7a66-maas-api-tls\") pod \"maas-api-e2e-extmod-a-9e4e63-b858ffbf4-z7whl\" (UID: \"d8169ec1-dc9b-44a4-a5ba-09ee2ccf7a66\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-9e4e63-b858ffbf4-z7whl"
Oct 30 19:34:00.377650 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:34:00.377553    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d8169ec1-dc9b-44a4-a5ba-09ee2ccf7a66-metrics-certs\") pod \"maas-api-e2e-extmod-a-9e4e63-b858ffbf4-z7whl\" (UID: \"d8169ec1-dc9b-44a4-a5ba-09ee2ccf7a66\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-9e4e63-b858ffbf4-z7whl"
Oct 30 19:34:00.377834 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:34:00.377660    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d8169ec1-dc9b-44a4-a5ba-09ee2ccf7a66-openshift-service-ca\") pod \"maas-api-e2e-extmod-a-9e4e63-b858ffbf4-z7whl\" (UID: \"d8169ec1-dc9b-44a4-a5ba-09ee2ccf7a66\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-9e4e63-b858ffbf4-z7whl"
Oct 30 19:34:00.377834 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:34:00.377724    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hpsr4\" (UniqueName: \"kubernetes.io/projected/d8169ec1-dc9b-44a4-a5ba-09ee2ccf7a66-kube-api-access-hpsr4\") pod \"maas-api-e2e-extmod-a-9e4e63-b858ffbf4-z7whl\" (UID: \"d8169ec1-dc9b-44a4-a5ba-09ee2ccf7a66\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-9e4e63-b858ffbf4-z7whl"
Oct 30 19:34:00.377834 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:34:00.377825    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d8169ec1-dc9b-44a4-a5ba-09ee2ccf7a66-maas-api-tls\") pod \"maas-api-e2e-extmod-a-9e4e63-b858ffbf4-z7whl\" (UID: \"d8169ec1-dc9b-44a4-a5ba-09ee2ccf7a66\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-9e4e63-b858ffbf4-z7whl"
Oct 30 19:34:00.378573 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:34:00.378492    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d8169ec1-dc9b-44a4-a5ba-09ee2ccf7a66-openshift-service-ca\") pod \"maas-api-e2e-extmod-a-9e4e63-b858ffbf4-z7whl\" (UID: \"d8169ec1-dc9b-44a4-a5ba-09ee2ccf7a66\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-9e4e63-b858ffbf4-z7whl"
Oct 30 19:34:00.380450 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:34:00.380423    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d8169ec1-dc9b-44a4-a5ba-09ee2ccf7a66-metrics-certs\") pod \"maas-api-e2e-extmod-a-9e4e63-b858ffbf4-z7whl\" (UID: \"d8169ec1-dc9b-44a4-a5ba-09ee2ccf7a66\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-9e4e63-b858ffbf4-z7whl"
Oct 30 19:34:00.380755 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:34:00.380735    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d8169ec1-dc9b-44a4-a5ba-09ee2ccf7a66-maas-api-tls\") pod \"maas-api-e2e-extmod-a-9e4e63-b858ffbf4-z7whl\" (UID: \"d8169ec1-dc9b-44a4-a5ba-09ee2ccf7a66\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-9e4e63-b858ffbf4-z7whl"
Oct 30 19:34:00.441234 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:34:00.441199    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hpsr4\" (UniqueName: \"kubernetes.io/projected/d8169ec1-dc9b-44a4-a5ba-09ee2ccf7a66-kube-api-access-hpsr4\") pod \"maas-api-e2e-extmod-a-9e4e63-b858ffbf4-z7whl\" (UID: \"d8169ec1-dc9b-44a4-a5ba-09ee2ccf7a66\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-9e4e63-b858ffbf4-z7whl"
Oct 30 19:34:00.716156 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:34:00.716066    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-9e4e63-b858ffbf4-z7whl"
Oct 30 19:34:01.099852 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:34:01.099816    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-9e4e63-b858ffbf4-z7whl"]
Oct 30 19:34:02.092735 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:34:02.092709    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-9e4e63-b858ffbf4-z7whl" event={"ID":"d8169ec1-dc9b-44a4-a5ba-09ee2ccf7a66","Type":"ContainerStarted","Data":"1f598a05b02be50f5a3125eef54bf188d1162bb66309662542db452adf7fba4c"}
Oct 30 19:34:03.124679 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:34:03.124641    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-9e4e63-b858ffbf4-z7whl" event={"ID":"d8169ec1-dc9b-44a4-a5ba-09ee2ccf7a66","Type":"ContainerStarted","Data":"b40de078a3db8c7e46fffcf3594f00302235928942de68e5bd2a9d222513da8e"}
Oct 30 19:34:03.125340 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:34:03.125321    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-9e4e63-b858ffbf4-z7whl"
Oct 30 19:34:03.247096 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:34:03.247043    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-9e4e63-b858ffbf4-z7whl" podStartSLOduration=2.272946057 podStartE2EDuration="3.247028551s" totalImagesPullingTime="974.082494ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:34:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:34:03.246106423 +0000 UTC m=+3607.493512448" watchObservedRunningTime="2026-10-30 19:34:03.247028551 +0000 UTC m=+3607.494434577"
Oct 30 19:34:05.618567 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:34:05.618532    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-jbsw6" containerName="extractor"
Oct 30 19:34:09.137441 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:34:09.137413    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-9e4e63-b858ffbf4-z7whl"
Oct 30 19:34:25.480433 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:34:25.477463    4888 patch_prober.go:29] interesting pod/e2e-extmod-a-9e4e63-openshift-default-5785d6dfd9-nbk9s container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.166:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:34:25.480433 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:34:25.477548    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-extmod-a-9e4e63-openshift-default-5785d6dfd9-nbk9s" podUID="5cc67731-e338-42cc-8076-5806861336d9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.166:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:34:27.343429 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:34:27.343394    4888 generic.go:356] "Generic (PLEG): container finished" podID="5cc67731-e338-42cc-8076-5806861336d9" containerID="48dd6a1cb16f3c9c13e8f9359ee07c0da725624613312a0452443c8bc3383aec" exitCode=0
Oct 30 19:34:27.343995 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:34:27.343442    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-9e4e63-openshift-default-5785d6dfd9-nbk9s" event={"ID":"5cc67731-e338-42cc-8076-5806861336d9","Type":"ContainerDied","Data":"48dd6a1cb16f3c9c13e8f9359ee07c0da725624613312a0452443c8bc3383aec"}
Oct 30 19:34:27.343995 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:34:27.343830    4888 scope.go:122] "RemoveContainer" containerID="48dd6a1cb16f3c9c13e8f9359ee07c0da725624613312a0452443c8bc3383aec"
Oct 30 19:34:27.345812 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:34:27.345775    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:34:27.345903 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:34:27.345869    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:34:27.345941 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:34:27.345925    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:34:28.359658 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:34:28.359619    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-9e4e63-openshift-default-5785d6dfd9-nbk9s" event={"ID":"5cc67731-e338-42cc-8076-5806861336d9","Type":"ContainerStarted","Data":"0fc2404f38518785f2d400650e7750467f59274faf3386fb9c3629a631a2062f"}
Oct 30 19:34:28.962698 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:34:28.962664    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-extmod-a-9e4e63-openshift-default-5785d6dfd9-nbk9s"
Oct 30 19:34:28.962970 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:34:28.962941    4888 patch_prober.go:29] interesting pod/e2e-extmod-a-9e4e63-openshift-default-5785d6dfd9-nbk9s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.166:15021/healthz/ready\": dial tcp 10.132.0.166:15021: connect: connection refused" start-of-body=
Oct 30 19:34:28.963098 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:34:28.962990    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-9e4e63-openshift-default-5785d6dfd9-nbk9s" podUID="5cc67731-e338-42cc-8076-5806861336d9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.166:15021/healthz/ready\": dial tcp 10.132.0.166:15021: connect: connection refused"
Oct 30 19:34:29.651788 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:34:29.651755    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-extmod-b-25e5ef-openshift-default-7855d5bc49-wnnkh"]
Oct 30 19:34:29.656095 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:34:29.656073    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-25e5ef-openshift-default-7855d5bc49-wnnkh"
Oct 30 19:34:29.712645 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:34:29.712610    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-extmod-b-25e5ef-openshift-default-dockercfg-q2cxl\""
Oct 30 19:34:29.833234 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:34:29.833202    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/65f89a7e-6069-4aa0-a285-f83cba89116a-istio-data\") pod \"e2e-extmod-b-25e5ef-openshift-default-7855d5bc49-wnnkh\" (UID: \"65f89a7e-6069-4aa0-a285-f83cba89116a\") " pod="openshift-ingress/e2e-extmod-b-25e5ef-openshift-default-7855d5bc49-wnnkh"
Oct 30 19:34:29.833610 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:34:29.833583    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sk2bh\" (UniqueName: \"kubernetes.io/projected/65f89a7e-6069-4aa0-a285-f83cba89116a-kube-api-access-sk2bh\") pod \"e2e-extmod-b-25e5ef-openshift-default-7855d5bc49-wnnkh\" (UID: \"65f89a7e-6069-4aa0-a285-f83cba89116a\") " pod="openshift-ingress/e2e-extmod-b-25e5ef-openshift-default-7855d5bc49-wnnkh"
Oct 30 19:34:29.833812 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:34:29.833793    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/65f89a7e-6069-4aa0-a285-f83cba89116a-workload-socket\") pod \"e2e-extmod-b-25e5ef-openshift-default-7855d5bc49-wnnkh\" (UID: \"65f89a7e-6069-4aa0-a285-f83cba89116a\") " pod="openshift-ingress/e2e-extmod-b-25e5ef-openshift-default-7855d5bc49-wnnkh"
Oct 30 19:34:29.839712 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:34:29.839574    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/65f89a7e-6069-4aa0-a285-f83cba89116a-istiod-ca-cert\") pod \"e2e-extmod-b-25e5ef-openshift-default-7855d5bc49-wnnkh\" (UID: \"65f89a7e-6069-4aa0-a285-f83cba89116a\") " pod="openshift-ingress/e2e-extmod-b-25e5ef-openshift-default-7855d5bc49-wnnkh"
Oct 30 19:34:29.839914 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:34:29.839894    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/65f89a7e-6069-4aa0-a285-f83cba89116a-credential-socket\") pod \"e2e-extmod-b-25e5ef-openshift-default-7855d5bc49-wnnkh\" (UID: \"65f89a7e-6069-4aa0-a285-f83cba89116a\") " pod="openshift-ingress/e2e-extmod-b-25e5ef-openshift-default-7855d5bc49-wnnkh"
Oct 30 19:34:29.843666 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:34:29.843644    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/65f89a7e-6069-4aa0-a285-f83cba89116a-istio-podinfo\") pod \"e2e-extmod-b-25e5ef-openshift-default-7855d5bc49-wnnkh\" (UID: \"65f89a7e-6069-4aa0-a285-f83cba89116a\") " pod="openshift-ingress/e2e-extmod-b-25e5ef-openshift-default-7855d5bc49-wnnkh"
Oct 30 19:34:29.843943 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:34:29.843924    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/65f89a7e-6069-4aa0-a285-f83cba89116a-istio-envoy\") pod \"e2e-extmod-b-25e5ef-openshift-default-7855d5bc49-wnnkh\" (UID: \"65f89a7e-6069-4aa0-a285-f83cba89116a\") " pod="openshift-ingress/e2e-extmod-b-25e5ef-openshift-default-7855d5bc49-wnnkh"
Oct 30 19:34:29.844179 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:34:29.844163    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/65f89a7e-6069-4aa0-a285-f83cba89116a-workload-certs\") pod \"e2e-extmod-b-25e5ef-openshift-default-7855d5bc49-wnnkh\" (UID: \"65f89a7e-6069-4aa0-a285-f83cba89116a\") " pod="openshift-ingress/e2e-extmod-b-25e5ef-openshift-default-7855d5bc49-wnnkh"
Oct 30 19:34:29.844438 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:34:29.844419    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/65f89a7e-6069-4aa0-a285-f83cba89116a-istio-token\") pod \"e2e-extmod-b-25e5ef-openshift-default-7855d5bc49-wnnkh\" (UID: \"65f89a7e-6069-4aa0-a285-f83cba89116a\") " pod="openshift-ingress/e2e-extmod-b-25e5ef-openshift-default-7855d5bc49-wnnkh"
Oct 30 19:34:29.845273 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:34:29.845252    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-b-25e5ef-openshift-default-7855d5bc49-wnnkh"]
Oct 30 19:34:29.945977 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:34:29.945886    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sk2bh\" (UniqueName: \"kubernetes.io/projected/65f89a7e-6069-4aa0-a285-f83cba89116a-kube-api-access-sk2bh\") pod \"e2e-extmod-b-25e5ef-openshift-default-7855d5bc49-wnnkh\" (UID: \"65f89a7e-6069-4aa0-a285-f83cba89116a\") " pod="openshift-ingress/e2e-extmod-b-25e5ef-openshift-default-7855d5bc49-wnnkh"
Oct 30 19:34:29.945977 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:34:29.945937    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/65f89a7e-6069-4aa0-a285-f83cba89116a-workload-socket\") pod \"e2e-extmod-b-25e5ef-openshift-default-7855d5bc49-wnnkh\" (UID: \"65f89a7e-6069-4aa0-a285-f83cba89116a\") " pod="openshift-ingress/e2e-extmod-b-25e5ef-openshift-default-7855d5bc49-wnnkh"
Oct 30 19:34:29.945977 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:34:29.945965    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/65f89a7e-6069-4aa0-a285-f83cba89116a-istiod-ca-cert\") pod \"e2e-extmod-b-25e5ef-openshift-default-7855d5bc49-wnnkh\" (UID: \"65f89a7e-6069-4aa0-a285-f83cba89116a\") " pod="openshift-ingress/e2e-extmod-b-25e5ef-openshift-default-7855d5bc49-wnnkh"
Oct 30 19:34:29.946429 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:34:29.945994    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/65f89a7e-6069-4aa0-a285-f83cba89116a-credential-socket\") pod \"e2e-extmod-b-25e5ef-openshift-default-7855d5bc49-wnnkh\" (UID: \"65f89a7e-6069-4aa0-a285-f83cba89116a\") " pod="openshift-ingress/e2e-extmod-b-25e5ef-openshift-default-7855d5bc49-wnnkh"
Oct 30 19:34:29.946429 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:34:29.946041    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/65f89a7e-6069-4aa0-a285-f83cba89116a-istio-podinfo\") pod \"e2e-extmod-b-25e5ef-openshift-default-7855d5bc49-wnnkh\" (UID: \"65f89a7e-6069-4aa0-a285-f83cba89116a\") " pod="openshift-ingress/e2e-extmod-b-25e5ef-openshift-default-7855d5bc49-wnnkh"
Oct 30 19:34:29.946429 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:34:29.946109    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/65f89a7e-6069-4aa0-a285-f83cba89116a-istio-envoy\") pod \"e2e-extmod-b-25e5ef-openshift-default-7855d5bc49-wnnkh\" (UID: \"65f89a7e-6069-4aa0-a285-f83cba89116a\") " pod="openshift-ingress/e2e-extmod-b-25e5ef-openshift-default-7855d5bc49-wnnkh"
Oct 30 19:34:29.946429 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:34:29.946190    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/65f89a7e-6069-4aa0-a285-f83cba89116a-workload-certs\") pod \"e2e-extmod-b-25e5ef-openshift-default-7855d5bc49-wnnkh\" (UID: \"65f89a7e-6069-4aa0-a285-f83cba89116a\") " pod="openshift-ingress/e2e-extmod-b-25e5ef-openshift-default-7855d5bc49-wnnkh"
Oct 30 19:34:29.946429 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:34:29.946304    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/65f89a7e-6069-4aa0-a285-f83cba89116a-istio-token\") pod \"e2e-extmod-b-25e5ef-openshift-default-7855d5bc49-wnnkh\" (UID: \"65f89a7e-6069-4aa0-a285-f83cba89116a\") " pod="openshift-ingress/e2e-extmod-b-25e5ef-openshift-default-7855d5bc49-wnnkh"
Oct 30 19:34:29.946429 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:34:29.946367    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/65f89a7e-6069-4aa0-a285-f83cba89116a-istio-data\") pod \"e2e-extmod-b-25e5ef-openshift-default-7855d5bc49-wnnkh\" (UID: \"65f89a7e-6069-4aa0-a285-f83cba89116a\") " pod="openshift-ingress/e2e-extmod-b-25e5ef-openshift-default-7855d5bc49-wnnkh"
Oct 30 19:34:29.946828 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:34:29.946451    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/65f89a7e-6069-4aa0-a285-f83cba89116a-workload-socket\") pod \"e2e-extmod-b-25e5ef-openshift-default-7855d5bc49-wnnkh\" (UID: \"65f89a7e-6069-4aa0-a285-f83cba89116a\") " pod="openshift-ingress/e2e-extmod-b-25e5ef-openshift-default-7855d5bc49-wnnkh"
Oct 30 19:34:29.946828 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:34:29.946486    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/65f89a7e-6069-4aa0-a285-f83cba89116a-credential-socket\") pod \"e2e-extmod-b-25e5ef-openshift-default-7855d5bc49-wnnkh\" (UID: \"65f89a7e-6069-4aa0-a285-f83cba89116a\") " pod="openshift-ingress/e2e-extmod-b-25e5ef-openshift-default-7855d5bc49-wnnkh"
Oct 30 19:34:29.946828 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:34:29.946731    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/65f89a7e-6069-4aa0-a285-f83cba89116a-istio-data\") pod \"e2e-extmod-b-25e5ef-openshift-default-7855d5bc49-wnnkh\" (UID: \"65f89a7e-6069-4aa0-a285-f83cba89116a\") " pod="openshift-ingress/e2e-extmod-b-25e5ef-openshift-default-7855d5bc49-wnnkh"
Oct 30 19:34:29.946828 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:34:29.946741    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/65f89a7e-6069-4aa0-a285-f83cba89116a-workload-certs\") pod \"e2e-extmod-b-25e5ef-openshift-default-7855d5bc49-wnnkh\" (UID: \"65f89a7e-6069-4aa0-a285-f83cba89116a\") " pod="openshift-ingress/e2e-extmod-b-25e5ef-openshift-default-7855d5bc49-wnnkh"
Oct 30 19:34:29.946994 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:34:29.946890    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/65f89a7e-6069-4aa0-a285-f83cba89116a-istiod-ca-cert\") pod \"e2e-extmod-b-25e5ef-openshift-default-7855d5bc49-wnnkh\" (UID: \"65f89a7e-6069-4aa0-a285-f83cba89116a\") " pod="openshift-ingress/e2e-extmod-b-25e5ef-openshift-default-7855d5bc49-wnnkh"
Oct 30 19:34:29.948913 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:34:29.948888    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/65f89a7e-6069-4aa0-a285-f83cba89116a-istio-envoy\") pod \"e2e-extmod-b-25e5ef-openshift-default-7855d5bc49-wnnkh\" (UID: \"65f89a7e-6069-4aa0-a285-f83cba89116a\") " pod="openshift-ingress/e2e-extmod-b-25e5ef-openshift-default-7855d5bc49-wnnkh"
Oct 30 19:34:29.949022 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:34:29.948968    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/65f89a7e-6069-4aa0-a285-f83cba89116a-istio-podinfo\") pod \"e2e-extmod-b-25e5ef-openshift-default-7855d5bc49-wnnkh\" (UID: \"65f89a7e-6069-4aa0-a285-f83cba89116a\") " pod="openshift-ingress/e2e-extmod-b-25e5ef-openshift-default-7855d5bc49-wnnkh"
Oct 30 19:34:29.962321 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:34:29.962263    4888 patch_prober.go:29] interesting pod/e2e-extmod-a-9e4e63-openshift-default-5785d6dfd9-nbk9s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.166:15021/healthz/ready\": dial tcp 10.132.0.166:15021: connect: connection refused" start-of-body=
Oct 30 19:34:29.962423 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:34:29.962351    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-9e4e63-openshift-default-5785d6dfd9-nbk9s" podUID="5cc67731-e338-42cc-8076-5806861336d9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.166:15021/healthz/ready\": dial tcp 10.132.0.166:15021: connect: connection refused"
Oct 30 19:34:30.008590 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:34:30.008554    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/65f89a7e-6069-4aa0-a285-f83cba89116a-istio-token\") pod \"e2e-extmod-b-25e5ef-openshift-default-7855d5bc49-wnnkh\" (UID: \"65f89a7e-6069-4aa0-a285-f83cba89116a\") " pod="openshift-ingress/e2e-extmod-b-25e5ef-openshift-default-7855d5bc49-wnnkh"
Oct 30 19:34:30.008983 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:34:30.008959    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sk2bh\" (UniqueName: \"kubernetes.io/projected/65f89a7e-6069-4aa0-a285-f83cba89116a-kube-api-access-sk2bh\") pod \"e2e-extmod-b-25e5ef-openshift-default-7855d5bc49-wnnkh\" (UID: \"65f89a7e-6069-4aa0-a285-f83cba89116a\") " pod="openshift-ingress/e2e-extmod-b-25e5ef-openshift-default-7855d5bc49-wnnkh"
Oct 30 19:34:30.287392 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:34:30.276480    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-25e5ef-openshift-default-7855d5bc49-wnnkh"
Oct 30 19:34:30.761948 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:34:30.761907    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-b-25e5ef-openshift-default-7855d5bc49-wnnkh"]
Oct 30 19:34:30.796900 ip-10-0-134-46 kubenswrapper[4888]: W1030 19:34:30.796874    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65f89a7e_6069_4aa0_a285_f83cba89116a.slice/crio-c6d3ddc6b17570ce5c0ba6506a3511137980be44c2ca37efd0302c41b5aef651 WatchSource:0}: Error finding container c6d3ddc6b17570ce5c0ba6506a3511137980be44c2ca37efd0302c41b5aef651: Status 404 returned error can't find the container with id c6d3ddc6b17570ce5c0ba6506a3511137980be44c2ca37efd0302c41b5aef651
Oct 30 19:34:30.799997 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:34:30.799965    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:34:30.800101 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:34:30.800033    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:34:30.800101 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:34:30.800059    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:34:30.962732 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:34:30.962701    4888 patch_prober.go:29] interesting pod/e2e-extmod-a-9e4e63-openshift-default-5785d6dfd9-nbk9s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.166:15021/healthz/ready\": dial tcp 10.132.0.166:15021: connect: connection refused" start-of-body=
Oct 30 19:34:30.962920 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:34:30.962762    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-9e4e63-openshift-default-5785d6dfd9-nbk9s" podUID="5cc67731-e338-42cc-8076-5806861336d9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.166:15021/healthz/ready\": dial tcp 10.132.0.166:15021: connect: connection refused"
Oct 30 19:34:31.426031 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:34:31.425996    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-25e5ef-openshift-default-7855d5bc49-wnnkh" event={"ID":"65f89a7e-6069-4aa0-a285-f83cba89116a","Type":"ContainerStarted","Data":"8e290a9bd38221aa30b5588024ed4367f8b85deb9e6c6a8026410431c1c29d3d"}
Oct 30 19:34:31.426212 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:34:31.426049    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-25e5ef-openshift-default-7855d5bc49-wnnkh" event={"ID":"65f89a7e-6069-4aa0-a285-f83cba89116a","Type":"ContainerStarted","Data":"c6d3ddc6b17570ce5c0ba6506a3511137980be44c2ca37efd0302c41b5aef651"}
Oct 30 19:34:31.568311 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:34:31.563898    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-extmod-b-25e5ef-openshift-default-7855d5bc49-wnnkh" podStartSLOduration=2.563882039 podStartE2EDuration="2.563882039s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:34:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:34:31.559225973 +0000 UTC m=+3635.806631998" watchObservedRunningTime="2026-10-30 19:34:31.563882039 +0000 UTC m=+3635.811288065"
Oct 30 19:34:31.962056 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:34:31.962020    4888 patch_prober.go:29] interesting pod/e2e-extmod-a-9e4e63-openshift-default-5785d6dfd9-nbk9s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.166:15021/healthz/ready\": dial tcp 10.132.0.166:15021: connect: connection refused" start-of-body=
Oct 30 19:34:31.962576 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:34:31.962077    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-9e4e63-openshift-default-5785d6dfd9-nbk9s" podUID="5cc67731-e338-42cc-8076-5806861336d9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.166:15021/healthz/ready\": dial tcp 10.132.0.166:15021: connect: connection refused"
Oct 30 19:34:32.289359 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:34:32.288708    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-extmod-b-25e5ef-openshift-default-7855d5bc49-wnnkh"
Oct 30 19:34:32.301863 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:34:32.300938    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-extmod-b-25e5ef-openshift-default-7855d5bc49-wnnkh"
Oct 30 19:34:32.441792 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:34:32.441761    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-extmod-b-25e5ef-openshift-default-7855d5bc49-wnnkh"
Oct 30 19:34:32.442947 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:34:32.442918    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-extmod-b-25e5ef-openshift-default-7855d5bc49-wnnkh"
Oct 30 19:34:32.962549 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:34:32.962518    4888 patch_prober.go:29] interesting pod/e2e-extmod-a-9e4e63-openshift-default-5785d6dfd9-nbk9s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.166:15021/healthz/ready\": dial tcp 10.132.0.166:15021: connect: connection refused" start-of-body=
Oct 30 19:34:32.962936 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:34:32.962578    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-9e4e63-openshift-default-5785d6dfd9-nbk9s" podUID="5cc67731-e338-42cc-8076-5806861336d9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.166:15021/healthz/ready\": dial tcp 10.132.0.166:15021: connect: connection refused"
Oct 30 19:34:34.074589 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:34:34.074544    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-extmod-a-9e4e63-openshift-default-5785d6dfd9-nbk9s"
Oct 30 19:34:34.075956 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:34:34.075929    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-extmod-a-9e4e63-openshift-default-5785d6dfd9-nbk9s"
Oct 30 19:34:34.078562 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:34:34.078538    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-extmod-a-9e4e63-openshift-default-5785d6dfd9-nbk9s"
Oct 30 19:34:35.618274 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:34:35.618201    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-jbsw6" containerName="extractor"
Oct 30 19:34:48.833638 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:34:48.833604    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-rjmdz"]
Oct 30 19:34:50.499572 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:34:50.499539    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-185e8b-5b56c5cf98-snzjb"]
Oct 30 19:34:50.504899 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:34:50.504881    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-185e8b-5b56c5cf98-snzjb"
Oct 30 19:34:50.560511 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:34:50.560484    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-extmod-b-185e8b-serving-cert\""
Oct 30 19:34:50.592130 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:34:50.592105    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-stccw\" (UniqueName: \"kubernetes.io/projected/e5d89a86-c9d8-41bd-9152-a2d8e284d9e6-kube-api-access-stccw\") pod \"maas-api-e2e-extmod-b-185e8b-5b56c5cf98-snzjb\" (UID: \"e5d89a86-c9d8-41bd-9152-a2d8e284d9e6\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-185e8b-5b56c5cf98-snzjb"
Oct 30 19:34:50.592268 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:34:50.592150    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e5d89a86-c9d8-41bd-9152-a2d8e284d9e6-maas-api-tls\") pod \"maas-api-e2e-extmod-b-185e8b-5b56c5cf98-snzjb\" (UID: \"e5d89a86-c9d8-41bd-9152-a2d8e284d9e6\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-185e8b-5b56c5cf98-snzjb"
Oct 30 19:34:50.592268 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:34:50.592243    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e5d89a86-c9d8-41bd-9152-a2d8e284d9e6-metrics-certs\") pod \"maas-api-e2e-extmod-b-185e8b-5b56c5cf98-snzjb\" (UID: \"e5d89a86-c9d8-41bd-9152-a2d8e284d9e6\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-185e8b-5b56c5cf98-snzjb"
Oct 30 19:34:50.592411 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:34:50.592396    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e5d89a86-c9d8-41bd-9152-a2d8e284d9e6-openshift-service-ca\") pod \"maas-api-e2e-extmod-b-185e8b-5b56c5cf98-snzjb\" (UID: \"e5d89a86-c9d8-41bd-9152-a2d8e284d9e6\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-185e8b-5b56c5cf98-snzjb"
Oct 30 19:34:50.615761 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:34:50.615734    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-185e8b-5b56c5cf98-snzjb"]
Oct 30 19:34:50.694072 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:34:50.694041    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e5d89a86-c9d8-41bd-9152-a2d8e284d9e6-openshift-service-ca\") pod \"maas-api-e2e-extmod-b-185e8b-5b56c5cf98-snzjb\" (UID: \"e5d89a86-c9d8-41bd-9152-a2d8e284d9e6\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-185e8b-5b56c5cf98-snzjb"
Oct 30 19:34:50.694236 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:34:50.694127    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-stccw\" (UniqueName: \"kubernetes.io/projected/e5d89a86-c9d8-41bd-9152-a2d8e284d9e6-kube-api-access-stccw\") pod \"maas-api-e2e-extmod-b-185e8b-5b56c5cf98-snzjb\" (UID: \"e5d89a86-c9d8-41bd-9152-a2d8e284d9e6\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-185e8b-5b56c5cf98-snzjb"
Oct 30 19:34:50.694236 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:34:50.694163    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e5d89a86-c9d8-41bd-9152-a2d8e284d9e6-maas-api-tls\") pod \"maas-api-e2e-extmod-b-185e8b-5b56c5cf98-snzjb\" (UID: \"e5d89a86-c9d8-41bd-9152-a2d8e284d9e6\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-185e8b-5b56c5cf98-snzjb"
Oct 30 19:34:50.694236 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:34:50.694195    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e5d89a86-c9d8-41bd-9152-a2d8e284d9e6-metrics-certs\") pod \"maas-api-e2e-extmod-b-185e8b-5b56c5cf98-snzjb\" (UID: \"e5d89a86-c9d8-41bd-9152-a2d8e284d9e6\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-185e8b-5b56c5cf98-snzjb"
Oct 30 19:34:50.694483 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:34:50.694277    4888 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-extmod-b-185e8b-serving-cert: secret "maas-api-e2e-extmod-b-185e8b-serving-cert" not found
Oct 30 19:34:50.694483 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:34:50.694396    4888 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e5d89a86-c9d8-41bd-9152-a2d8e284d9e6-maas-api-tls podName:e5d89a86-c9d8-41bd-9152-a2d8e284d9e6 nodeName:}" failed. No retries permitted until 2026-10-30 19:34:51.194376445 +0000 UTC m=+3655.441782462 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/e5d89a86-c9d8-41bd-9152-a2d8e284d9e6-maas-api-tls") pod "maas-api-e2e-extmod-b-185e8b-5b56c5cf98-snzjb" (UID: "e5d89a86-c9d8-41bd-9152-a2d8e284d9e6") : secret "maas-api-e2e-extmod-b-185e8b-serving-cert" not found
Oct 30 19:34:50.694830 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:34:50.694807    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e5d89a86-c9d8-41bd-9152-a2d8e284d9e6-openshift-service-ca\") pod \"maas-api-e2e-extmod-b-185e8b-5b56c5cf98-snzjb\" (UID: \"e5d89a86-c9d8-41bd-9152-a2d8e284d9e6\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-185e8b-5b56c5cf98-snzjb"
Oct 30 19:34:50.696711 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:34:50.696694    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e5d89a86-c9d8-41bd-9152-a2d8e284d9e6-metrics-certs\") pod \"maas-api-e2e-extmod-b-185e8b-5b56c5cf98-snzjb\" (UID: \"e5d89a86-c9d8-41bd-9152-a2d8e284d9e6\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-185e8b-5b56c5cf98-snzjb"
Oct 30 19:34:50.762350 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:34:50.762251    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-stccw\" (UniqueName: \"kubernetes.io/projected/e5d89a86-c9d8-41bd-9152-a2d8e284d9e6-kube-api-access-stccw\") pod \"maas-api-e2e-extmod-b-185e8b-5b56c5cf98-snzjb\" (UID: \"e5d89a86-c9d8-41bd-9152-a2d8e284d9e6\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-185e8b-5b56c5cf98-snzjb"
Oct 30 19:34:51.200777 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:34:51.200746    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e5d89a86-c9d8-41bd-9152-a2d8e284d9e6-maas-api-tls\") pod \"maas-api-e2e-extmod-b-185e8b-5b56c5cf98-snzjb\" (UID: \"e5d89a86-c9d8-41bd-9152-a2d8e284d9e6\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-185e8b-5b56c5cf98-snzjb"
Oct 30 19:34:51.203528 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:34:51.203499    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e5d89a86-c9d8-41bd-9152-a2d8e284d9e6-maas-api-tls\") pod \"maas-api-e2e-extmod-b-185e8b-5b56c5cf98-snzjb\" (UID: \"e5d89a86-c9d8-41bd-9152-a2d8e284d9e6\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-185e8b-5b56c5cf98-snzjb"
Oct 30 19:34:51.420009 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:34:51.419969    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-185e8b-5b56c5cf98-snzjb"
Oct 30 19:34:51.804756 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:34:51.804731    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-185e8b-5b56c5cf98-snzjb"]
Oct 30 19:34:51.805171 ip-10-0-134-46 kubenswrapper[4888]: W1030 19:34:51.804982    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5d89a86_c9d8_41bd_9152_a2d8e284d9e6.slice/crio-736256c27bc3c3c4353e3fe92a1bd25ed8e1f762e8eda7d9450f2625f2c0687f WatchSource:0}: Error finding container 736256c27bc3c3c4353e3fe92a1bd25ed8e1f762e8eda7d9450f2625f2c0687f: Status 404 returned error can't find the container with id 736256c27bc3c3c4353e3fe92a1bd25ed8e1f762e8eda7d9450f2625f2c0687f
Oct 30 19:34:52.598915 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:34:52.598885    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-185e8b-5b56c5cf98-snzjb" event={"ID":"e5d89a86-c9d8-41bd-9152-a2d8e284d9e6","Type":"ContainerStarted","Data":"736256c27bc3c3c4353e3fe92a1bd25ed8e1f762e8eda7d9450f2625f2c0687f"}
Oct 30 19:34:53.610218 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:34:53.610173    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-185e8b-5b56c5cf98-snzjb" event={"ID":"e5d89a86-c9d8-41bd-9152-a2d8e284d9e6","Type":"ContainerStarted","Data":"2606d91761abd9c2de0d24cdfde4b048b36c5f8eab65f866be9efa31a43fef65"}
Oct 30 19:34:53.610789 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:34:53.610314    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-185e8b-5b56c5cf98-snzjb"
Oct 30 19:34:53.736261 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:34:53.736201    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-185e8b-5b56c5cf98-snzjb" podStartSLOduration=2.919428916 podStartE2EDuration="3.73618257s" totalImagesPullingTime="816.753654ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:34:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:34:53.732476432 +0000 UTC m=+3657.979882457" watchObservedRunningTime="2026-10-30 19:34:53.73618257 +0000 UTC m=+3657.983588597"
Oct 30 19:34:59.624332 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:34:59.624275    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-185e8b-5b56c5cf98-snzjb"
Oct 30 19:35:05.619924 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:35:05.619886    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-jbsw6" containerName="extractor"
Oct 30 19:35:18.867453 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:35:18.867418    4888 generic.go:356] "Generic (PLEG): container finished" podID="6381d3a5-bf3a-454a-b888-3aaea17c86f4" containerID="fc7ae7314397c53e7e758a8ca85c7cdea891a75a12d8f00e1610b36d04b6cc28" exitCode=0
Oct 30 19:35:18.867990 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:35:18.867488    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-185e8b-openshift-default-59ff49bb5c-tflbd" event={"ID":"6381d3a5-bf3a-454a-b888-3aaea17c86f4","Type":"ContainerDied","Data":"fc7ae7314397c53e7e758a8ca85c7cdea891a75a12d8f00e1610b36d04b6cc28"}
Oct 30 19:35:18.867990 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:35:18.867910    4888 scope.go:122] "RemoveContainer" containerID="fc7ae7314397c53e7e758a8ca85c7cdea891a75a12d8f00e1610b36d04b6cc28"
Oct 30 19:35:18.869651 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:35:18.869623    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:35:18.869796 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:35:18.869690    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:35:18.869796 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:35:18.869725    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:35:19.910123 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:35:19.910084    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-185e8b-openshift-default-59ff49bb5c-tflbd" event={"ID":"6381d3a5-bf3a-454a-b888-3aaea17c86f4","Type":"ContainerStarted","Data":"d3476df7b0f7c76866f269dd6ab139d7678635163f3a6846d00651b96a1cd556"}
Oct 30 19:35:20.446271 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:35:20.446241    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-extmod-b-185e8b-openshift-default-59ff49bb5c-tflbd"
Oct 30 19:35:20.447567 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:35:20.447543    4888 patch_prober.go:29] interesting pod/e2e-extmod-b-185e8b-openshift-default-59ff49bb5c-tflbd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.170:15021/healthz/ready\": dial tcp 10.132.0.170:15021: connect: connection refused" start-of-body=
Oct 30 19:35:20.447701 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:35:20.447585    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-185e8b-openshift-default-59ff49bb5c-tflbd" podUID="6381d3a5-bf3a-454a-b888-3aaea17c86f4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.170:15021/healthz/ready\": dial tcp 10.132.0.170:15021: connect: connection refused"
Oct 30 19:35:21.446732 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:35:21.446696    4888 patch_prober.go:29] interesting pod/e2e-extmod-b-185e8b-openshift-default-59ff49bb5c-tflbd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.170:15021/healthz/ready\": dial tcp 10.132.0.170:15021: connect: connection refused" start-of-body=
Oct 30 19:35:21.447134 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:35:21.446752    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-185e8b-openshift-default-59ff49bb5c-tflbd" podUID="6381d3a5-bf3a-454a-b888-3aaea17c86f4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.170:15021/healthz/ready\": dial tcp 10.132.0.170:15021: connect: connection refused"
Oct 30 19:35:21.766904 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:35:21.766819    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-extmod-b-185e8b-openshift-default-59ff49bb5c-tflbd"
Oct 30 19:35:22.446404 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:35:22.446370    4888 patch_prober.go:29] interesting pod/e2e-extmod-b-185e8b-openshift-default-59ff49bb5c-tflbd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.170:15021/healthz/ready\": dial tcp 10.132.0.170:15021: connect: connection refused" start-of-body=
Oct 30 19:35:22.446585 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:35:22.446426    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-185e8b-openshift-default-59ff49bb5c-tflbd" podUID="6381d3a5-bf3a-454a-b888-3aaea17c86f4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.170:15021/healthz/ready\": dial tcp 10.132.0.170:15021: connect: connection refused"
Oct 30 19:35:23.446152 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:35:23.446116    4888 patch_prober.go:29] interesting pod/e2e-extmod-b-185e8b-openshift-default-59ff49bb5c-tflbd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.170:15021/healthz/ready\": dial tcp 10.132.0.170:15021: connect: connection refused" start-of-body=
Oct 30 19:35:23.446593 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:35:23.446167    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-185e8b-openshift-default-59ff49bb5c-tflbd" podUID="6381d3a5-bf3a-454a-b888-3aaea17c86f4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.170:15021/healthz/ready\": dial tcp 10.132.0.170:15021: connect: connection refused"
Oct 30 19:35:24.446013 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:35:24.445973    4888 patch_prober.go:29] interesting pod/e2e-extmod-b-185e8b-openshift-default-59ff49bb5c-tflbd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.170:15021/healthz/ready\": dial tcp 10.132.0.170:15021: connect: connection refused" start-of-body=
Oct 30 19:35:24.446322 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:35:24.446028    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-185e8b-openshift-default-59ff49bb5c-tflbd" podUID="6381d3a5-bf3a-454a-b888-3aaea17c86f4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.170:15021/healthz/ready\": dial tcp 10.132.0.170:15021: connect: connection refused"
Oct 30 19:35:25.451245 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:35:25.451218    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-extmod-b-185e8b-openshift-default-59ff49bb5c-tflbd"
Oct 30 19:35:25.452438 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:35:25.452414    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-extmod-b-185e8b-openshift-default-59ff49bb5c-tflbd"
Oct 30 19:35:35.620669 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:35:35.620629    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-jbsw6" containerName="extractor"
Oct 30 19:35:41.506195 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:35:41.506163    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-rjmdz"]
Oct 30 19:35:45.719153 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:35:45.719124    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-345670-5dd96f489f-wvj5b"]
Oct 30 19:35:45.723631 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:35:45.723613    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-345670-5dd96f489f-wvj5b"
Oct 30 19:35:45.777914 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:35:45.777886    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-extmod-b-345670\""
Oct 30 19:35:45.782697 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:35:45.782663    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-345670-5dd96f489f-wvj5b"]
Oct 30 19:35:45.823765 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:35:45.823731    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hnghl\" (UniqueName: \"kubernetes.io/projected/bba5ec16-efe1-4a7a-a966-ef948a7c2b2e-kube-api-access-hnghl\") pod \"payload-pre-processing-e2e-extmod-b-345670-5dd96f489f-wvj5b\" (UID: \"bba5ec16-efe1-4a7a-a966-ef948a7c2b2e\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-345670-5dd96f489f-wvj5b"
Oct 30 19:35:45.823951 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:35:45.823796    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bba5ec16-efe1-4a7a-a966-ef948a7c2b2e-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-b-345670-5dd96f489f-wvj5b\" (UID: \"bba5ec16-efe1-4a7a-a966-ef948a7c2b2e\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-345670-5dd96f489f-wvj5b"
Oct 30 19:35:45.828576 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:35:45.828555    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-extmod-b-345670-dockercfg-g5qjx\""
Oct 30 19:35:45.925835 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:35:45.925808    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hnghl\" (UniqueName: \"kubernetes.io/projected/bba5ec16-efe1-4a7a-a966-ef948a7c2b2e-kube-api-access-hnghl\") pod \"payload-pre-processing-e2e-extmod-b-345670-5dd96f489f-wvj5b\" (UID: \"bba5ec16-efe1-4a7a-a966-ef948a7c2b2e\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-345670-5dd96f489f-wvj5b"
Oct 30 19:35:45.926001 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:35:45.925869    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bba5ec16-efe1-4a7a-a966-ef948a7c2b2e-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-b-345670-5dd96f489f-wvj5b\" (UID: \"bba5ec16-efe1-4a7a-a966-ef948a7c2b2e\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-345670-5dd96f489f-wvj5b"
Oct 30 19:35:45.926624 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:35:45.926603    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bba5ec16-efe1-4a7a-a966-ef948a7c2b2e-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-b-345670-5dd96f489f-wvj5b\" (UID: \"bba5ec16-efe1-4a7a-a966-ef948a7c2b2e\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-345670-5dd96f489f-wvj5b"
Oct 30 19:35:45.988483 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:35:45.988428    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hnghl\" (UniqueName: \"kubernetes.io/projected/bba5ec16-efe1-4a7a-a966-ef948a7c2b2e-kube-api-access-hnghl\") pod \"payload-pre-processing-e2e-extmod-b-345670-5dd96f489f-wvj5b\" (UID: \"bba5ec16-efe1-4a7a-a966-ef948a7c2b2e\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-345670-5dd96f489f-wvj5b"
Oct 30 19:35:46.036807 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:35:46.036784    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-345670-5dd96f489f-wvj5b"
Oct 30 19:35:46.417599 ip-10-0-134-46 kubenswrapper[4888]: W1030 19:35:46.417571    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbba5ec16_efe1_4a7a_a966_ef948a7c2b2e.slice/crio-e798642adfe71d623916c51034b95fc675a81f0a2b7a05612144473d0d82d8a5 WatchSource:0}: Error finding container e798642adfe71d623916c51034b95fc675a81f0a2b7a05612144473d0d82d8a5: Status 404 returned error can't find the container with id e798642adfe71d623916c51034b95fc675a81f0a2b7a05612144473d0d82d8a5
Oct 30 19:35:46.417964 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:35:46.417938    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-345670-5dd96f489f-wvj5b"]
Oct 30 19:35:47.123100 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:35:47.123066    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-345670-5dd96f489f-wvj5b" event={"ID":"bba5ec16-efe1-4a7a-a966-ef948a7c2b2e","Type":"ContainerStarted","Data":"72665270c956a269f8bebf9089be209a1a3ce4573d1eca33e2daf8d690928e64"}
Oct 30 19:35:47.123583 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:35:47.123121    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-345670-5dd96f489f-wvj5b" event={"ID":"bba5ec16-efe1-4a7a-a966-ef948a7c2b2e","Type":"ContainerStarted","Data":"e798642adfe71d623916c51034b95fc675a81f0a2b7a05612144473d0d82d8a5"}
Oct 30 19:35:47.123583 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:35:47.123311    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-345670-5dd96f489f-wvj5b"
Oct 30 19:35:47.256003 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:35:47.248492    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-345670-5dd96f489f-wvj5b" podStartSLOduration=2.24847101 podStartE2EDuration="2.24847101s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:35:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:35:47.247541313 +0000 UTC m=+3711.494947352" watchObservedRunningTime="2026-10-30 19:35:47.24847101 +0000 UTC m=+3711.495877205"
Oct 30 19:35:58.135156 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:35:58.135128    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-345670-5dd96f489f-wvj5b"
Oct 30 19:35:59.460094 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:35:59.460054    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-76a1ef-56fdd6dcd8-fz8cg"]
Oct 30 19:35:59.464467 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:35:59.464444    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-76a1ef-56fdd6dcd8-fz8cg"
Oct 30 19:35:59.505859 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:35:59.505828    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vcgrk\" (UniqueName: \"kubernetes.io/projected/faa528eb-1c1e-4060-8e7b-317b1900541c-kube-api-access-vcgrk\") pod \"maas-api-e2e-extmod-a-76a1ef-56fdd6dcd8-fz8cg\" (UID: \"faa528eb-1c1e-4060-8e7b-317b1900541c\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-76a1ef-56fdd6dcd8-fz8cg"
Oct 30 19:35:59.506026 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:35:59.505896    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/faa528eb-1c1e-4060-8e7b-317b1900541c-openshift-service-ca\") pod \"maas-api-e2e-extmod-a-76a1ef-56fdd6dcd8-fz8cg\" (UID: \"faa528eb-1c1e-4060-8e7b-317b1900541c\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-76a1ef-56fdd6dcd8-fz8cg"
Oct 30 19:35:59.506026 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:35:59.505988    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/faa528eb-1c1e-4060-8e7b-317b1900541c-maas-api-tls\") pod \"maas-api-e2e-extmod-a-76a1ef-56fdd6dcd8-fz8cg\" (UID: \"faa528eb-1c1e-4060-8e7b-317b1900541c\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-76a1ef-56fdd6dcd8-fz8cg"
Oct 30 19:35:59.506177 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:35:59.506159    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/faa528eb-1c1e-4060-8e7b-317b1900541c-metrics-certs\") pod \"maas-api-e2e-extmod-a-76a1ef-56fdd6dcd8-fz8cg\" (UID: \"faa528eb-1c1e-4060-8e7b-317b1900541c\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-76a1ef-56fdd6dcd8-fz8cg"
Oct 30 19:35:59.570702 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:35:59.570670    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-extmod-a-76a1ef-serving-cert\""
Oct 30 19:35:59.574519 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:35:59.574490    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-76a1ef-56fdd6dcd8-fz8cg"]
Oct 30 19:35:59.607979 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:35:59.607939    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vcgrk\" (UniqueName: \"kubernetes.io/projected/faa528eb-1c1e-4060-8e7b-317b1900541c-kube-api-access-vcgrk\") pod \"maas-api-e2e-extmod-a-76a1ef-56fdd6dcd8-fz8cg\" (UID: \"faa528eb-1c1e-4060-8e7b-317b1900541c\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-76a1ef-56fdd6dcd8-fz8cg"
Oct 30 19:35:59.608187 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:35:59.608126    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/faa528eb-1c1e-4060-8e7b-317b1900541c-openshift-service-ca\") pod \"maas-api-e2e-extmod-a-76a1ef-56fdd6dcd8-fz8cg\" (UID: \"faa528eb-1c1e-4060-8e7b-317b1900541c\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-76a1ef-56fdd6dcd8-fz8cg"
Oct 30 19:35:59.608187 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:35:59.608164    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/faa528eb-1c1e-4060-8e7b-317b1900541c-maas-api-tls\") pod \"maas-api-e2e-extmod-a-76a1ef-56fdd6dcd8-fz8cg\" (UID: \"faa528eb-1c1e-4060-8e7b-317b1900541c\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-76a1ef-56fdd6dcd8-fz8cg"
Oct 30 19:35:59.608347 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:35:59.608271    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/faa528eb-1c1e-4060-8e7b-317b1900541c-metrics-certs\") pod \"maas-api-e2e-extmod-a-76a1ef-56fdd6dcd8-fz8cg\" (UID: \"faa528eb-1c1e-4060-8e7b-317b1900541c\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-76a1ef-56fdd6dcd8-fz8cg"
Oct 30 19:35:59.608918 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:35:59.608893    4888 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-extmod-a-76a1ef-serving-cert: secret "maas-api-e2e-extmod-a-76a1ef-serving-cert" not found
Oct 30 19:35:59.609103 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:35:59.609077    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/faa528eb-1c1e-4060-8e7b-317b1900541c-openshift-service-ca\") pod \"maas-api-e2e-extmod-a-76a1ef-56fdd6dcd8-fz8cg\" (UID: \"faa528eb-1c1e-4060-8e7b-317b1900541c\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-76a1ef-56fdd6dcd8-fz8cg"
Oct 30 19:35:59.609211 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:35:59.609195    4888 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/faa528eb-1c1e-4060-8e7b-317b1900541c-maas-api-tls podName:faa528eb-1c1e-4060-8e7b-317b1900541c nodeName:}" failed. No retries permitted until 2026-10-30 19:36:00.109075564 +0000 UTC m=+3724.356481591 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/faa528eb-1c1e-4060-8e7b-317b1900541c-maas-api-tls") pod "maas-api-e2e-extmod-a-76a1ef-56fdd6dcd8-fz8cg" (UID: "faa528eb-1c1e-4060-8e7b-317b1900541c") : secret "maas-api-e2e-extmod-a-76a1ef-serving-cert" not found
Oct 30 19:35:59.611784 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:35:59.611757    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/faa528eb-1c1e-4060-8e7b-317b1900541c-metrics-certs\") pod \"maas-api-e2e-extmod-a-76a1ef-56fdd6dcd8-fz8cg\" (UID: \"faa528eb-1c1e-4060-8e7b-317b1900541c\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-76a1ef-56fdd6dcd8-fz8cg"
Oct 30 19:35:59.673929 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:35:59.673897    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vcgrk\" (UniqueName: \"kubernetes.io/projected/faa528eb-1c1e-4060-8e7b-317b1900541c-kube-api-access-vcgrk\") pod \"maas-api-e2e-extmod-a-76a1ef-56fdd6dcd8-fz8cg\" (UID: \"faa528eb-1c1e-4060-8e7b-317b1900541c\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-76a1ef-56fdd6dcd8-fz8cg"
Oct 30 19:36:00.118546 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:36:00.118508    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/faa528eb-1c1e-4060-8e7b-317b1900541c-maas-api-tls\") pod \"maas-api-e2e-extmod-a-76a1ef-56fdd6dcd8-fz8cg\" (UID: \"faa528eb-1c1e-4060-8e7b-317b1900541c\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-76a1ef-56fdd6dcd8-fz8cg"
Oct 30 19:36:00.121337 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:36:00.121269    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/faa528eb-1c1e-4060-8e7b-317b1900541c-maas-api-tls\") pod \"maas-api-e2e-extmod-a-76a1ef-56fdd6dcd8-fz8cg\" (UID: \"faa528eb-1c1e-4060-8e7b-317b1900541c\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-76a1ef-56fdd6dcd8-fz8cg"
Oct 30 19:36:00.378267 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:36:00.378194    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-76a1ef-56fdd6dcd8-fz8cg"
Oct 30 19:36:00.775159 ip-10-0-134-46 kubenswrapper[4888]: W1030 19:36:00.775126    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfaa528eb_1c1e_4060_8e7b_317b1900541c.slice/crio-2bf15168a565226aa5651814b556ed56a6a43aac555db84f36cb42ef97ef1d91 WatchSource:0}: Error finding container 2bf15168a565226aa5651814b556ed56a6a43aac555db84f36cb42ef97ef1d91: Status 404 returned error can't find the container with id 2bf15168a565226aa5651814b556ed56a6a43aac555db84f36cb42ef97ef1d91
Oct 30 19:36:00.775504 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:36:00.775189    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-76a1ef-56fdd6dcd8-fz8cg"]
Oct 30 19:36:01.229360 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:36:01.229321    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-76a1ef-56fdd6dcd8-fz8cg" event={"ID":"faa528eb-1c1e-4060-8e7b-317b1900541c","Type":"ContainerStarted","Data":"2bf15168a565226aa5651814b556ed56a6a43aac555db84f36cb42ef97ef1d91"}
Oct 30 19:36:02.239657 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:36:02.239611    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-76a1ef-56fdd6dcd8-fz8cg" event={"ID":"faa528eb-1c1e-4060-8e7b-317b1900541c","Type":"ContainerStarted","Data":"c4dfeec9c5920c3287d3c056d9112d710793457140229c9afb60deeee9c13fb9"}
Oct 30 19:36:02.240203 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:36:02.239743    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-76a1ef-56fdd6dcd8-fz8cg"
Oct 30 19:36:02.362617 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:36:02.362570    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-76a1ef-56fdd6dcd8-fz8cg" podStartSLOduration=2.509192175 podStartE2EDuration="3.36255724s" totalImagesPullingTime="853.365065ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:35:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:36:02.359750181 +0000 UTC m=+3726.607156231" watchObservedRunningTime="2026-10-30 19:36:02.36255724 +0000 UTC m=+3726.609963266"
Oct 30 19:36:05.619741 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:36:05.619663    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-jbsw6" containerName="extractor"
Oct 30 19:36:08.252595 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:36:08.252567    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-76a1ef-56fdd6dcd8-fz8cg"
Oct 30 19:36:19.821676 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:36:19.821636    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-rjmdz"]
Oct 30 19:36:35.622046 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:36:35.622009    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-jbsw6" containerName="extractor"
Oct 30 19:36:48.485474 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:36:48.485381    4888 patch_prober.go:29] interesting pod/e2e-extmod-b-25e5ef-openshift-default-7855d5bc49-wnnkh container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.173:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:36:48.485474 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:36:48.485443    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-extmod-b-25e5ef-openshift-default-7855d5bc49-wnnkh" podUID="65f89a7e-6069-4aa0-a285-f83cba89116a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.173:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:36:50.647449 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:36:50.647414    4888 generic.go:356] "Generic (PLEG): container finished" podID="65f89a7e-6069-4aa0-a285-f83cba89116a" containerID="8e290a9bd38221aa30b5588024ed4367f8b85deb9e6c6a8026410431c1c29d3d" exitCode=0
Oct 30 19:36:50.648027 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:36:50.647484    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-25e5ef-openshift-default-7855d5bc49-wnnkh" event={"ID":"65f89a7e-6069-4aa0-a285-f83cba89116a","Type":"ContainerDied","Data":"8e290a9bd38221aa30b5588024ed4367f8b85deb9e6c6a8026410431c1c29d3d"}
Oct 30 19:36:50.648027 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:36:50.647924    4888 scope.go:122] "RemoveContainer" containerID="8e290a9bd38221aa30b5588024ed4367f8b85deb9e6c6a8026410431c1c29d3d"
Oct 30 19:36:50.649634 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:36:50.649604    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:36:50.649736 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:36:50.649674    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:36:50.649736 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:36:50.649716    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:36:51.656951 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:36:51.656911    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-25e5ef-openshift-default-7855d5bc49-wnnkh" event={"ID":"65f89a7e-6069-4aa0-a285-f83cba89116a","Type":"ContainerStarted","Data":"e2003ca119c1e926a822f1e0b67efe0913e99234f3e985458688a30576e51657"}
Oct 30 19:36:52.284705 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:36:52.284669    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-extmod-b-25e5ef-openshift-default-7855d5bc49-wnnkh"
Oct 30 19:36:52.285082 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:36:52.285002    4888 patch_prober.go:29] interesting pod/e2e-extmod-b-25e5ef-openshift-default-7855d5bc49-wnnkh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.173:15021/healthz/ready\": dial tcp 10.132.0.173:15021: connect: connection refused" start-of-body=
Oct 30 19:36:52.285240 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:36:52.285220    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-25e5ef-openshift-default-7855d5bc49-wnnkh" podUID="65f89a7e-6069-4aa0-a285-f83cba89116a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.173:15021/healthz/ready\": dial tcp 10.132.0.173:15021: connect: connection refused"
Oct 30 19:36:53.284811 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:36:53.284772    4888 patch_prober.go:29] interesting pod/e2e-extmod-b-25e5ef-openshift-default-7855d5bc49-wnnkh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.173:15021/healthz/ready\": dial tcp 10.132.0.173:15021: connect: connection refused" start-of-body=
Oct 30 19:36:53.285436 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:36:53.285395    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-25e5ef-openshift-default-7855d5bc49-wnnkh" podUID="65f89a7e-6069-4aa0-a285-f83cba89116a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.173:15021/healthz/ready\": dial tcp 10.132.0.173:15021: connect: connection refused"
Oct 30 19:36:54.286308 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:36:54.284357    4888 patch_prober.go:29] interesting pod/e2e-extmod-b-25e5ef-openshift-default-7855d5bc49-wnnkh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.173:15021/healthz/ready\": dial tcp 10.132.0.173:15021: connect: connection refused" start-of-body=
Oct 30 19:36:54.286308 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:36:54.284427    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-25e5ef-openshift-default-7855d5bc49-wnnkh" podUID="65f89a7e-6069-4aa0-a285-f83cba89116a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.173:15021/healthz/ready\": dial tcp 10.132.0.173:15021: connect: connection refused"
Oct 30 19:36:55.283689 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:36:55.283649    4888 patch_prober.go:29] interesting pod/e2e-extmod-b-25e5ef-openshift-default-7855d5bc49-wnnkh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.173:15021/healthz/ready\": dial tcp 10.132.0.173:15021: connect: connection refused" start-of-body=
Oct 30 19:36:55.283978 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:36:55.283950    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-25e5ef-openshift-default-7855d5bc49-wnnkh" podUID="65f89a7e-6069-4aa0-a285-f83cba89116a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.173:15021/healthz/ready\": dial tcp 10.132.0.173:15021: connect: connection refused"
Oct 30 19:36:56.295312 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:36:56.291711    4888 patch_prober.go:29] interesting pod/e2e-extmod-b-25e5ef-openshift-default-7855d5bc49-wnnkh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.173:15021/healthz/ready\": dial tcp 10.132.0.173:15021: connect: connection refused" start-of-body=
Oct 30 19:36:56.295312 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:36:56.291788    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-25e5ef-openshift-default-7855d5bc49-wnnkh" podUID="65f89a7e-6069-4aa0-a285-f83cba89116a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.173:15021/healthz/ready\": dial tcp 10.132.0.173:15021: connect: connection refused"
Oct 30 19:36:57.287352 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:36:57.287323    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-extmod-b-25e5ef-openshift-default-7855d5bc49-wnnkh"
Oct 30 19:36:57.287828 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:36:57.287800    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-extmod-b-25e5ef-openshift-default-7855d5bc49-wnnkh"
Oct 30 19:36:57.288486 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:36:57.288466    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-extmod-b-25e5ef-openshift-default-7855d5bc49-wnnkh"
Oct 30 19:37:01.284220 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:37:01.284182    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-rjmdz"]
Oct 30 19:37:05.618730 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:37:05.618688    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-jbsw6" containerName="extractor"
Oct 30 19:37:27.429762 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:37:27.429727    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-da1b8b-55c47d44b6-mf594"]
Oct 30 19:37:27.430476 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:37:27.429980    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-da1b8b-55c47d44b6-mf594" podUID="53be2715-fb83-4227-9b79-cc1dcd732396" containerName="maas-api" containerID="cri-o://26a1b2512fd03daffd228bfceeaec2befb18eedac023380aaf360f0d2cc89380" gracePeriod=30
Oct 30 19:37:27.723878 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:37:27.723855    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-da1b8b-55c47d44b6-mf594"
Oct 30 19:37:27.888152 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:37:27.888121    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/53be2715-fb83-4227-9b79-cc1dcd732396-metrics-certs\" (UniqueName: \"kubernetes.io/secret/53be2715-fb83-4227-9b79-cc1dcd732396-metrics-certs\") pod \"53be2715-fb83-4227-9b79-cc1dcd732396\" (UID: \"53be2715-fb83-4227-9b79-cc1dcd732396\") "
Oct 30 19:37:27.888330 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:37:27.888176    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/53be2715-fb83-4227-9b79-cc1dcd732396-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/53be2715-fb83-4227-9b79-cc1dcd732396-openshift-service-ca\") pod \"53be2715-fb83-4227-9b79-cc1dcd732396\" (UID: \"53be2715-fb83-4227-9b79-cc1dcd732396\") "
Oct 30 19:37:27.888330 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:37:27.888254    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/53be2715-fb83-4227-9b79-cc1dcd732396-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/53be2715-fb83-4227-9b79-cc1dcd732396-maas-api-tls\") pod \"53be2715-fb83-4227-9b79-cc1dcd732396\" (UID: \"53be2715-fb83-4227-9b79-cc1dcd732396\") "
Oct 30 19:37:27.888330 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:37:27.888323    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/53be2715-fb83-4227-9b79-cc1dcd732396-kube-api-access-44qj7\" (UniqueName: \"kubernetes.io/projected/53be2715-fb83-4227-9b79-cc1dcd732396-kube-api-access-44qj7\") pod \"53be2715-fb83-4227-9b79-cc1dcd732396\" (UID: \"53be2715-fb83-4227-9b79-cc1dcd732396\") "
Oct 30 19:37:27.888565 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:37:27.888543    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/53be2715-fb83-4227-9b79-cc1dcd732396-openshift-service-ca" pod "53be2715-fb83-4227-9b79-cc1dcd732396" (UID: "53be2715-fb83-4227-9b79-cc1dcd732396"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:37:27.888852 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:37:27.888827    4888 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/53be2715-fb83-4227-9b79-cc1dcd732396-openshift-service-ca\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:37:27.896349 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:37:27.892574    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/53be2715-fb83-4227-9b79-cc1dcd732396-maas-api-tls" pod "53be2715-fb83-4227-9b79-cc1dcd732396" (UID: "53be2715-fb83-4227-9b79-cc1dcd732396"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:37:27.900502 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:37:27.900471    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/53be2715-fb83-4227-9b79-cc1dcd732396-kube-api-access-44qj7" pod "53be2715-fb83-4227-9b79-cc1dcd732396" (UID: "53be2715-fb83-4227-9b79-cc1dcd732396"). InnerVolumeSpecName "kube-api-access-44qj7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:37:27.906443 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:37:27.906407    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/53be2715-fb83-4227-9b79-cc1dcd732396-metrics-certs" pod "53be2715-fb83-4227-9b79-cc1dcd732396" (UID: "53be2715-fb83-4227-9b79-cc1dcd732396"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:37:27.956933 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:37:27.956861    4888 generic.go:356] "Generic (PLEG): container finished" podID="53be2715-fb83-4227-9b79-cc1dcd732396" containerID="26a1b2512fd03daffd228bfceeaec2befb18eedac023380aaf360f0d2cc89380" exitCode=0
Oct 30 19:37:27.956933 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:37:27.956907    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-da1b8b-55c47d44b6-mf594"
Oct 30 19:37:27.957110 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:37:27.956945    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-da1b8b-55c47d44b6-mf594" event={"ID":"53be2715-fb83-4227-9b79-cc1dcd732396","Type":"ContainerDied","Data":"26a1b2512fd03daffd228bfceeaec2befb18eedac023380aaf360f0d2cc89380"}
Oct 30 19:37:27.957110 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:37:27.957000    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-da1b8b-55c47d44b6-mf594" event={"ID":"53be2715-fb83-4227-9b79-cc1dcd732396","Type":"ContainerDied","Data":"d70105ca1de645aa7021907cd643e486c6db8d5adcf87488f98ccb7b87919a99"}
Oct 30 19:37:27.957110 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:37:27.957023    4888 scope.go:122] "RemoveContainer" containerID="26a1b2512fd03daffd228bfceeaec2befb18eedac023380aaf360f0d2cc89380"
Oct 30 19:37:27.967551 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:37:27.967534    4888 scope.go:122] "RemoveContainer" containerID="26a1b2512fd03daffd228bfceeaec2befb18eedac023380aaf360f0d2cc89380"
Oct 30 19:37:27.967824 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:37:27.967806    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"26a1b2512fd03daffd228bfceeaec2befb18eedac023380aaf360f0d2cc89380\": container with ID starting with 26a1b2512fd03daffd228bfceeaec2befb18eedac023380aaf360f0d2cc89380 not found: ID does not exist" containerID="26a1b2512fd03daffd228bfceeaec2befb18eedac023380aaf360f0d2cc89380"
Oct 30 19:37:27.967897 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:37:27.967829    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"26a1b2512fd03daffd228bfceeaec2befb18eedac023380aaf360f0d2cc89380"} err="failed to get container status \"26a1b2512fd03daffd228bfceeaec2befb18eedac023380aaf360f0d2cc89380\": rpc error: code = NotFound desc = could not find container \"26a1b2512fd03daffd228bfceeaec2befb18eedac023380aaf360f0d2cc89380\": container with ID starting with 26a1b2512fd03daffd228bfceeaec2befb18eedac023380aaf360f0d2cc89380 not found: ID does not exist"
Oct 30 19:37:27.990350 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:37:27.990271    4888 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/53be2715-fb83-4227-9b79-cc1dcd732396-metrics-certs\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:37:27.990635 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:37:27.990610    4888 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/53be2715-fb83-4227-9b79-cc1dcd732396-maas-api-tls\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:37:27.990813 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:37:27.990640    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-44qj7\" (UniqueName: \"kubernetes.io/projected/53be2715-fb83-4227-9b79-cc1dcd732396-kube-api-access-44qj7\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:37:28.130417 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:37:28.130386    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-da1b8b-55c47d44b6-mf594"]
Oct 30 19:37:28.135866 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:37:28.135838    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-da1b8b-55c47d44b6-mf594"]
Oct 30 19:37:29.891997 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:37:29.891965    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="53be2715-fb83-4227-9b79-cc1dcd732396" path="/var/lib/kubelet/pods/53be2715-fb83-4227-9b79-cc1dcd732396/volumes"
Oct 30 19:37:35.619153 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:37:35.619076    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-jbsw6" containerName="extractor"
Oct 30 19:37:37.797557 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:37:37.797522    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w4-da1b8b-openshift-default-7876fcf7c5-p4wc5"]
Oct 30 19:37:37.798035 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:37:37.797770    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w4-da1b8b-openshift-default-7876fcf7c5-p4wc5" podUID="73a3358c-ef67-4b35-a010-46c168a95f1e" containerName="istio-proxy" containerID="cri-o://e935d77e8b1bc9d4a13053d6547fb4793ed00bc8827a743a610ebfda6db1a8c5" gracePeriod=30
Oct 30 19:37:39.423824 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:37:39.423788    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-76a1ef-56fdd6dcd8-fz8cg"]
Oct 30 19:37:39.424225 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:37:39.423999    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-76a1ef-56fdd6dcd8-fz8cg" podUID="faa528eb-1c1e-4060-8e7b-317b1900541c" containerName="maas-api" containerID="cri-o://c4dfeec9c5920c3287d3c056d9112d710793457140229c9afb60deeee9c13fb9" gracePeriod=30
Oct 30 19:37:39.721232 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:37:39.721210    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-76a1ef-56fdd6dcd8-fz8cg"
Oct 30 19:37:39.758217 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:37:39.758193    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/faa528eb-1c1e-4060-8e7b-317b1900541c-kube-api-access-vcgrk\" (UniqueName: \"kubernetes.io/projected/faa528eb-1c1e-4060-8e7b-317b1900541c-kube-api-access-vcgrk\") pod \"faa528eb-1c1e-4060-8e7b-317b1900541c\" (UID: \"faa528eb-1c1e-4060-8e7b-317b1900541c\") "
Oct 30 19:37:39.758410 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:37:39.758253    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/faa528eb-1c1e-4060-8e7b-317b1900541c-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/faa528eb-1c1e-4060-8e7b-317b1900541c-maas-api-tls\") pod \"faa528eb-1c1e-4060-8e7b-317b1900541c\" (UID: \"faa528eb-1c1e-4060-8e7b-317b1900541c\") "
Oct 30 19:37:39.758488 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:37:39.758443    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/faa528eb-1c1e-4060-8e7b-317b1900541c-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/faa528eb-1c1e-4060-8e7b-317b1900541c-openshift-service-ca\") pod \"faa528eb-1c1e-4060-8e7b-317b1900541c\" (UID: \"faa528eb-1c1e-4060-8e7b-317b1900541c\") "
Oct 30 19:37:39.758553 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:37:39.758538    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/faa528eb-1c1e-4060-8e7b-317b1900541c-metrics-certs\" (UniqueName: \"kubernetes.io/secret/faa528eb-1c1e-4060-8e7b-317b1900541c-metrics-certs\") pod \"faa528eb-1c1e-4060-8e7b-317b1900541c\" (UID: \"faa528eb-1c1e-4060-8e7b-317b1900541c\") "
Oct 30 19:37:39.758876 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:37:39.758856    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/faa528eb-1c1e-4060-8e7b-317b1900541c-openshift-service-ca" pod "faa528eb-1c1e-4060-8e7b-317b1900541c" (UID: "faa528eb-1c1e-4060-8e7b-317b1900541c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:37:39.758992 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:37:39.758979    4888 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/faa528eb-1c1e-4060-8e7b-317b1900541c-openshift-service-ca\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:37:39.760500 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:37:39.760476    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/faa528eb-1c1e-4060-8e7b-317b1900541c-kube-api-access-vcgrk" pod "faa528eb-1c1e-4060-8e7b-317b1900541c" (UID: "faa528eb-1c1e-4060-8e7b-317b1900541c"). InnerVolumeSpecName "kube-api-access-vcgrk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:37:39.760618 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:37:39.760525    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/faa528eb-1c1e-4060-8e7b-317b1900541c-maas-api-tls" pod "faa528eb-1c1e-4060-8e7b-317b1900541c" (UID: "faa528eb-1c1e-4060-8e7b-317b1900541c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:37:39.760847 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:37:39.760832    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/faa528eb-1c1e-4060-8e7b-317b1900541c-metrics-certs" pod "faa528eb-1c1e-4060-8e7b-317b1900541c" (UID: "faa528eb-1c1e-4060-8e7b-317b1900541c"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:37:39.860193 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:37:39.860161    4888 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/faa528eb-1c1e-4060-8e7b-317b1900541c-metrics-certs\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:37:39.860193 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:37:39.860196    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vcgrk\" (UniqueName: \"kubernetes.io/projected/faa528eb-1c1e-4060-8e7b-317b1900541c-kube-api-access-vcgrk\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:37:39.860434 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:37:39.860212    4888 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/faa528eb-1c1e-4060-8e7b-317b1900541c-maas-api-tls\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:37:40.059368 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:37:40.059268    4888 generic.go:356] "Generic (PLEG): container finished" podID="faa528eb-1c1e-4060-8e7b-317b1900541c" containerID="c4dfeec9c5920c3287d3c056d9112d710793457140229c9afb60deeee9c13fb9" exitCode=0
Oct 30 19:37:40.059368 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:37:40.059335    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-76a1ef-56fdd6dcd8-fz8cg"
Oct 30 19:37:40.059368 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:37:40.059356    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-76a1ef-56fdd6dcd8-fz8cg" event={"ID":"faa528eb-1c1e-4060-8e7b-317b1900541c","Type":"ContainerDied","Data":"c4dfeec9c5920c3287d3c056d9112d710793457140229c9afb60deeee9c13fb9"}
Oct 30 19:37:40.059641 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:37:40.059417    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-76a1ef-56fdd6dcd8-fz8cg" event={"ID":"faa528eb-1c1e-4060-8e7b-317b1900541c","Type":"ContainerDied","Data":"2bf15168a565226aa5651814b556ed56a6a43aac555db84f36cb42ef97ef1d91"}
Oct 30 19:37:40.059641 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:37:40.059442    4888 scope.go:122] "RemoveContainer" containerID="c4dfeec9c5920c3287d3c056d9112d710793457140229c9afb60deeee9c13fb9"
Oct 30 19:37:40.070029 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:37:40.070006    4888 scope.go:122] "RemoveContainer" containerID="c4dfeec9c5920c3287d3c056d9112d710793457140229c9afb60deeee9c13fb9"
Oct 30 19:37:40.070384 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:37:40.070339    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c4dfeec9c5920c3287d3c056d9112d710793457140229c9afb60deeee9c13fb9\": container with ID starting with c4dfeec9c5920c3287d3c056d9112d710793457140229c9afb60deeee9c13fb9 not found: ID does not exist" containerID="c4dfeec9c5920c3287d3c056d9112d710793457140229c9afb60deeee9c13fb9"
Oct 30 19:37:40.070584 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:37:40.070379    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c4dfeec9c5920c3287d3c056d9112d710793457140229c9afb60deeee9c13fb9"} err="failed to get container status \"c4dfeec9c5920c3287d3c056d9112d710793457140229c9afb60deeee9c13fb9\": rpc error: code = NotFound desc = could not find container \"c4dfeec9c5920c3287d3c056d9112d710793457140229c9afb60deeee9c13fb9\": container with ID starting with c4dfeec9c5920c3287d3c056d9112d710793457140229c9afb60deeee9c13fb9 not found: ID does not exist"
Oct 30 19:37:40.240624 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:37:40.240594    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-76a1ef-56fdd6dcd8-fz8cg"]
Oct 30 19:37:40.247589 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:37:40.247559    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-76a1ef-56fdd6dcd8-fz8cg"]
Oct 30 19:37:40.463597 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:37:40.463567    4888 patch_prober.go:29] interesting pod/e2e-shared-a-w4-da1b8b-openshift-default-7876fcf7c5-p4wc5 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 19:37:40.463970 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:37:40.463624    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w4-da1b8b-openshift-default-7876fcf7c5-p4wc5" podUID="73a3358c-ef67-4b35-a010-46c168a95f1e" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 19:37:41.892680 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:37:41.892650    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="faa528eb-1c1e-4060-8e7b-317b1900541c" path="/var/lib/kubelet/pods/faa528eb-1c1e-4060-8e7b-317b1900541c/volumes"
Oct 30 19:37:43.080787 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:37:43.080760    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w4-da1b8b-openshift-default-7876fcf7c5-p4wc5"
Oct 30 19:37:43.092501 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:37:43.092473    4888 generic.go:356] "Generic (PLEG): container finished" podID="73a3358c-ef67-4b35-a010-46c168a95f1e" containerID="e935d77e8b1bc9d4a13053d6547fb4793ed00bc8827a743a610ebfda6db1a8c5" exitCode=0
Oct 30 19:37:43.092662 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:37:43.092539    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w4-da1b8b-openshift-default-7876fcf7c5-p4wc5"
Oct 30 19:37:43.092662 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:37:43.092567    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w4-da1b8b-openshift-default-7876fcf7c5-p4wc5" event={"ID":"73a3358c-ef67-4b35-a010-46c168a95f1e","Type":"ContainerDied","Data":"e935d77e8b1bc9d4a13053d6547fb4793ed00bc8827a743a610ebfda6db1a8c5"}
Oct 30 19:37:43.092662 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:37:43.092632    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w4-da1b8b-openshift-default-7876fcf7c5-p4wc5" event={"ID":"73a3358c-ef67-4b35-a010-46c168a95f1e","Type":"ContainerDied","Data":"1141c20b0e65bcfeb8bb40010c32cb030baf19a23eab9afff19bced9582ea610"}
Oct 30 19:37:43.092814 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:37:43.092663    4888 scope.go:122] "RemoveContainer" containerID="e935d77e8b1bc9d4a13053d6547fb4793ed00bc8827a743a610ebfda6db1a8c5"
Oct 30 19:37:43.103459 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:37:43.103438    4888 scope.go:122] "RemoveContainer" containerID="0cd319ad64495916f49bc6f0c96bfb426c5f4908cc00f00238416829cd194448"
Oct 30 19:37:43.115418 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:37:43.115401    4888 scope.go:122] "RemoveContainer" containerID="e935d77e8b1bc9d4a13053d6547fb4793ed00bc8827a743a610ebfda6db1a8c5"
Oct 30 19:37:43.115720 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:37:43.115697    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e935d77e8b1bc9d4a13053d6547fb4793ed00bc8827a743a610ebfda6db1a8c5\": container with ID starting with e935d77e8b1bc9d4a13053d6547fb4793ed00bc8827a743a610ebfda6db1a8c5 not found: ID does not exist" containerID="e935d77e8b1bc9d4a13053d6547fb4793ed00bc8827a743a610ebfda6db1a8c5"
Oct 30 19:37:43.115793 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:37:43.115732    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e935d77e8b1bc9d4a13053d6547fb4793ed00bc8827a743a610ebfda6db1a8c5"} err="failed to get container status \"e935d77e8b1bc9d4a13053d6547fb4793ed00bc8827a743a610ebfda6db1a8c5\": rpc error: code = NotFound desc = could not find container \"e935d77e8b1bc9d4a13053d6547fb4793ed00bc8827a743a610ebfda6db1a8c5\": container with ID starting with e935d77e8b1bc9d4a13053d6547fb4793ed00bc8827a743a610ebfda6db1a8c5 not found: ID does not exist"
Oct 30 19:37:43.115793 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:37:43.115759    4888 scope.go:122] "RemoveContainer" containerID="0cd319ad64495916f49bc6f0c96bfb426c5f4908cc00f00238416829cd194448"
Oct 30 19:37:43.116009 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:37:43.115990    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0cd319ad64495916f49bc6f0c96bfb426c5f4908cc00f00238416829cd194448\": container with ID starting with 0cd319ad64495916f49bc6f0c96bfb426c5f4908cc00f00238416829cd194448 not found: ID does not exist" containerID="0cd319ad64495916f49bc6f0c96bfb426c5f4908cc00f00238416829cd194448"
Oct 30 19:37:43.116069 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:37:43.116020    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0cd319ad64495916f49bc6f0c96bfb426c5f4908cc00f00238416829cd194448"} err="failed to get container status \"0cd319ad64495916f49bc6f0c96bfb426c5f4908cc00f00238416829cd194448\": rpc error: code = NotFound desc = could not find container \"0cd319ad64495916f49bc6f0c96bfb426c5f4908cc00f00238416829cd194448\": container with ID starting with 0cd319ad64495916f49bc6f0c96bfb426c5f4908cc00f00238416829cd194448 not found: ID does not exist"
Oct 30 19:37:43.204582 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:37:43.204555    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/73a3358c-ef67-4b35-a010-46c168a95f1e-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/73a3358c-ef67-4b35-a010-46c168a95f1e-workload-socket\") pod \"73a3358c-ef67-4b35-a010-46c168a95f1e\" (UID: \"73a3358c-ef67-4b35-a010-46c168a95f1e\") "
Oct 30 19:37:43.204754 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:37:43.204593    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/73a3358c-ef67-4b35-a010-46c168a95f1e-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/73a3358c-ef67-4b35-a010-46c168a95f1e-istio-envoy\") pod \"73a3358c-ef67-4b35-a010-46c168a95f1e\" (UID: \"73a3358c-ef67-4b35-a010-46c168a95f1e\") "
Oct 30 19:37:43.204754 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:37:43.204616    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/73a3358c-ef67-4b35-a010-46c168a95f1e-kube-api-access-ww4n4\" (UniqueName: \"kubernetes.io/projected/73a3358c-ef67-4b35-a010-46c168a95f1e-kube-api-access-ww4n4\") pod \"73a3358c-ef67-4b35-a010-46c168a95f1e\" (UID: \"73a3358c-ef67-4b35-a010-46c168a95f1e\") "
Oct 30 19:37:43.204754 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:37:43.204651    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/73a3358c-ef67-4b35-a010-46c168a95f1e-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/73a3358c-ef67-4b35-a010-46c168a95f1e-istiod-ca-cert\") pod \"73a3358c-ef67-4b35-a010-46c168a95f1e\" (UID: \"73a3358c-ef67-4b35-a010-46c168a95f1e\") "
Oct 30 19:37:43.204754 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:37:43.204712    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/73a3358c-ef67-4b35-a010-46c168a95f1e-istio-token\" (UniqueName: \"kubernetes.io/projected/73a3358c-ef67-4b35-a010-46c168a95f1e-istio-token\") pod \"73a3358c-ef67-4b35-a010-46c168a95f1e\" (UID: \"73a3358c-ef67-4b35-a010-46c168a95f1e\") "
Oct 30 19:37:43.204976 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:37:43.204797    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/73a3358c-ef67-4b35-a010-46c168a95f1e-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/73a3358c-ef67-4b35-a010-46c168a95f1e-istio-podinfo\") pod \"73a3358c-ef67-4b35-a010-46c168a95f1e\" (UID: \"73a3358c-ef67-4b35-a010-46c168a95f1e\") "
Oct 30 19:37:43.204976 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:37:43.204870    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73a3358c-ef67-4b35-a010-46c168a95f1e-workload-socket" pod "73a3358c-ef67-4b35-a010-46c168a95f1e" (UID: "73a3358c-ef67-4b35-a010-46c168a95f1e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:37:43.204976 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:37:43.204921    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/73a3358c-ef67-4b35-a010-46c168a95f1e-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/73a3358c-ef67-4b35-a010-46c168a95f1e-istio-data\") pod \"73a3358c-ef67-4b35-a010-46c168a95f1e\" (UID: \"73a3358c-ef67-4b35-a010-46c168a95f1e\") "
Oct 30 19:37:43.205149 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:37:43.205053    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/73a3358c-ef67-4b35-a010-46c168a95f1e-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/73a3358c-ef67-4b35-a010-46c168a95f1e-workload-certs\") pod \"73a3358c-ef67-4b35-a010-46c168a95f1e\" (UID: \"73a3358c-ef67-4b35-a010-46c168a95f1e\") "
Oct 30 19:37:43.205149 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:37:43.205088    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/73a3358c-ef67-4b35-a010-46c168a95f1e-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/73a3358c-ef67-4b35-a010-46c168a95f1e-credential-socket\") pod \"73a3358c-ef67-4b35-a010-46c168a95f1e\" (UID: \"73a3358c-ef67-4b35-a010-46c168a95f1e\") "
Oct 30 19:37:43.205488 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:37:43.205340    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73a3358c-ef67-4b35-a010-46c168a95f1e-istio-data" pod "73a3358c-ef67-4b35-a010-46c168a95f1e" (UID: "73a3358c-ef67-4b35-a010-46c168a95f1e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:37:43.205598 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:37:43.205530    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73a3358c-ef67-4b35-a010-46c168a95f1e-credential-socket" pod "73a3358c-ef67-4b35-a010-46c168a95f1e" (UID: "73a3358c-ef67-4b35-a010-46c168a95f1e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:37:43.205598 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:37:43.205597    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/73a3358c-ef67-4b35-a010-46c168a95f1e-istiod-ca-cert" pod "73a3358c-ef67-4b35-a010-46c168a95f1e" (UID: "73a3358c-ef67-4b35-a010-46c168a95f1e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:37:43.205888 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:37:43.205851    4888 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/73a3358c-ef67-4b35-a010-46c168a95f1e-credential-socket\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:37:43.206039 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:37:43.205944    4888 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/73a3358c-ef67-4b35-a010-46c168a95f1e-workload-socket\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:37:43.206039 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:37:43.206000    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73a3358c-ef67-4b35-a010-46c168a95f1e-workload-certs" pod "73a3358c-ef67-4b35-a010-46c168a95f1e" (UID: "73a3358c-ef67-4b35-a010-46c168a95f1e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:37:43.206039 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:37:43.205987    4888 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/73a3358c-ef67-4b35-a010-46c168a95f1e-istiod-ca-cert\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:37:43.206229 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:37:43.206073    4888 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/73a3358c-ef67-4b35-a010-46c168a95f1e-istio-data\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:37:43.207324 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:37:43.207300    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73a3358c-ef67-4b35-a010-46c168a95f1e-istio-envoy" pod "73a3358c-ef67-4b35-a010-46c168a95f1e" (UID: "73a3358c-ef67-4b35-a010-46c168a95f1e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:37:43.207453 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:37:43.207384    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/73a3358c-ef67-4b35-a010-46c168a95f1e-istio-podinfo" pod "73a3358c-ef67-4b35-a010-46c168a95f1e" (UID: "73a3358c-ef67-4b35-a010-46c168a95f1e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:37:43.207646 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:37:43.207625    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/73a3358c-ef67-4b35-a010-46c168a95f1e-istio-token" pod "73a3358c-ef67-4b35-a010-46c168a95f1e" (UID: "73a3358c-ef67-4b35-a010-46c168a95f1e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:37:43.207854 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:37:43.207825    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/73a3358c-ef67-4b35-a010-46c168a95f1e-kube-api-access-ww4n4" pod "73a3358c-ef67-4b35-a010-46c168a95f1e" (UID: "73a3358c-ef67-4b35-a010-46c168a95f1e"). InnerVolumeSpecName "kube-api-access-ww4n4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:37:43.307629 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:37:43.307597    4888 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/73a3358c-ef67-4b35-a010-46c168a95f1e-workload-certs\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:37:43.307629 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:37:43.307628    4888 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/73a3358c-ef67-4b35-a010-46c168a95f1e-istio-envoy\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:37:43.307833 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:37:43.307643    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ww4n4\" (UniqueName: \"kubernetes.io/projected/73a3358c-ef67-4b35-a010-46c168a95f1e-kube-api-access-ww4n4\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:37:43.307833 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:37:43.307661    4888 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/73a3358c-ef67-4b35-a010-46c168a95f1e-istio-token\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:37:43.307833 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:37:43.307675    4888 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/73a3358c-ef67-4b35-a010-46c168a95f1e-istio-podinfo\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:37:43.518962 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:37:43.518930    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w4-da1b8b-openshift-default-7876fcf7c5-p4wc5"]
Oct 30 19:37:43.523516 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:37:43.523486    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w4-da1b8b-openshift-default-7876fcf7c5-p4wc5"]
Oct 30 19:37:43.892055 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:37:43.892019    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="73a3358c-ef67-4b35-a010-46c168a95f1e" path="/var/lib/kubelet/pods/73a3358c-ef67-4b35-a010-46c168a95f1e/volumes"
Oct 30 19:37:44.381721 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:37:44.381685    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-rjmdz"]
Oct 30 19:37:53.912697 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:37:53.912664    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w4-0f83f5-55dd878599-fk8l7"]
Oct 30 19:37:58.913367 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:37:58.913334    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w4-0f83f5-55dd878599-fk8l7" podUID="3f801255-a7eb-4a5f-b80f-82c3c0a6a8a6" containerName="payload-pre-processing" containerID="cri-o://9d491dd29d68775c97dcbc17ecc25e2230b972be1d98680c14c2baf555b0402e" gracePeriod=25
Oct 30 19:37:59.182965 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:37:59.182940    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w4-0f83f5-55dd878599-fk8l7"
Oct 30 19:37:59.233956 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:37:59.233920    4888 generic.go:356] "Generic (PLEG): container finished" podID="3f801255-a7eb-4a5f-b80f-82c3c0a6a8a6" containerID="9d491dd29d68775c97dcbc17ecc25e2230b972be1d98680c14c2baf555b0402e" exitCode=0
Oct 30 19:37:59.234145 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:37:59.233979    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w4-0f83f5-55dd878599-fk8l7"
Oct 30 19:37:59.234145 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:37:59.234015    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w4-0f83f5-55dd878599-fk8l7" event={"ID":"3f801255-a7eb-4a5f-b80f-82c3c0a6a8a6","Type":"ContainerDied","Data":"9d491dd29d68775c97dcbc17ecc25e2230b972be1d98680c14c2baf555b0402e"}
Oct 30 19:37:59.234145 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:37:59.234080    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w4-0f83f5-55dd878599-fk8l7" event={"ID":"3f801255-a7eb-4a5f-b80f-82c3c0a6a8a6","Type":"ContainerDied","Data":"fd347c74b1b874afb380a2b3cf3787d8d80dc5645e78d5bf193eac0fa16742d8"}
Oct 30 19:37:59.234145 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:37:59.234110    4888 scope.go:122] "RemoveContainer" containerID="9d491dd29d68775c97dcbc17ecc25e2230b972be1d98680c14c2baf555b0402e"
Oct 30 19:37:59.242500 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:37:59.242470    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3f801255-a7eb-4a5f-b80f-82c3c0a6a8a6-kube-api-access-cdvkd\" (UniqueName: \"kubernetes.io/projected/3f801255-a7eb-4a5f-b80f-82c3c0a6a8a6-kube-api-access-cdvkd\") pod \"3f801255-a7eb-4a5f-b80f-82c3c0a6a8a6\" (UID: \"3f801255-a7eb-4a5f-b80f-82c3c0a6a8a6\") "
Oct 30 19:37:59.242619 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:37:59.242599    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3f801255-a7eb-4a5f-b80f-82c3c0a6a8a6-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3f801255-a7eb-4a5f-b80f-82c3c0a6a8a6-plugins-config-volume\") pod \"3f801255-a7eb-4a5f-b80f-82c3c0a6a8a6\" (UID: \"3f801255-a7eb-4a5f-b80f-82c3c0a6a8a6\") "
Oct 30 19:37:59.242992 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:37:59.242969    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3f801255-a7eb-4a5f-b80f-82c3c0a6a8a6-plugins-config-volume" pod "3f801255-a7eb-4a5f-b80f-82c3c0a6a8a6" (UID: "3f801255-a7eb-4a5f-b80f-82c3c0a6a8a6"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:37:59.243203 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:37:59.243182    4888 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3f801255-a7eb-4a5f-b80f-82c3c0a6a8a6-plugins-config-volume\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:37:59.245197 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:37:59.245178    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3f801255-a7eb-4a5f-b80f-82c3c0a6a8a6-kube-api-access-cdvkd" pod "3f801255-a7eb-4a5f-b80f-82c3c0a6a8a6" (UID: "3f801255-a7eb-4a5f-b80f-82c3c0a6a8a6"). InnerVolumeSpecName "kube-api-access-cdvkd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:37:59.246205 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:37:59.246176    4888 scope.go:122] "RemoveContainer" containerID="9d491dd29d68775c97dcbc17ecc25e2230b972be1d98680c14c2baf555b0402e"
Oct 30 19:37:59.246545 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:37:59.246517    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9d491dd29d68775c97dcbc17ecc25e2230b972be1d98680c14c2baf555b0402e\": container with ID starting with 9d491dd29d68775c97dcbc17ecc25e2230b972be1d98680c14c2baf555b0402e not found: ID does not exist" containerID="9d491dd29d68775c97dcbc17ecc25e2230b972be1d98680c14c2baf555b0402e"
Oct 30 19:37:59.246613 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:37:59.246556    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9d491dd29d68775c97dcbc17ecc25e2230b972be1d98680c14c2baf555b0402e"} err="failed to get container status \"9d491dd29d68775c97dcbc17ecc25e2230b972be1d98680c14c2baf555b0402e\": rpc error: code = NotFound desc = could not find container \"9d491dd29d68775c97dcbc17ecc25e2230b972be1d98680c14c2baf555b0402e\": container with ID starting with 9d491dd29d68775c97dcbc17ecc25e2230b972be1d98680c14c2baf555b0402e not found: ID does not exist"
Oct 30 19:37:59.344055 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:37:59.344026    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cdvkd\" (UniqueName: \"kubernetes.io/projected/3f801255-a7eb-4a5f-b80f-82c3c0a6a8a6-kube-api-access-cdvkd\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:37:59.670498 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:37:59.670466    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w4-0f83f5-55dd878599-fk8l7"]
Oct 30 19:37:59.673585 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:37:59.673562    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w4-0f83f5-55dd878599-fk8l7"]
Oct 30 19:37:59.897065 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:37:59.897005    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3f801255-a7eb-4a5f-b80f-82c3c0a6a8a6" path="/var/lib/kubelet/pods/3f801255-a7eb-4a5f-b80f-82c3c0a6a8a6/volumes"
Oct 30 19:38:02.324362 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:02.324335    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-a-7fdf6b-openshift-default-65b5b7b9b5-8q2gn"]
Oct 30 19:38:02.324794 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:02.324525    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-extmod-a-7fdf6b-openshift-default-65b5b7b9b5-8q2gn" podUID="6adbce7d-d243-4f53-b2f0-e1db8b942eb1" containerName="istio-proxy" containerID="cri-o://634d90cecd836dd461e5e997d9b9f40a1b832e04238b11c6d484e26dc1e040e8" gracePeriod=30
Oct 30 19:38:04.366209 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:04.366170    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-3b6013/distinct-2-w3-3b6013-kserve-7cc5454db-mqknj"]
Oct 30 19:38:04.415111 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:04.415078    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-3b6013/distinct-w3-3b6013-kserve-857f55d885-8fcrq"]
Oct 30 19:38:04.418640 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:04.418605    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-3b6013/premium-w3-3b6013-kserve-74df5fb95d-k747z"]
Oct 30 19:38:05.618114 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:38:05.618082    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-jbsw6" containerName="extractor"
Oct 30 19:38:07.609064 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:07.609004    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-7fdf6b-openshift-default-65b5b7b9b5-8q2gn"
Oct 30 19:38:07.778958 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:07.778919    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6adbce7d-d243-4f53-b2f0-e1db8b942eb1-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6adbce7d-d243-4f53-b2f0-e1db8b942eb1-istio-data\") pod \"6adbce7d-d243-4f53-b2f0-e1db8b942eb1\" (UID: \"6adbce7d-d243-4f53-b2f0-e1db8b942eb1\") "
Oct 30 19:38:07.779158 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:07.778990    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6adbce7d-d243-4f53-b2f0-e1db8b942eb1-kube-api-access-xzdsr\" (UniqueName: \"kubernetes.io/projected/6adbce7d-d243-4f53-b2f0-e1db8b942eb1-kube-api-access-xzdsr\") pod \"6adbce7d-d243-4f53-b2f0-e1db8b942eb1\" (UID: \"6adbce7d-d243-4f53-b2f0-e1db8b942eb1\") "
Oct 30 19:38:07.779158 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:07.779036    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6adbce7d-d243-4f53-b2f0-e1db8b942eb1-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6adbce7d-d243-4f53-b2f0-e1db8b942eb1-workload-socket\") pod \"6adbce7d-d243-4f53-b2f0-e1db8b942eb1\" (UID: \"6adbce7d-d243-4f53-b2f0-e1db8b942eb1\") "
Oct 30 19:38:07.779158 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:07.779079    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/6adbce7d-d243-4f53-b2f0-e1db8b942eb1-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6adbce7d-d243-4f53-b2f0-e1db8b942eb1-istio-podinfo\") pod \"6adbce7d-d243-4f53-b2f0-e1db8b942eb1\" (UID: \"6adbce7d-d243-4f53-b2f0-e1db8b942eb1\") "
Oct 30 19:38:07.779158 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:07.779133    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6adbce7d-d243-4f53-b2f0-e1db8b942eb1-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6adbce7d-d243-4f53-b2f0-e1db8b942eb1-istiod-ca-cert\") pod \"6adbce7d-d243-4f53-b2f0-e1db8b942eb1\" (UID: \"6adbce7d-d243-4f53-b2f0-e1db8b942eb1\") "
Oct 30 19:38:07.779407 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:07.779172    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6adbce7d-d243-4f53-b2f0-e1db8b942eb1-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6adbce7d-d243-4f53-b2f0-e1db8b942eb1-credential-socket\") pod \"6adbce7d-d243-4f53-b2f0-e1db8b942eb1\" (UID: \"6adbce7d-d243-4f53-b2f0-e1db8b942eb1\") "
Oct 30 19:38:07.779407 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:07.779201    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6adbce7d-d243-4f53-b2f0-e1db8b942eb1-istio-data" pod "6adbce7d-d243-4f53-b2f0-e1db8b942eb1" (UID: "6adbce7d-d243-4f53-b2f0-e1db8b942eb1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:38:07.779407 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:07.779240    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6adbce7d-d243-4f53-b2f0-e1db8b942eb1-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6adbce7d-d243-4f53-b2f0-e1db8b942eb1-istio-envoy\") pod \"6adbce7d-d243-4f53-b2f0-e1db8b942eb1\" (UID: \"6adbce7d-d243-4f53-b2f0-e1db8b942eb1\") "
Oct 30 19:38:07.779407 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:07.779270    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6adbce7d-d243-4f53-b2f0-e1db8b942eb1-istio-token\" (UniqueName: \"kubernetes.io/projected/6adbce7d-d243-4f53-b2f0-e1db8b942eb1-istio-token\") pod \"6adbce7d-d243-4f53-b2f0-e1db8b942eb1\" (UID: \"6adbce7d-d243-4f53-b2f0-e1db8b942eb1\") "
Oct 30 19:38:07.779650 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:07.779427    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6adbce7d-d243-4f53-b2f0-e1db8b942eb1-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6adbce7d-d243-4f53-b2f0-e1db8b942eb1-workload-certs\") pod \"6adbce7d-d243-4f53-b2f0-e1db8b942eb1\" (UID: \"6adbce7d-d243-4f53-b2f0-e1db8b942eb1\") "
Oct 30 19:38:07.779650 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:07.779553    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6adbce7d-d243-4f53-b2f0-e1db8b942eb1-credential-socket" pod "6adbce7d-d243-4f53-b2f0-e1db8b942eb1" (UID: "6adbce7d-d243-4f53-b2f0-e1db8b942eb1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:38:07.779650 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:07.779578    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6adbce7d-d243-4f53-b2f0-e1db8b942eb1-istiod-ca-cert" pod "6adbce7d-d243-4f53-b2f0-e1db8b942eb1" (UID: "6adbce7d-d243-4f53-b2f0-e1db8b942eb1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:38:07.779898 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:07.779667    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6adbce7d-d243-4f53-b2f0-e1db8b942eb1-workload-socket" pod "6adbce7d-d243-4f53-b2f0-e1db8b942eb1" (UID: "6adbce7d-d243-4f53-b2f0-e1db8b942eb1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:38:07.779898 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:07.779821    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6adbce7d-d243-4f53-b2f0-e1db8b942eb1-workload-certs" pod "6adbce7d-d243-4f53-b2f0-e1db8b942eb1" (UID: "6adbce7d-d243-4f53-b2f0-e1db8b942eb1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:38:07.780091 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:07.779994    4888 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6adbce7d-d243-4f53-b2f0-e1db8b942eb1-workload-certs\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:38:07.780091 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:07.780012    4888 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6adbce7d-d243-4f53-b2f0-e1db8b942eb1-istio-data\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:38:07.780091 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:07.780025    4888 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6adbce7d-d243-4f53-b2f0-e1db8b942eb1-workload-socket\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:38:07.780091 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:07.780040    4888 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6adbce7d-d243-4f53-b2f0-e1db8b942eb1-istiod-ca-cert\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:38:07.780091 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:07.780054    4888 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6adbce7d-d243-4f53-b2f0-e1db8b942eb1-credential-socket\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:38:07.781682 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:07.781654    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6adbce7d-d243-4f53-b2f0-e1db8b942eb1-kube-api-access-xzdsr" pod "6adbce7d-d243-4f53-b2f0-e1db8b942eb1" (UID: "6adbce7d-d243-4f53-b2f0-e1db8b942eb1"). InnerVolumeSpecName "kube-api-access-xzdsr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:38:07.781822 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:07.781720    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6adbce7d-d243-4f53-b2f0-e1db8b942eb1-istio-token" pod "6adbce7d-d243-4f53-b2f0-e1db8b942eb1" (UID: "6adbce7d-d243-4f53-b2f0-e1db8b942eb1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:38:07.782116 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:07.782094    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6adbce7d-d243-4f53-b2f0-e1db8b942eb1-istio-podinfo" pod "6adbce7d-d243-4f53-b2f0-e1db8b942eb1" (UID: "6adbce7d-d243-4f53-b2f0-e1db8b942eb1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:38:07.782256 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:07.782239    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6adbce7d-d243-4f53-b2f0-e1db8b942eb1-istio-envoy" pod "6adbce7d-d243-4f53-b2f0-e1db8b942eb1" (UID: "6adbce7d-d243-4f53-b2f0-e1db8b942eb1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:38:07.881025 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:07.880932    4888 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6adbce7d-d243-4f53-b2f0-e1db8b942eb1-istio-envoy\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:38:07.881025 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:07.880968    4888 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6adbce7d-d243-4f53-b2f0-e1db8b942eb1-istio-token\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:38:07.881025 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:07.880983    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xzdsr\" (UniqueName: \"kubernetes.io/projected/6adbce7d-d243-4f53-b2f0-e1db8b942eb1-kube-api-access-xzdsr\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:38:07.881025 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:07.880999    4888 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6adbce7d-d243-4f53-b2f0-e1db8b942eb1-istio-podinfo\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:38:08.307077 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:08.306991    4888 generic.go:356] "Generic (PLEG): container finished" podID="6adbce7d-d243-4f53-b2f0-e1db8b942eb1" containerID="634d90cecd836dd461e5e997d9b9f40a1b832e04238b11c6d484e26dc1e040e8" exitCode=0
Oct 30 19:38:08.307262 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:08.307074    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-7fdf6b-openshift-default-65b5b7b9b5-8q2gn"
Oct 30 19:38:08.307262 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:08.307073    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-7fdf6b-openshift-default-65b5b7b9b5-8q2gn" event={"ID":"6adbce7d-d243-4f53-b2f0-e1db8b942eb1","Type":"ContainerDied","Data":"634d90cecd836dd461e5e997d9b9f40a1b832e04238b11c6d484e26dc1e040e8"}
Oct 30 19:38:08.307262 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:08.307178    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-7fdf6b-openshift-default-65b5b7b9b5-8q2gn" event={"ID":"6adbce7d-d243-4f53-b2f0-e1db8b942eb1","Type":"ContainerDied","Data":"269f6a4e5c8275b6fb8524153fbf98517109a46316a6b9d81ad5c30473a75d2d"}
Oct 30 19:38:08.307262 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:08.307208    4888 scope.go:122] "RemoveContainer" containerID="634d90cecd836dd461e5e997d9b9f40a1b832e04238b11c6d484e26dc1e040e8"
Oct 30 19:38:08.318373 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:08.318352    4888 scope.go:122] "RemoveContainer" containerID="fdb6c2b135cb669ebf140f23063ba340a8cc37075c9dd94958b33da87e797b4d"
Oct 30 19:38:08.328795 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:08.328774    4888 scope.go:122] "RemoveContainer" containerID="634d90cecd836dd461e5e997d9b9f40a1b832e04238b11c6d484e26dc1e040e8"
Oct 30 19:38:08.329079 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:38:08.329056    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"634d90cecd836dd461e5e997d9b9f40a1b832e04238b11c6d484e26dc1e040e8\": container with ID starting with 634d90cecd836dd461e5e997d9b9f40a1b832e04238b11c6d484e26dc1e040e8 not found: ID does not exist" containerID="634d90cecd836dd461e5e997d9b9f40a1b832e04238b11c6d484e26dc1e040e8"
Oct 30 19:38:08.329152 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:08.329090    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"634d90cecd836dd461e5e997d9b9f40a1b832e04238b11c6d484e26dc1e040e8"} err="failed to get container status \"634d90cecd836dd461e5e997d9b9f40a1b832e04238b11c6d484e26dc1e040e8\": rpc error: code = NotFound desc = could not find container \"634d90cecd836dd461e5e997d9b9f40a1b832e04238b11c6d484e26dc1e040e8\": container with ID starting with 634d90cecd836dd461e5e997d9b9f40a1b832e04238b11c6d484e26dc1e040e8 not found: ID does not exist"
Oct 30 19:38:08.329152 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:08.329110    4888 scope.go:122] "RemoveContainer" containerID="fdb6c2b135cb669ebf140f23063ba340a8cc37075c9dd94958b33da87e797b4d"
Oct 30 19:38:08.329415 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:38:08.329397    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fdb6c2b135cb669ebf140f23063ba340a8cc37075c9dd94958b33da87e797b4d\": container with ID starting with fdb6c2b135cb669ebf140f23063ba340a8cc37075c9dd94958b33da87e797b4d not found: ID does not exist" containerID="fdb6c2b135cb669ebf140f23063ba340a8cc37075c9dd94958b33da87e797b4d"
Oct 30 19:38:08.329473 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:08.329422    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fdb6c2b135cb669ebf140f23063ba340a8cc37075c9dd94958b33da87e797b4d"} err="failed to get container status \"fdb6c2b135cb669ebf140f23063ba340a8cc37075c9dd94958b33da87e797b4d\": rpc error: code = NotFound desc = could not find container \"fdb6c2b135cb669ebf140f23063ba340a8cc37075c9dd94958b33da87e797b4d\": container with ID starting with fdb6c2b135cb669ebf140f23063ba340a8cc37075c9dd94958b33da87e797b4d not found: ID does not exist"
Oct 30 19:38:08.480070 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:08.480027    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-a-7fdf6b-openshift-default-65b5b7b9b5-8q2gn"]
Oct 30 19:38:08.528786 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:08.528752    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-extmod-a-7fdf6b-openshift-default-65b5b7b9b5-8q2gn"]
Oct 30 19:38:09.891096 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:09.891063    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6adbce7d-d243-4f53-b2f0-e1db8b942eb1" path="/var/lib/kubelet/pods/6adbce7d-d243-4f53-b2f0-e1db8b942eb1/volumes"
Oct 30 19:38:11.574074 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:11.574034    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-3b6013-84b5f84d8b-hfxgw"]
Oct 30 19:38:11.718079 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:11.718037    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-3b6013-57fbf8ccf5-bz979"]
Oct 30 19:38:14.323071 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:14.323033    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-185e8b-98bb8ccf5-lcntb"]
Oct 30 19:38:16.574931 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:16.574892    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w3-3b6013-84b5f84d8b-hfxgw" podUID="337a7530-cf88-4466-818c-d025be569509" containerName="payload-processing" containerID="cri-o://77c4fc34120888101629aa521c214162d2dd7365dc335fdc65ac5bbf29b2708b" gracePeriod=25
Oct 30 19:38:16.718695 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:16.718638    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-3b6013-57fbf8ccf5-bz979" podUID="42a5ae09-5a33-4744-9e64-8e0f0084cd97" containerName="payload-pre-processing" containerID="cri-o://db05e15f0e6b4bcf13a007cc06289b5ea825e1febc79948819c7f11b0d1a7e79" gracePeriod=25
Oct 30 19:38:19.323362 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:19.323323    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-185e8b-98bb8ccf5-lcntb" podUID="56a9a3f1-852b-458f-979f-1b815f8175d7" containerName="payload-pre-processing" containerID="cri-o://1573430f9974d1b1c497479a9bbef5c737901a3ea9c6bb4b21dfc3cfd71c2250" gracePeriod=25
Oct 30 19:38:19.388393 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:19.388344    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w3-3b6013/distinct-2-w3-3b6013-kserve-7cc5454db-mqknj" podUID="e76c7b20-be2f-4ed2-8dc4-bef127ecc0c9" containerName="main" containerID="cri-o://539d053fe28c0a83e4a27cf2c3b304bc6df03c142a6f0cfbf7d6be44c8d708d4" gracePeriod=45
Oct 30 19:38:19.441051 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:19.441006    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w3-3b6013/premium-w3-3b6013-kserve-74df5fb95d-k747z" podUID="a9309776-20d5-4c06-bcae-04b8e54493e8" containerName="main" containerID="cri-o://b8f7a11879729dfdaeb18f36e51d7057e97444b00a31e47fbb3c2e005ae8f364" gracePeriod=45
Oct 30 19:38:19.441276 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:19.441240    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w3-3b6013/distinct-w3-3b6013-kserve-857f55d885-8fcrq" podUID="51927877-4d99-4797-82f7-1684f6ecda6b" containerName="main" containerID="cri-o://03797d706f04895778b42aca80253000864743c3106c238d298f3f547c4b2755" gracePeriod=45
Oct 30 19:38:19.446789 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:19.446747    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-3b6013/premium-w3-3b6013-kserve-74df5fb95d-k747z" podUID="a9309776-20d5-4c06-bcae-04b8e54493e8" containerName="main" probeResult="failure" output="Get \"https://10.132.0.82:8000/ready\": EOF"
Oct 30 19:38:19.499466 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:19.499439    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-rjmdz"]
Oct 30 19:38:19.502592 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:38:19.502563    4888 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod51927877_4d99_4797_82f7_1684f6ecda6b.slice/crio-03797d706f04895778b42aca80253000864743c3106c238d298f3f547c4b2755.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda9309776_20d5_4c06_bcae_04b8e54493e8.slice/crio-b8f7a11879729dfdaeb18f36e51d7057e97444b00a31e47fbb3c2e005ae8f364.scope\": RecentStats: unable to find data in memory cache]"
Oct 30 19:38:19.587901 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:19.587813    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-3b6013/distinct-w3-3b6013-kserve-857f55d885-8fcrq" podUID="51927877-4d99-4797-82f7-1684f6ecda6b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.92:8000/ready\": dial tcp 10.132.0.92:8000: connect: connection refused"
Oct 30 19:38:19.683923 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:19.683895    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-185e8b-98bb8ccf5-lcntb"
Oct 30 19:38:19.730310 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:19.730248    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/56a9a3f1-852b-458f-979f-1b815f8175d7-kube-api-access-jqrp2\" (UniqueName: \"kubernetes.io/projected/56a9a3f1-852b-458f-979f-1b815f8175d7-kube-api-access-jqrp2\") pod \"56a9a3f1-852b-458f-979f-1b815f8175d7\" (UID: \"56a9a3f1-852b-458f-979f-1b815f8175d7\") "
Oct 30 19:38:19.730488 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:19.730392    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/56a9a3f1-852b-458f-979f-1b815f8175d7-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/56a9a3f1-852b-458f-979f-1b815f8175d7-plugins-config-volume\") pod \"56a9a3f1-852b-458f-979f-1b815f8175d7\" (UID: \"56a9a3f1-852b-458f-979f-1b815f8175d7\") "
Oct 30 19:38:19.731536 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:19.730974    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/56a9a3f1-852b-458f-979f-1b815f8175d7-plugins-config-volume" pod "56a9a3f1-852b-458f-979f-1b815f8175d7" (UID: "56a9a3f1-852b-458f-979f-1b815f8175d7"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:38:19.733733 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:19.733698    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/56a9a3f1-852b-458f-979f-1b815f8175d7-kube-api-access-jqrp2" pod "56a9a3f1-852b-458f-979f-1b815f8175d7" (UID: "56a9a3f1-852b-458f-979f-1b815f8175d7"). InnerVolumeSpecName "kube-api-access-jqrp2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:38:19.741343 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:19.741321    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-3b6013/distinct-2-w3-3b6013-kserve-7cc5454db-mqknj"
Oct 30 19:38:19.807650 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:19.807628    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-3b6013/distinct-w3-3b6013-kserve-857f55d885-8fcrq"
Oct 30 19:38:19.812728 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:19.812704    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-3b6013/premium-w3-3b6013-kserve-74df5fb95d-k747z"
Oct 30 19:38:19.831748 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:19.831724    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/51927877-4d99-4797-82f7-1684f6ecda6b-kube-api-access-6nd5j\" (UniqueName: \"kubernetes.io/projected/51927877-4d99-4797-82f7-1684f6ecda6b-kube-api-access-6nd5j\") pod \"51927877-4d99-4797-82f7-1684f6ecda6b\" (UID: \"51927877-4d99-4797-82f7-1684f6ecda6b\") "
Oct 30 19:38:19.831877 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:19.831777    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/51927877-4d99-4797-82f7-1684f6ecda6b-tls-certs\" (UniqueName: \"kubernetes.io/secret/51927877-4d99-4797-82f7-1684f6ecda6b-tls-certs\") pod \"51927877-4d99-4797-82f7-1684f6ecda6b\" (UID: \"51927877-4d99-4797-82f7-1684f6ecda6b\") "
Oct 30 19:38:19.831877 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:19.831797    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/51927877-4d99-4797-82f7-1684f6ecda6b-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/51927877-4d99-4797-82f7-1684f6ecda6b-tmp-dir\") pod \"51927877-4d99-4797-82f7-1684f6ecda6b\" (UID: \"51927877-4d99-4797-82f7-1684f6ecda6b\") "
Oct 30 19:38:19.831877 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:19.831816    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e76c7b20-be2f-4ed2-8dc4-bef127ecc0c9-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e76c7b20-be2f-4ed2-8dc4-bef127ecc0c9-tmp-dir\") pod \"e76c7b20-be2f-4ed2-8dc4-bef127ecc0c9\" (UID: \"e76c7b20-be2f-4ed2-8dc4-bef127ecc0c9\") "
Oct 30 19:38:19.831877 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:19.831836    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e76c7b20-be2f-4ed2-8dc4-bef127ecc0c9-tls-certs\" (UniqueName: \"kubernetes.io/secret/e76c7b20-be2f-4ed2-8dc4-bef127ecc0c9-tls-certs\") pod \"e76c7b20-be2f-4ed2-8dc4-bef127ecc0c9\" (UID: \"e76c7b20-be2f-4ed2-8dc4-bef127ecc0c9\") "
Oct 30 19:38:19.831877 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:19.831868    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e76c7b20-be2f-4ed2-8dc4-bef127ecc0c9-kube-api-access-h22l6\" (UniqueName: \"kubernetes.io/projected/e76c7b20-be2f-4ed2-8dc4-bef127ecc0c9-kube-api-access-h22l6\") pod \"e76c7b20-be2f-4ed2-8dc4-bef127ecc0c9\" (UID: \"e76c7b20-be2f-4ed2-8dc4-bef127ecc0c9\") "
Oct 30 19:38:19.832158 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:19.831917    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e76c7b20-be2f-4ed2-8dc4-bef127ecc0c9-home\" (UniqueName: \"kubernetes.io/empty-dir/e76c7b20-be2f-4ed2-8dc4-bef127ecc0c9-home\") pod \"e76c7b20-be2f-4ed2-8dc4-bef127ecc0c9\" (UID: \"e76c7b20-be2f-4ed2-8dc4-bef127ecc0c9\") "
Oct 30 19:38:19.832158 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:19.832039    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/51927877-4d99-4797-82f7-1684f6ecda6b-home\" (UniqueName: \"kubernetes.io/empty-dir/51927877-4d99-4797-82f7-1684f6ecda6b-home\") pod \"51927877-4d99-4797-82f7-1684f6ecda6b\" (UID: \"51927877-4d99-4797-82f7-1684f6ecda6b\") "
Oct 30 19:38:19.832158 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:19.832082    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e76c7b20-be2f-4ed2-8dc4-bef127ecc0c9-tmp-dir" pod "e76c7b20-be2f-4ed2-8dc4-bef127ecc0c9" (UID: "e76c7b20-be2f-4ed2-8dc4-bef127ecc0c9"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:38:19.832158 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:19.832091    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e76c7b20-be2f-4ed2-8dc4-bef127ecc0c9-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e76c7b20-be2f-4ed2-8dc4-bef127ecc0c9-model-cache\") pod \"e76c7b20-be2f-4ed2-8dc4-bef127ecc0c9\" (UID: \"e76c7b20-be2f-4ed2-8dc4-bef127ecc0c9\") "
Oct 30 19:38:19.832158 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:19.832089    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/51927877-4d99-4797-82f7-1684f6ecda6b-tmp-dir" pod "51927877-4d99-4797-82f7-1684f6ecda6b" (UID: "51927877-4d99-4797-82f7-1684f6ecda6b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:38:19.832158 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:19.832136    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/51927877-4d99-4797-82f7-1684f6ecda6b-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/51927877-4d99-4797-82f7-1684f6ecda6b-model-cache\") pod \"51927877-4d99-4797-82f7-1684f6ecda6b\" (UID: \"51927877-4d99-4797-82f7-1684f6ecda6b\") "
Oct 30 19:38:19.832517 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:19.832211    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e76c7b20-be2f-4ed2-8dc4-bef127ecc0c9-dshm\" (UniqueName: \"kubernetes.io/empty-dir/e76c7b20-be2f-4ed2-8dc4-bef127ecc0c9-dshm\") pod \"e76c7b20-be2f-4ed2-8dc4-bef127ecc0c9\" (UID: \"e76c7b20-be2f-4ed2-8dc4-bef127ecc0c9\") "
Oct 30 19:38:19.832517 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:19.832240    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/51927877-4d99-4797-82f7-1684f6ecda6b-dshm\" (UniqueName: \"kubernetes.io/empty-dir/51927877-4d99-4797-82f7-1684f6ecda6b-dshm\") pod \"51927877-4d99-4797-82f7-1684f6ecda6b\" (UID: \"51927877-4d99-4797-82f7-1684f6ecda6b\") "
Oct 30 19:38:19.832517 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:19.832260    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e76c7b20-be2f-4ed2-8dc4-bef127ecc0c9-home" pod "e76c7b20-be2f-4ed2-8dc4-bef127ecc0c9" (UID: "e76c7b20-be2f-4ed2-8dc4-bef127ecc0c9"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:38:19.832517 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:19.832411    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/51927877-4d99-4797-82f7-1684f6ecda6b-home" pod "51927877-4d99-4797-82f7-1684f6ecda6b" (UID: "51927877-4d99-4797-82f7-1684f6ecda6b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:38:19.832517 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:19.832449    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e76c7b20-be2f-4ed2-8dc4-bef127ecc0c9-model-cache" pod "e76c7b20-be2f-4ed2-8dc4-bef127ecc0c9" (UID: "e76c7b20-be2f-4ed2-8dc4-bef127ecc0c9"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:38:19.832517 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:19.832488    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/51927877-4d99-4797-82f7-1684f6ecda6b-model-cache" pod "51927877-4d99-4797-82f7-1684f6ecda6b" (UID: "51927877-4d99-4797-82f7-1684f6ecda6b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:38:19.833049 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:19.833008    4888 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/51927877-4d99-4797-82f7-1684f6ecda6b-tmp-dir\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:38:19.833381 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:19.833053    4888 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e76c7b20-be2f-4ed2-8dc4-bef127ecc0c9-tmp-dir\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:38:19.833381 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:19.833067    4888 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e76c7b20-be2f-4ed2-8dc4-bef127ecc0c9-home\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:38:19.833381 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:19.833079    4888 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/51927877-4d99-4797-82f7-1684f6ecda6b-home\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:38:19.833381 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:19.833092    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jqrp2\" (UniqueName: \"kubernetes.io/projected/56a9a3f1-852b-458f-979f-1b815f8175d7-kube-api-access-jqrp2\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:38:19.833381 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:19.833106    4888 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e76c7b20-be2f-4ed2-8dc4-bef127ecc0c9-model-cache\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:38:19.833381 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:19.833121    4888 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/51927877-4d99-4797-82f7-1684f6ecda6b-model-cache\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:38:19.833381 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:19.833137    4888 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/56a9a3f1-852b-458f-979f-1b815f8175d7-plugins-config-volume\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:38:19.834974 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:19.834941    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e76c7b20-be2f-4ed2-8dc4-bef127ecc0c9-kube-api-access-h22l6" pod "e76c7b20-be2f-4ed2-8dc4-bef127ecc0c9" (UID: "e76c7b20-be2f-4ed2-8dc4-bef127ecc0c9"). InnerVolumeSpecName "kube-api-access-h22l6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:38:19.835161 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:19.835130    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e76c7b20-be2f-4ed2-8dc4-bef127ecc0c9-dshm" pod "e76c7b20-be2f-4ed2-8dc4-bef127ecc0c9" (UID: "e76c7b20-be2f-4ed2-8dc4-bef127ecc0c9"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:38:19.835426 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:19.835390    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/51927877-4d99-4797-82f7-1684f6ecda6b-dshm" pod "51927877-4d99-4797-82f7-1684f6ecda6b" (UID: "51927877-4d99-4797-82f7-1684f6ecda6b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:38:19.835426 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:19.835399    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/51927877-4d99-4797-82f7-1684f6ecda6b-kube-api-access-6nd5j" pod "51927877-4d99-4797-82f7-1684f6ecda6b" (UID: "51927877-4d99-4797-82f7-1684f6ecda6b"). InnerVolumeSpecName "kube-api-access-6nd5j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:38:19.835880 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:19.835856    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e76c7b20-be2f-4ed2-8dc4-bef127ecc0c9-tls-certs" pod "e76c7b20-be2f-4ed2-8dc4-bef127ecc0c9" (UID: "e76c7b20-be2f-4ed2-8dc4-bef127ecc0c9"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:38:19.836022 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:19.835998    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/51927877-4d99-4797-82f7-1684f6ecda6b-tls-certs" pod "51927877-4d99-4797-82f7-1684f6ecda6b" (UID: "51927877-4d99-4797-82f7-1684f6ecda6b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:38:19.933740 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:19.933709    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a9309776-20d5-4c06-bcae-04b8e54493e8-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a9309776-20d5-4c06-bcae-04b8e54493e8-model-cache\") pod \"a9309776-20d5-4c06-bcae-04b8e54493e8\" (UID: \"a9309776-20d5-4c06-bcae-04b8e54493e8\") "
Oct 30 19:38:19.933920 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:19.933788    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a9309776-20d5-4c06-bcae-04b8e54493e8-tls-certs\" (UniqueName: \"kubernetes.io/secret/a9309776-20d5-4c06-bcae-04b8e54493e8-tls-certs\") pod \"a9309776-20d5-4c06-bcae-04b8e54493e8\" (UID: \"a9309776-20d5-4c06-bcae-04b8e54493e8\") "
Oct 30 19:38:19.933990 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:19.933931    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a9309776-20d5-4c06-bcae-04b8e54493e8-kube-api-access-l4dd9\" (UniqueName: \"kubernetes.io/projected/a9309776-20d5-4c06-bcae-04b8e54493e8-kube-api-access-l4dd9\") pod \"a9309776-20d5-4c06-bcae-04b8e54493e8\" (UID: \"a9309776-20d5-4c06-bcae-04b8e54493e8\") "
Oct 30 19:38:19.934046 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:19.934005    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a9309776-20d5-4c06-bcae-04b8e54493e8-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a9309776-20d5-4c06-bcae-04b8e54493e8-tmp-dir\") pod \"a9309776-20d5-4c06-bcae-04b8e54493e8\" (UID: \"a9309776-20d5-4c06-bcae-04b8e54493e8\") "
Oct 30 19:38:19.934112 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:19.934057    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a9309776-20d5-4c06-bcae-04b8e54493e8-dshm\" (UniqueName: \"kubernetes.io/empty-dir/a9309776-20d5-4c06-bcae-04b8e54493e8-dshm\") pod \"a9309776-20d5-4c06-bcae-04b8e54493e8\" (UID: \"a9309776-20d5-4c06-bcae-04b8e54493e8\") "
Oct 30 19:38:19.934170 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:19.933999    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a9309776-20d5-4c06-bcae-04b8e54493e8-model-cache" pod "a9309776-20d5-4c06-bcae-04b8e54493e8" (UID: "a9309776-20d5-4c06-bcae-04b8e54493e8"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:38:19.934228 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:19.934192    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a9309776-20d5-4c06-bcae-04b8e54493e8-home\" (UniqueName: \"kubernetes.io/empty-dir/a9309776-20d5-4c06-bcae-04b8e54493e8-home\") pod \"a9309776-20d5-4c06-bcae-04b8e54493e8\" (UID: \"a9309776-20d5-4c06-bcae-04b8e54493e8\") "
Oct 30 19:38:19.934306 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:19.934237    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a9309776-20d5-4c06-bcae-04b8e54493e8-tmp-dir" pod "a9309776-20d5-4c06-bcae-04b8e54493e8" (UID: "a9309776-20d5-4c06-bcae-04b8e54493e8"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:38:19.934706 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:19.934670    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6nd5j\" (UniqueName: \"kubernetes.io/projected/51927877-4d99-4797-82f7-1684f6ecda6b-kube-api-access-6nd5j\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:38:19.934706 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:19.934699    4888 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a9309776-20d5-4c06-bcae-04b8e54493e8-model-cache\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:38:19.935069 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:19.934715    4888 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/51927877-4d99-4797-82f7-1684f6ecda6b-tls-certs\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:38:19.935069 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:19.934730    4888 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e76c7b20-be2f-4ed2-8dc4-bef127ecc0c9-tls-certs\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:38:19.935069 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:19.934743    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h22l6\" (UniqueName: \"kubernetes.io/projected/e76c7b20-be2f-4ed2-8dc4-bef127ecc0c9-kube-api-access-h22l6\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:38:19.935069 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:19.934758    4888 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a9309776-20d5-4c06-bcae-04b8e54493e8-tmp-dir\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:38:19.935069 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:19.934773    4888 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e76c7b20-be2f-4ed2-8dc4-bef127ecc0c9-dshm\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:38:19.935069 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:19.934786    4888 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/51927877-4d99-4797-82f7-1684f6ecda6b-dshm\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:38:19.935069 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:19.934614    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a9309776-20d5-4c06-bcae-04b8e54493e8-home" pod "a9309776-20d5-4c06-bcae-04b8e54493e8" (UID: "a9309776-20d5-4c06-bcae-04b8e54493e8"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:38:19.936741 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:19.936715    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a9309776-20d5-4c06-bcae-04b8e54493e8-dshm" pod "a9309776-20d5-4c06-bcae-04b8e54493e8" (UID: "a9309776-20d5-4c06-bcae-04b8e54493e8"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:38:19.936928 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:19.936902    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a9309776-20d5-4c06-bcae-04b8e54493e8-tls-certs" pod "a9309776-20d5-4c06-bcae-04b8e54493e8" (UID: "a9309776-20d5-4c06-bcae-04b8e54493e8"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:38:19.937026 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:19.936957    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a9309776-20d5-4c06-bcae-04b8e54493e8-kube-api-access-l4dd9" pod "a9309776-20d5-4c06-bcae-04b8e54493e8" (UID: "a9309776-20d5-4c06-bcae-04b8e54493e8"). InnerVolumeSpecName "kube-api-access-l4dd9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:38:20.036404 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:20.036371    4888 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a9309776-20d5-4c06-bcae-04b8e54493e8-tls-certs\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:38:20.036404 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:20.036396    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l4dd9\" (UniqueName: \"kubernetes.io/projected/a9309776-20d5-4c06-bcae-04b8e54493e8-kube-api-access-l4dd9\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:38:20.036404 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:20.036407    4888 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a9309776-20d5-4c06-bcae-04b8e54493e8-dshm\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:38:20.036404 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:20.036416    4888 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a9309776-20d5-4c06-bcae-04b8e54493e8-home\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:38:20.213841 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:20.213747    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-3b6013-57fbf8ccf5-bz979" podUID="42a5ae09-5a33-4744-9e64-8e0f0084cd97" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:38:20.214009 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:20.213747    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w3-3b6013-84b5f84d8b-hfxgw" podUID="337a7530-cf88-4466-818c-d025be569509" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:38:20.406684 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:20.406650    4888 generic.go:356] "Generic (PLEG): container finished" podID="a9309776-20d5-4c06-bcae-04b8e54493e8" containerID="b8f7a11879729dfdaeb18f36e51d7057e97444b00a31e47fbb3c2e005ae8f364" exitCode=0
Oct 30 19:38:20.407150 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:20.406700    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-3b6013/premium-w3-3b6013-kserve-74df5fb95d-k747z"
Oct 30 19:38:20.407150 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:20.406735    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-3b6013/premium-w3-3b6013-kserve-74df5fb95d-k747z" event={"ID":"a9309776-20d5-4c06-bcae-04b8e54493e8","Type":"ContainerDied","Data":"b8f7a11879729dfdaeb18f36e51d7057e97444b00a31e47fbb3c2e005ae8f364"}
Oct 30 19:38:20.407150 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:20.406779    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-3b6013/premium-w3-3b6013-kserve-74df5fb95d-k747z" event={"ID":"a9309776-20d5-4c06-bcae-04b8e54493e8","Type":"ContainerDied","Data":"0b6cfeb036c0dac3dae47741a8d8eb9840cf17d8f68c2b8ec66069d49d694c6c"}
Oct 30 19:38:20.407150 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:20.406795    4888 scope.go:122] "RemoveContainer" containerID="b8f7a11879729dfdaeb18f36e51d7057e97444b00a31e47fbb3c2e005ae8f364"
Oct 30 19:38:20.408171 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:20.408150    4888 generic.go:356] "Generic (PLEG): container finished" podID="56a9a3f1-852b-458f-979f-1b815f8175d7" containerID="1573430f9974d1b1c497479a9bbef5c737901a3ea9c6bb4b21dfc3cfd71c2250" exitCode=0
Oct 30 19:38:20.408247 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:20.408197    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-185e8b-98bb8ccf5-lcntb"
Oct 30 19:38:20.408328 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:20.408237    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-185e8b-98bb8ccf5-lcntb" event={"ID":"56a9a3f1-852b-458f-979f-1b815f8175d7","Type":"ContainerDied","Data":"1573430f9974d1b1c497479a9bbef5c737901a3ea9c6bb4b21dfc3cfd71c2250"}
Oct 30 19:38:20.408328 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:20.408302    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-185e8b-98bb8ccf5-lcntb" event={"ID":"56a9a3f1-852b-458f-979f-1b815f8175d7","Type":"ContainerDied","Data":"80603a992eb834c3774ab316c4fdd9c82058e335089f604aca3fc9c256260278"}
Oct 30 19:38:20.409926 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:20.409759    4888 generic.go:356] "Generic (PLEG): container finished" podID="51927877-4d99-4797-82f7-1684f6ecda6b" containerID="03797d706f04895778b42aca80253000864743c3106c238d298f3f547c4b2755" exitCode=0
Oct 30 19:38:20.409926 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:20.409793    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-3b6013/distinct-w3-3b6013-kserve-857f55d885-8fcrq"
Oct 30 19:38:20.409926 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:20.409800    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-3b6013/distinct-w3-3b6013-kserve-857f55d885-8fcrq" event={"ID":"51927877-4d99-4797-82f7-1684f6ecda6b","Type":"ContainerDied","Data":"03797d706f04895778b42aca80253000864743c3106c238d298f3f547c4b2755"}
Oct 30 19:38:20.409926 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:20.409835    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-3b6013/distinct-w3-3b6013-kserve-857f55d885-8fcrq" event={"ID":"51927877-4d99-4797-82f7-1684f6ecda6b","Type":"ContainerDied","Data":"070a4da7eb6bd941325e341b3d2f5c0fef8a171d095f7c8a649c10d6fcf96c59"}
Oct 30 19:38:20.411865 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:20.411842    4888 generic.go:356] "Generic (PLEG): container finished" podID="e76c7b20-be2f-4ed2-8dc4-bef127ecc0c9" containerID="539d053fe28c0a83e4a27cf2c3b304bc6df03c142a6f0cfbf7d6be44c8d708d4" exitCode=0
Oct 30 19:38:20.411975 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:20.411910    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-3b6013/distinct-2-w3-3b6013-kserve-7cc5454db-mqknj"
Oct 30 19:38:20.412078 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:20.412050    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-3b6013/distinct-2-w3-3b6013-kserve-7cc5454db-mqknj" event={"ID":"e76c7b20-be2f-4ed2-8dc4-bef127ecc0c9","Type":"ContainerDied","Data":"539d053fe28c0a83e4a27cf2c3b304bc6df03c142a6f0cfbf7d6be44c8d708d4"}
Oct 30 19:38:20.412149 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:20.412087    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-3b6013/distinct-2-w3-3b6013-kserve-7cc5454db-mqknj" event={"ID":"e76c7b20-be2f-4ed2-8dc4-bef127ecc0c9","Type":"ContainerDied","Data":"53f445b7941fb221dfa77916e51c718be86985aff233d4c90f5fe6b43a29a282"}
Oct 30 19:38:20.419748 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:20.419733    4888 scope.go:122] "RemoveContainer" containerID="b8f7a11879729dfdaeb18f36e51d7057e97444b00a31e47fbb3c2e005ae8f364"
Oct 30 19:38:20.419997 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:38:20.419976    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b8f7a11879729dfdaeb18f36e51d7057e97444b00a31e47fbb3c2e005ae8f364\": container with ID starting with b8f7a11879729dfdaeb18f36e51d7057e97444b00a31e47fbb3c2e005ae8f364 not found: ID does not exist" containerID="b8f7a11879729dfdaeb18f36e51d7057e97444b00a31e47fbb3c2e005ae8f364"
Oct 30 19:38:20.420052 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:20.420003    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b8f7a11879729dfdaeb18f36e51d7057e97444b00a31e47fbb3c2e005ae8f364"} err="failed to get container status \"b8f7a11879729dfdaeb18f36e51d7057e97444b00a31e47fbb3c2e005ae8f364\": rpc error: code = NotFound desc = could not find container \"b8f7a11879729dfdaeb18f36e51d7057e97444b00a31e47fbb3c2e005ae8f364\": container with ID starting with b8f7a11879729dfdaeb18f36e51d7057e97444b00a31e47fbb3c2e005ae8f364 not found: ID does not exist"
Oct 30 19:38:20.420052 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:20.420020    4888 scope.go:122] "RemoveContainer" containerID="1573430f9974d1b1c497479a9bbef5c737901a3ea9c6bb4b21dfc3cfd71c2250"
Oct 30 19:38:20.429647 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:20.429629    4888 scope.go:122] "RemoveContainer" containerID="1573430f9974d1b1c497479a9bbef5c737901a3ea9c6bb4b21dfc3cfd71c2250"
Oct 30 19:38:20.429901 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:38:20.429882    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1573430f9974d1b1c497479a9bbef5c737901a3ea9c6bb4b21dfc3cfd71c2250\": container with ID starting with 1573430f9974d1b1c497479a9bbef5c737901a3ea9c6bb4b21dfc3cfd71c2250 not found: ID does not exist" containerID="1573430f9974d1b1c497479a9bbef5c737901a3ea9c6bb4b21dfc3cfd71c2250"
Oct 30 19:38:20.429981 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:20.429911    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1573430f9974d1b1c497479a9bbef5c737901a3ea9c6bb4b21dfc3cfd71c2250"} err="failed to get container status \"1573430f9974d1b1c497479a9bbef5c737901a3ea9c6bb4b21dfc3cfd71c2250\": rpc error: code = NotFound desc = could not find container \"1573430f9974d1b1c497479a9bbef5c737901a3ea9c6bb4b21dfc3cfd71c2250\": container with ID starting with 1573430f9974d1b1c497479a9bbef5c737901a3ea9c6bb4b21dfc3cfd71c2250 not found: ID does not exist"
Oct 30 19:38:20.429981 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:20.429931    4888 scope.go:122] "RemoveContainer" containerID="03797d706f04895778b42aca80253000864743c3106c238d298f3f547c4b2755"
Oct 30 19:38:20.438750 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:20.438734    4888 scope.go:122] "RemoveContainer" containerID="03797d706f04895778b42aca80253000864743c3106c238d298f3f547c4b2755"
Oct 30 19:38:20.439009 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:38:20.438977    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"03797d706f04895778b42aca80253000864743c3106c238d298f3f547c4b2755\": container with ID starting with 03797d706f04895778b42aca80253000864743c3106c238d298f3f547c4b2755 not found: ID does not exist" containerID="03797d706f04895778b42aca80253000864743c3106c238d298f3f547c4b2755"
Oct 30 19:38:20.439057 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:20.439011    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"03797d706f04895778b42aca80253000864743c3106c238d298f3f547c4b2755"} err="failed to get container status \"03797d706f04895778b42aca80253000864743c3106c238d298f3f547c4b2755\": rpc error: code = NotFound desc = could not find container \"03797d706f04895778b42aca80253000864743c3106c238d298f3f547c4b2755\": container with ID starting with 03797d706f04895778b42aca80253000864743c3106c238d298f3f547c4b2755 not found: ID does not exist"
Oct 30 19:38:20.439057 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:20.439028    4888 scope.go:122] "RemoveContainer" containerID="539d053fe28c0a83e4a27cf2c3b304bc6df03c142a6f0cfbf7d6be44c8d708d4"
Oct 30 19:38:20.447502 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:20.447487    4888 scope.go:122] "RemoveContainer" containerID="539d053fe28c0a83e4a27cf2c3b304bc6df03c142a6f0cfbf7d6be44c8d708d4"
Oct 30 19:38:20.447761 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:38:20.447743    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"539d053fe28c0a83e4a27cf2c3b304bc6df03c142a6f0cfbf7d6be44c8d708d4\": container with ID starting with 539d053fe28c0a83e4a27cf2c3b304bc6df03c142a6f0cfbf7d6be44c8d708d4 not found: ID does not exist" containerID="539d053fe28c0a83e4a27cf2c3b304bc6df03c142a6f0cfbf7d6be44c8d708d4"
Oct 30 19:38:20.447826 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:20.447766    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"539d053fe28c0a83e4a27cf2c3b304bc6df03c142a6f0cfbf7d6be44c8d708d4"} err="failed to get container status \"539d053fe28c0a83e4a27cf2c3b304bc6df03c142a6f0cfbf7d6be44c8d708d4\": rpc error: code = NotFound desc = could not find container \"539d053fe28c0a83e4a27cf2c3b304bc6df03c142a6f0cfbf7d6be44c8d708d4\": container with ID starting with 539d053fe28c0a83e4a27cf2c3b304bc6df03c142a6f0cfbf7d6be44c8d708d4 not found: ID does not exist"
Oct 30 19:38:20.577608 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:20.577582    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-3b6013/premium-w3-3b6013-kserve-74df5fb95d-k747z"]
Oct 30 19:38:20.582615 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:20.582590    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w3-3b6013/premium-w3-3b6013-kserve-74df5fb95d-k747z"]
Oct 30 19:38:20.679993 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:20.679960    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-3b6013/distinct-2-w3-3b6013-kserve-7cc5454db-mqknj" podUID="e76c7b20-be2f-4ed2-8dc4-bef127ecc0c9" containerName="main" probeResult="failure" output="Get \"https://10.132.0.98:8000/ready\": context deadline exceeded"
Oct 30 19:38:20.697348 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:20.697324    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-185e8b-98bb8ccf5-lcntb"]
Oct 30 19:38:20.700367 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:20.700345    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-185e8b-98bb8ccf5-lcntb"]
Oct 30 19:38:20.812543 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:20.812512    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-3b6013/distinct-2-w3-3b6013-kserve-7cc5454db-mqknj"]
Oct 30 19:38:20.816664 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:20.816642    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w3-3b6013/distinct-2-w3-3b6013-kserve-7cc5454db-mqknj"]
Oct 30 19:38:20.927708 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:20.927637    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-3b6013/distinct-w3-3b6013-kserve-857f55d885-8fcrq"]
Oct 30 19:38:20.932384 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:20.932361    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w3-3b6013/distinct-w3-3b6013-kserve-857f55d885-8fcrq"]
Oct 30 19:38:21.891035 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:21.891000    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="51927877-4d99-4797-82f7-1684f6ecda6b" path="/var/lib/kubelet/pods/51927877-4d99-4797-82f7-1684f6ecda6b/volumes"
Oct 30 19:38:21.891438 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:21.891418    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="56a9a3f1-852b-458f-979f-1b815f8175d7" path="/var/lib/kubelet/pods/56a9a3f1-852b-458f-979f-1b815f8175d7/volumes"
Oct 30 19:38:21.891806 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:21.891792    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a9309776-20d5-4c06-bcae-04b8e54493e8" path="/var/lib/kubelet/pods/a9309776-20d5-4c06-bcae-04b8e54493e8/volumes"
Oct 30 19:38:21.892492 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:21.892478    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e76c7b20-be2f-4ed2-8dc4-bef127ecc0c9" path="/var/lib/kubelet/pods/e76c7b20-be2f-4ed2-8dc4-bef127ecc0c9/volumes"
Oct 30 19:38:30.214482 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:30.214434    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w3-3b6013-84b5f84d8b-hfxgw" podUID="337a7530-cf88-4466-818c-d025be569509" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:38:30.214976 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:30.214564    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-3b6013-57fbf8ccf5-bz979" podUID="42a5ae09-5a33-4744-9e64-8e0f0084cd97" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:38:35.620487 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:38:35.620447    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-jbsw6" containerName="extractor"
Oct 30 19:38:36.860031 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:36.860006    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w3-3b6013-84b5f84d8b-hfxgw"
Oct 30 19:38:36.978690 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:36.978661    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-3b6013-57fbf8ccf5-bz979"
Oct 30 19:38:37.032891 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:37.032818    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/337a7530-cf88-4466-818c-d025be569509-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/337a7530-cf88-4466-818c-d025be569509-plugins-config-volume\") pod \"337a7530-cf88-4466-818c-d025be569509\" (UID: \"337a7530-cf88-4466-818c-d025be569509\") "
Oct 30 19:38:37.033077 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:37.033050    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/337a7530-cf88-4466-818c-d025be569509-kube-api-access-24xql\" (UniqueName: \"kubernetes.io/projected/337a7530-cf88-4466-818c-d025be569509-kube-api-access-24xql\") pod \"337a7530-cf88-4466-818c-d025be569509\" (UID: \"337a7530-cf88-4466-818c-d025be569509\") "
Oct 30 19:38:37.033221 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:37.033199    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/337a7530-cf88-4466-818c-d025be569509-plugins-config-volume" pod "337a7530-cf88-4466-818c-d025be569509" (UID: "337a7530-cf88-4466-818c-d025be569509"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:38:37.033739 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:37.033709    4888 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/337a7530-cf88-4466-818c-d025be569509-plugins-config-volume\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:38:37.035653 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:37.035628    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/337a7530-cf88-4466-818c-d025be569509-kube-api-access-24xql" pod "337a7530-cf88-4466-818c-d025be569509" (UID: "337a7530-cf88-4466-818c-d025be569509"). InnerVolumeSpecName "kube-api-access-24xql". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:38:37.134701 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:37.134628    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/42a5ae09-5a33-4744-9e64-8e0f0084cd97-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/42a5ae09-5a33-4744-9e64-8e0f0084cd97-plugins-config-volume\") pod \"42a5ae09-5a33-4744-9e64-8e0f0084cd97\" (UID: \"42a5ae09-5a33-4744-9e64-8e0f0084cd97\") "
Oct 30 19:38:37.134842 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:37.134705    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/42a5ae09-5a33-4744-9e64-8e0f0084cd97-kube-api-access-t5z72\" (UniqueName: \"kubernetes.io/projected/42a5ae09-5a33-4744-9e64-8e0f0084cd97-kube-api-access-t5z72\") pod \"42a5ae09-5a33-4744-9e64-8e0f0084cd97\" (UID: \"42a5ae09-5a33-4744-9e64-8e0f0084cd97\") "
Oct 30 19:38:37.134971 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:37.134956    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-24xql\" (UniqueName: \"kubernetes.io/projected/337a7530-cf88-4466-818c-d025be569509-kube-api-access-24xql\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:38:37.135114 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:37.135088    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/42a5ae09-5a33-4744-9e64-8e0f0084cd97-plugins-config-volume" pod "42a5ae09-5a33-4744-9e64-8e0f0084cd97" (UID: "42a5ae09-5a33-4744-9e64-8e0f0084cd97"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:38:37.137023 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:37.136996    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/42a5ae09-5a33-4744-9e64-8e0f0084cd97-kube-api-access-t5z72" pod "42a5ae09-5a33-4744-9e64-8e0f0084cd97" (UID: "42a5ae09-5a33-4744-9e64-8e0f0084cd97"). InnerVolumeSpecName "kube-api-access-t5z72". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:38:37.235729 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:37.235700    4888 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/42a5ae09-5a33-4744-9e64-8e0f0084cd97-plugins-config-volume\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:38:37.235729 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:37.235728    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t5z72\" (UniqueName: \"kubernetes.io/projected/42a5ae09-5a33-4744-9e64-8e0f0084cd97-kube-api-access-t5z72\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:38:37.542718 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:37.542637    4888 generic.go:356] "Generic (PLEG): container finished" podID="337a7530-cf88-4466-818c-d025be569509" containerID="77c4fc34120888101629aa521c214162d2dd7365dc335fdc65ac5bbf29b2708b" exitCode=0
Oct 30 19:38:37.542718 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:37.542681    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w3-3b6013-84b5f84d8b-hfxgw"
Oct 30 19:38:37.542942 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:37.542723    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w3-3b6013-84b5f84d8b-hfxgw" event={"ID":"337a7530-cf88-4466-818c-d025be569509","Type":"ContainerDied","Data":"77c4fc34120888101629aa521c214162d2dd7365dc335fdc65ac5bbf29b2708b"}
Oct 30 19:38:37.542942 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:37.542780    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w3-3b6013-84b5f84d8b-hfxgw" event={"ID":"337a7530-cf88-4466-818c-d025be569509","Type":"ContainerDied","Data":"dae5121e63ac3736c431c53778eefbe3b6e72299987a75ff013a6bb1409222e1"}
Oct 30 19:38:37.542942 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:37.542809    4888 scope.go:122] "RemoveContainer" containerID="77c4fc34120888101629aa521c214162d2dd7365dc335fdc65ac5bbf29b2708b"
Oct 30 19:38:37.544450 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:37.544416    4888 generic.go:356] "Generic (PLEG): container finished" podID="42a5ae09-5a33-4744-9e64-8e0f0084cd97" containerID="db05e15f0e6b4bcf13a007cc06289b5ea825e1febc79948819c7f11b0d1a7e79" exitCode=0
Oct 30 19:38:37.544565 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:37.544452    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-3b6013-57fbf8ccf5-bz979"
Oct 30 19:38:37.544565 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:37.544499    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-3b6013-57fbf8ccf5-bz979" event={"ID":"42a5ae09-5a33-4744-9e64-8e0f0084cd97","Type":"ContainerDied","Data":"db05e15f0e6b4bcf13a007cc06289b5ea825e1febc79948819c7f11b0d1a7e79"}
Oct 30 19:38:37.544565 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:37.544552    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-3b6013-57fbf8ccf5-bz979" event={"ID":"42a5ae09-5a33-4744-9e64-8e0f0084cd97","Type":"ContainerDied","Data":"44beab25f0159f8a6f392da563e7c5ac672e8b6b8493e12c1226b59805e34703"}
Oct 30 19:38:37.554296 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:37.554259    4888 scope.go:122] "RemoveContainer" containerID="77c4fc34120888101629aa521c214162d2dd7365dc335fdc65ac5bbf29b2708b"
Oct 30 19:38:37.554615 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:38:37.554589    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"77c4fc34120888101629aa521c214162d2dd7365dc335fdc65ac5bbf29b2708b\": container with ID starting with 77c4fc34120888101629aa521c214162d2dd7365dc335fdc65ac5bbf29b2708b not found: ID does not exist" containerID="77c4fc34120888101629aa521c214162d2dd7365dc335fdc65ac5bbf29b2708b"
Oct 30 19:38:37.554698 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:37.554621    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"77c4fc34120888101629aa521c214162d2dd7365dc335fdc65ac5bbf29b2708b"} err="failed to get container status \"77c4fc34120888101629aa521c214162d2dd7365dc335fdc65ac5bbf29b2708b\": rpc error: code = NotFound desc = could not find container \"77c4fc34120888101629aa521c214162d2dd7365dc335fdc65ac5bbf29b2708b\": container with ID starting with 77c4fc34120888101629aa521c214162d2dd7365dc335fdc65ac5bbf29b2708b not found: ID does not exist"
Oct 30 19:38:37.554698 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:37.554642    4888 scope.go:122] "RemoveContainer" containerID="db05e15f0e6b4bcf13a007cc06289b5ea825e1febc79948819c7f11b0d1a7e79"
Oct 30 19:38:37.564337 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:37.564316    4888 scope.go:122] "RemoveContainer" containerID="db05e15f0e6b4bcf13a007cc06289b5ea825e1febc79948819c7f11b0d1a7e79"
Oct 30 19:38:37.564670 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:38:37.564653    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"db05e15f0e6b4bcf13a007cc06289b5ea825e1febc79948819c7f11b0d1a7e79\": container with ID starting with db05e15f0e6b4bcf13a007cc06289b5ea825e1febc79948819c7f11b0d1a7e79 not found: ID does not exist" containerID="db05e15f0e6b4bcf13a007cc06289b5ea825e1febc79948819c7f11b0d1a7e79"
Oct 30 19:38:37.564732 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:37.564677    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"db05e15f0e6b4bcf13a007cc06289b5ea825e1febc79948819c7f11b0d1a7e79"} err="failed to get container status \"db05e15f0e6b4bcf13a007cc06289b5ea825e1febc79948819c7f11b0d1a7e79\": rpc error: code = NotFound desc = could not find container \"db05e15f0e6b4bcf13a007cc06289b5ea825e1febc79948819c7f11b0d1a7e79\": container with ID starting with db05e15f0e6b4bcf13a007cc06289b5ea825e1febc79948819c7f11b0d1a7e79 not found: ID does not exist"
Oct 30 19:38:37.719221 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:37.719188    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-3b6013-57fbf8ccf5-bz979"]
Oct 30 19:38:37.769333 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:37.769301    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-3b6013-57fbf8ccf5-bz979"]
Oct 30 19:38:37.851722 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:37.851685    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-3b6013-84b5f84d8b-hfxgw"]
Oct 30 19:38:37.857395 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:37.857366    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-3b6013-84b5f84d8b-hfxgw"]
Oct 30 19:38:37.891053 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:37.891018    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="337a7530-cf88-4466-818c-d025be569509" path="/var/lib/kubelet/pods/337a7530-cf88-4466-818c-d025be569509/volumes"
Oct 30 19:38:37.891717 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:37.891698    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="42a5ae09-5a33-4744-9e64-8e0f0084cd97" path="/var/lib/kubelet/pods/42a5ae09-5a33-4744-9e64-8e0f0084cd97/volumes"
Oct 30 19:38:52.667792 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:52.667755    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-9e4e63-fbff868b4-hx2k4"]
Oct 30 19:38:52.785386 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:52.785355    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-9e4e63-6bb4cfdc5-5mxz7"]
Oct 30 19:38:55.223068 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:55.223037    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-rjmdz"]
Oct 30 19:38:56.860392 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:56.860369    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-7x4rb_382e8b31-a7fa-43f3-9f15-4c61627b0c04/console-operator/2.log"
Oct 30 19:38:56.867766 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:56.867742    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xfcdl_3f3fe764-2652-46c9-8e54-7882c0e6bcf1/ovn-acl-logging/0.log"
Oct 30 19:38:56.871510 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:56.871493    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-7x4rb_382e8b31-a7fa-43f3-9f15-4c61627b0c04/console-operator/2.log"
Oct 30 19:38:56.878210 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:56.878193    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xfcdl_3f3fe764-2652-46c9-8e54-7882c0e6bcf1/ovn-acl-logging/0.log"
Oct 30 19:38:57.668384 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:57.668340    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-extmod-a-9e4e63-fbff868b4-hx2k4" podUID="0b08bd02-8702-44a6-819c-c7f9d044a826" containerName="payload-processing" containerID="cri-o://c543c4eaacb0d508d6be0d746f4ceba53c12292c2b6a6644899b229ffe2d2706" gracePeriod=25
Oct 30 19:38:57.785860 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:57.785762    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-9e4e63-6bb4cfdc5-5mxz7" podUID="63b61d8b-43d3-439f-bc73-639e205a90d1" containerName="payload-pre-processing" containerID="cri-o://a6706f7b4e48f4af4ede9d48f5649040ac4f75defcf15530a018f83054e9b66f" gracePeriod=25
Oct 30 19:38:57.955378 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:57.955339    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-a-9e4e63-fbff868b4-hx2k4"
Oct 30 19:38:58.050247 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:58.050224    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-9e4e63-6bb4cfdc5-5mxz7"
Oct 30 19:38:58.090530 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:58.090501    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0b08bd02-8702-44a6-819c-c7f9d044a826-kube-api-access-cppjd\" (UniqueName: \"kubernetes.io/projected/0b08bd02-8702-44a6-819c-c7f9d044a826-kube-api-access-cppjd\") pod \"0b08bd02-8702-44a6-819c-c7f9d044a826\" (UID: \"0b08bd02-8702-44a6-819c-c7f9d044a826\") "
Oct 30 19:38:58.090857 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:58.090579    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0b08bd02-8702-44a6-819c-c7f9d044a826-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0b08bd02-8702-44a6-819c-c7f9d044a826-plugins-config-volume\") pod \"0b08bd02-8702-44a6-819c-c7f9d044a826\" (UID: \"0b08bd02-8702-44a6-819c-c7f9d044a826\") "
Oct 30 19:38:58.091010 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:58.090986    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0b08bd02-8702-44a6-819c-c7f9d044a826-plugins-config-volume" pod "0b08bd02-8702-44a6-819c-c7f9d044a826" (UID: "0b08bd02-8702-44a6-819c-c7f9d044a826"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:38:58.091592 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:58.091570    4888 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0b08bd02-8702-44a6-819c-c7f9d044a826-plugins-config-volume\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:38:58.092735 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:58.092699    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0b08bd02-8702-44a6-819c-c7f9d044a826-kube-api-access-cppjd" pod "0b08bd02-8702-44a6-819c-c7f9d044a826" (UID: "0b08bd02-8702-44a6-819c-c7f9d044a826"). InnerVolumeSpecName "kube-api-access-cppjd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:38:58.192693 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:58.192667    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/63b61d8b-43d3-439f-bc73-639e205a90d1-kube-api-access-8d7q2\" (UniqueName: \"kubernetes.io/projected/63b61d8b-43d3-439f-bc73-639e205a90d1-kube-api-access-8d7q2\") pod \"63b61d8b-43d3-439f-bc73-639e205a90d1\" (UID: \"63b61d8b-43d3-439f-bc73-639e205a90d1\") "
Oct 30 19:38:58.192882 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:58.192713    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/63b61d8b-43d3-439f-bc73-639e205a90d1-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/63b61d8b-43d3-439f-bc73-639e205a90d1-plugins-config-volume\") pod \"63b61d8b-43d3-439f-bc73-639e205a90d1\" (UID: \"63b61d8b-43d3-439f-bc73-639e205a90d1\") "
Oct 30 19:38:58.192975 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:58.192960    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cppjd\" (UniqueName: \"kubernetes.io/projected/0b08bd02-8702-44a6-819c-c7f9d044a826-kube-api-access-cppjd\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:38:58.193162 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:58.193139    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/63b61d8b-43d3-439f-bc73-639e205a90d1-plugins-config-volume" pod "63b61d8b-43d3-439f-bc73-639e205a90d1" (UID: "63b61d8b-43d3-439f-bc73-639e205a90d1"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:38:58.194906 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:58.194880    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/63b61d8b-43d3-439f-bc73-639e205a90d1-kube-api-access-8d7q2" pod "63b61d8b-43d3-439f-bc73-639e205a90d1" (UID: "63b61d8b-43d3-439f-bc73-639e205a90d1"). InnerVolumeSpecName "kube-api-access-8d7q2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:38:58.294159 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:58.294093    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8d7q2\" (UniqueName: \"kubernetes.io/projected/63b61d8b-43d3-439f-bc73-639e205a90d1-kube-api-access-8d7q2\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:38:58.294159 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:58.294119    4888 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/63b61d8b-43d3-439f-bc73-639e205a90d1-plugins-config-volume\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:38:58.728855 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:58.728824    4888 generic.go:356] "Generic (PLEG): container finished" podID="0b08bd02-8702-44a6-819c-c7f9d044a826" containerID="c543c4eaacb0d508d6be0d746f4ceba53c12292c2b6a6644899b229ffe2d2706" exitCode=0
Oct 30 19:38:58.729023 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:58.728871    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-a-9e4e63-fbff868b4-hx2k4"
Oct 30 19:38:58.729023 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:58.728908    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-a-9e4e63-fbff868b4-hx2k4" event={"ID":"0b08bd02-8702-44a6-819c-c7f9d044a826","Type":"ContainerDied","Data":"c543c4eaacb0d508d6be0d746f4ceba53c12292c2b6a6644899b229ffe2d2706"}
Oct 30 19:38:58.729023 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:58.728962    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-a-9e4e63-fbff868b4-hx2k4" event={"ID":"0b08bd02-8702-44a6-819c-c7f9d044a826","Type":"ContainerDied","Data":"f871eeea688140f48c7580d507130d55790aa598e266a91ff65b4f958da5e574"}
Oct 30 19:38:58.729023 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:58.728988    4888 scope.go:122] "RemoveContainer" containerID="c543c4eaacb0d508d6be0d746f4ceba53c12292c2b6a6644899b229ffe2d2706"
Oct 30 19:38:58.730370 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:58.730349    4888 generic.go:356] "Generic (PLEG): container finished" podID="63b61d8b-43d3-439f-bc73-639e205a90d1" containerID="a6706f7b4e48f4af4ede9d48f5649040ac4f75defcf15530a018f83054e9b66f" exitCode=0
Oct 30 19:38:58.730464 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:58.730402    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-9e4e63-6bb4cfdc5-5mxz7"
Oct 30 19:38:58.730464 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:58.730425    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-9e4e63-6bb4cfdc5-5mxz7" event={"ID":"63b61d8b-43d3-439f-bc73-639e205a90d1","Type":"ContainerDied","Data":"a6706f7b4e48f4af4ede9d48f5649040ac4f75defcf15530a018f83054e9b66f"}
Oct 30 19:38:58.730587 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:58.730493    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-9e4e63-6bb4cfdc5-5mxz7" event={"ID":"63b61d8b-43d3-439f-bc73-639e205a90d1","Type":"ContainerDied","Data":"26e92411bf8a6d216712acd8717705bc2ff82e104def7f1a545200c4eafc4ed8"}
Oct 30 19:38:58.741185 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:58.741166    4888 scope.go:122] "RemoveContainer" containerID="c543c4eaacb0d508d6be0d746f4ceba53c12292c2b6a6644899b229ffe2d2706"
Oct 30 19:38:58.741536 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:38:58.741516    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c543c4eaacb0d508d6be0d746f4ceba53c12292c2b6a6644899b229ffe2d2706\": container with ID starting with c543c4eaacb0d508d6be0d746f4ceba53c12292c2b6a6644899b229ffe2d2706 not found: ID does not exist" containerID="c543c4eaacb0d508d6be0d746f4ceba53c12292c2b6a6644899b229ffe2d2706"
Oct 30 19:38:58.741646 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:58.741541    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c543c4eaacb0d508d6be0d746f4ceba53c12292c2b6a6644899b229ffe2d2706"} err="failed to get container status \"c543c4eaacb0d508d6be0d746f4ceba53c12292c2b6a6644899b229ffe2d2706\": rpc error: code = NotFound desc = could not find container \"c543c4eaacb0d508d6be0d746f4ceba53c12292c2b6a6644899b229ffe2d2706\": container with ID starting with c543c4eaacb0d508d6be0d746f4ceba53c12292c2b6a6644899b229ffe2d2706 not found: ID does not exist"
Oct 30 19:38:58.741646 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:58.741556    4888 scope.go:122] "RemoveContainer" containerID="a6706f7b4e48f4af4ede9d48f5649040ac4f75defcf15530a018f83054e9b66f"
Oct 30 19:38:58.751108 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:58.751093    4888 scope.go:122] "RemoveContainer" containerID="a6706f7b4e48f4af4ede9d48f5649040ac4f75defcf15530a018f83054e9b66f"
Oct 30 19:38:58.751390 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:38:58.751372    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a6706f7b4e48f4af4ede9d48f5649040ac4f75defcf15530a018f83054e9b66f\": container with ID starting with a6706f7b4e48f4af4ede9d48f5649040ac4f75defcf15530a018f83054e9b66f not found: ID does not exist" containerID="a6706f7b4e48f4af4ede9d48f5649040ac4f75defcf15530a018f83054e9b66f"
Oct 30 19:38:58.751436 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:58.751396    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a6706f7b4e48f4af4ede9d48f5649040ac4f75defcf15530a018f83054e9b66f"} err="failed to get container status \"a6706f7b4e48f4af4ede9d48f5649040ac4f75defcf15530a018f83054e9b66f\": rpc error: code = NotFound desc = could not find container \"a6706f7b4e48f4af4ede9d48f5649040ac4f75defcf15530a018f83054e9b66f\": container with ID starting with a6706f7b4e48f4af4ede9d48f5649040ac4f75defcf15530a018f83054e9b66f not found: ID does not exist"
Oct 30 19:38:58.926320 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:58.923501    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-9e4e63-6bb4cfdc5-5mxz7"]
Oct 30 19:38:58.983823 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:58.983748    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-9e4e63-6bb4cfdc5-5mxz7"]
Oct 30 19:38:59.118491 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:59.118453    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-9e4e63-fbff868b4-hx2k4"]
Oct 30 19:38:59.144165 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:59.144140    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-9e4e63-fbff868b4-hx2k4"]
Oct 30 19:38:59.892385 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:59.892353    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0b08bd02-8702-44a6-819c-c7f9d044a826" path="/var/lib/kubelet/pods/0b08bd02-8702-44a6-819c-c7f9d044a826/volumes"
Oct 30 19:38:59.892875 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:38:59.892856    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="63b61d8b-43d3-439f-bc73-639e205a90d1" path="/var/lib/kubelet/pods/63b61d8b-43d3-439f-bc73-639e205a90d1/volumes"
Oct 30 19:39:05.619121 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:39:05.619041    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-jbsw6" containerName="extractor"
Oct 30 19:39:19.260251 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:19.260219    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-extmod-a-98b389-openshift-default-57c5bc4c69-t5g6x"]
Oct 30 19:39:19.260771 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:19.260756    4888 cpu_manager.go:414] "containerMap: removing container" podUID="0b08bd02-8702-44a6-819c-c7f9d044a826" containerName="payload-processing"
Oct 30 19:39:19.260823 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:19.260772    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0b08bd02-8702-44a6-819c-c7f9d044a826" containerName="payload-processing"
Oct 30 19:39:19.260823 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:19.260796    4888 cpu_manager.go:414] "containerMap: removing container" podUID="a9309776-20d5-4c06-bcae-04b8e54493e8" containerName="main"
Oct 30 19:39:19.260823 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:19.260802    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a9309776-20d5-4c06-bcae-04b8e54493e8" containerName="main"
Oct 30 19:39:19.260823 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:19.260815    4888 cpu_manager.go:414] "containerMap: removing container" podUID="63b61d8b-43d3-439f-bc73-639e205a90d1" containerName="payload-pre-processing"
Oct 30 19:39:19.260823 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:19.260820    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="63b61d8b-43d3-439f-bc73-639e205a90d1" containerName="payload-pre-processing"
Oct 30 19:39:19.260984 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:19.260827    4888 cpu_manager.go:414] "containerMap: removing container" podUID="53be2715-fb83-4227-9b79-cc1dcd732396" containerName="maas-api"
Oct 30 19:39:19.260984 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:19.260832    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="53be2715-fb83-4227-9b79-cc1dcd732396" containerName="maas-api"
Oct 30 19:39:19.260984 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:19.260841    4888 cpu_manager.go:414] "containerMap: removing container" podUID="faa528eb-1c1e-4060-8e7b-317b1900541c" containerName="maas-api"
Oct 30 19:39:19.260984 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:19.260847    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="faa528eb-1c1e-4060-8e7b-317b1900541c" containerName="maas-api"
Oct 30 19:39:19.260984 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:19.260855    4888 cpu_manager.go:414] "containerMap: removing container" podUID="e76c7b20-be2f-4ed2-8dc4-bef127ecc0c9" containerName="main"
Oct 30 19:39:19.260984 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:19.260860    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e76c7b20-be2f-4ed2-8dc4-bef127ecc0c9" containerName="main"
Oct 30 19:39:19.260984 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:19.260871    4888 cpu_manager.go:414] "containerMap: removing container" podUID="42a5ae09-5a33-4744-9e64-8e0f0084cd97" containerName="payload-pre-processing"
Oct 30 19:39:19.260984 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:19.260877    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="42a5ae09-5a33-4744-9e64-8e0f0084cd97" containerName="payload-pre-processing"
Oct 30 19:39:19.260984 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:19.260884    4888 cpu_manager.go:414] "containerMap: removing container" podUID="6adbce7d-d243-4f53-b2f0-e1db8b942eb1" containerName="istio-proxy"
Oct 30 19:39:19.260984 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:19.260889    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6adbce7d-d243-4f53-b2f0-e1db8b942eb1" containerName="istio-proxy"
Oct 30 19:39:19.260984 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:19.260896    4888 cpu_manager.go:414] "containerMap: removing container" podUID="6adbce7d-d243-4f53-b2f0-e1db8b942eb1" containerName="istio-proxy"
Oct 30 19:39:19.260984 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:19.260900    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6adbce7d-d243-4f53-b2f0-e1db8b942eb1" containerName="istio-proxy"
Oct 30 19:39:19.260984 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:19.260911    4888 cpu_manager.go:414] "containerMap: removing container" podUID="56a9a3f1-852b-458f-979f-1b815f8175d7" containerName="payload-pre-processing"
Oct 30 19:39:19.260984 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:19.260915    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="56a9a3f1-852b-458f-979f-1b815f8175d7" containerName="payload-pre-processing"
Oct 30 19:39:19.260984 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:19.260922    4888 cpu_manager.go:414] "containerMap: removing container" podUID="3f801255-a7eb-4a5f-b80f-82c3c0a6a8a6" containerName="payload-pre-processing"
Oct 30 19:39:19.260984 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:19.260926    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3f801255-a7eb-4a5f-b80f-82c3c0a6a8a6" containerName="payload-pre-processing"
Oct 30 19:39:19.260984 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:19.260933    4888 cpu_manager.go:414] "containerMap: removing container" podUID="337a7530-cf88-4466-818c-d025be569509" containerName="payload-processing"
Oct 30 19:39:19.260984 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:19.260937    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="337a7530-cf88-4466-818c-d025be569509" containerName="payload-processing"
Oct 30 19:39:19.260984 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:19.260944    4888 cpu_manager.go:414] "containerMap: removing container" podUID="73a3358c-ef67-4b35-a010-46c168a95f1e" containerName="istio-proxy"
Oct 30 19:39:19.260984 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:19.260950    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="73a3358c-ef67-4b35-a010-46c168a95f1e" containerName="istio-proxy"
Oct 30 19:39:19.260984 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:19.260960    4888 cpu_manager.go:414] "containerMap: removing container" podUID="73a3358c-ef67-4b35-a010-46c168a95f1e" containerName="istio-proxy"
Oct 30 19:39:19.260984 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:19.260964    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="73a3358c-ef67-4b35-a010-46c168a95f1e" containerName="istio-proxy"
Oct 30 19:39:19.260984 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:19.260970    4888 cpu_manager.go:414] "containerMap: removing container" podUID="51927877-4d99-4797-82f7-1684f6ecda6b" containerName="main"
Oct 30 19:39:19.260984 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:19.260975    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="51927877-4d99-4797-82f7-1684f6ecda6b" containerName="main"
Oct 30 19:39:19.261747 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:19.261052    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="337a7530-cf88-4466-818c-d025be569509" containerName="payload-processing"
Oct 30 19:39:19.261747 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:19.261062    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="73a3358c-ef67-4b35-a010-46c168a95f1e" containerName="istio-proxy"
Oct 30 19:39:19.261747 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:19.261068    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="73a3358c-ef67-4b35-a010-46c168a95f1e" containerName="istio-proxy"
Oct 30 19:39:19.261747 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:19.261074    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="51927877-4d99-4797-82f7-1684f6ecda6b" containerName="main"
Oct 30 19:39:19.261747 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:19.261080    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="0b08bd02-8702-44a6-819c-c7f9d044a826" containerName="payload-processing"
Oct 30 19:39:19.261747 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:19.261089    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="a9309776-20d5-4c06-bcae-04b8e54493e8" containerName="main"
Oct 30 19:39:19.261747 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:19.261095    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="63b61d8b-43d3-439f-bc73-639e205a90d1" containerName="payload-pre-processing"
Oct 30 19:39:19.261747 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:19.261101    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="53be2715-fb83-4227-9b79-cc1dcd732396" containerName="maas-api"
Oct 30 19:39:19.261747 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:19.261105    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="faa528eb-1c1e-4060-8e7b-317b1900541c" containerName="maas-api"
Oct 30 19:39:19.261747 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:19.261110    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="e76c7b20-be2f-4ed2-8dc4-bef127ecc0c9" containerName="main"
Oct 30 19:39:19.261747 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:19.261116    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="42a5ae09-5a33-4744-9e64-8e0f0084cd97" containerName="payload-pre-processing"
Oct 30 19:39:19.261747 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:19.261121    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="6adbce7d-d243-4f53-b2f0-e1db8b942eb1" containerName="istio-proxy"
Oct 30 19:39:19.261747 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:19.261127    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="6adbce7d-d243-4f53-b2f0-e1db8b942eb1" containerName="istio-proxy"
Oct 30 19:39:19.261747 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:19.261132    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="56a9a3f1-852b-458f-979f-1b815f8175d7" containerName="payload-pre-processing"
Oct 30 19:39:19.261747 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:19.261139    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="3f801255-a7eb-4a5f-b80f-82c3c0a6a8a6" containerName="payload-pre-processing"
Oct 30 19:39:19.264743 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:19.264724    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-98b389-openshift-default-57c5bc4c69-t5g6x"
Oct 30 19:39:19.318447 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:19.318419    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-extmod-a-98b389-openshift-default-dockercfg-gdltq\""
Oct 30 19:39:19.357551 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:19.357516    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9f8e71e3-8cd2-4213-952e-e9db1194df27-workload-certs\") pod \"e2e-extmod-a-98b389-openshift-default-57c5bc4c69-t5g6x\" (UID: \"9f8e71e3-8cd2-4213-952e-e9db1194df27\") " pod="openshift-ingress/e2e-extmod-a-98b389-openshift-default-57c5bc4c69-t5g6x"
Oct 30 19:39:19.357704 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:19.357557    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4blg2\" (UniqueName: \"kubernetes.io/projected/9f8e71e3-8cd2-4213-952e-e9db1194df27-kube-api-access-4blg2\") pod \"e2e-extmod-a-98b389-openshift-default-57c5bc4c69-t5g6x\" (UID: \"9f8e71e3-8cd2-4213-952e-e9db1194df27\") " pod="openshift-ingress/e2e-extmod-a-98b389-openshift-default-57c5bc4c69-t5g6x"
Oct 30 19:39:19.357704 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:19.357641    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9f8e71e3-8cd2-4213-952e-e9db1194df27-istio-token\") pod \"e2e-extmod-a-98b389-openshift-default-57c5bc4c69-t5g6x\" (UID: \"9f8e71e3-8cd2-4213-952e-e9db1194df27\") " pod="openshift-ingress/e2e-extmod-a-98b389-openshift-default-57c5bc4c69-t5g6x"
Oct 30 19:39:19.357704 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:19.357679    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f8e71e3-8cd2-4213-952e-e9db1194df27-workload-socket\") pod \"e2e-extmod-a-98b389-openshift-default-57c5bc4c69-t5g6x\" (UID: \"9f8e71e3-8cd2-4213-952e-e9db1194df27\") " pod="openshift-ingress/e2e-extmod-a-98b389-openshift-default-57c5bc4c69-t5g6x"
Oct 30 19:39:19.357833 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:19.357777    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9f8e71e3-8cd2-4213-952e-e9db1194df27-istio-envoy\") pod \"e2e-extmod-a-98b389-openshift-default-57c5bc4c69-t5g6x\" (UID: \"9f8e71e3-8cd2-4213-952e-e9db1194df27\") " pod="openshift-ingress/e2e-extmod-a-98b389-openshift-default-57c5bc4c69-t5g6x"
Oct 30 19:39:19.357833 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:19.357822    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9f8e71e3-8cd2-4213-952e-e9db1194df27-istiod-ca-cert\") pod \"e2e-extmod-a-98b389-openshift-default-57c5bc4c69-t5g6x\" (UID: \"9f8e71e3-8cd2-4213-952e-e9db1194df27\") " pod="openshift-ingress/e2e-extmod-a-98b389-openshift-default-57c5bc4c69-t5g6x"
Oct 30 19:39:19.357919 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:19.357872    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9f8e71e3-8cd2-4213-952e-e9db1194df27-istio-data\") pod \"e2e-extmod-a-98b389-openshift-default-57c5bc4c69-t5g6x\" (UID: \"9f8e71e3-8cd2-4213-952e-e9db1194df27\") " pod="openshift-ingress/e2e-extmod-a-98b389-openshift-default-57c5bc4c69-t5g6x"
Oct 30 19:39:19.358054 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:19.358031    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9f8e71e3-8cd2-4213-952e-e9db1194df27-istio-podinfo\") pod \"e2e-extmod-a-98b389-openshift-default-57c5bc4c69-t5g6x\" (UID: \"9f8e71e3-8cd2-4213-952e-e9db1194df27\") " pod="openshift-ingress/e2e-extmod-a-98b389-openshift-default-57c5bc4c69-t5g6x"
Oct 30 19:39:19.358123 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:19.358097    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f8e71e3-8cd2-4213-952e-e9db1194df27-credential-socket\") pod \"e2e-extmod-a-98b389-openshift-default-57c5bc4c69-t5g6x\" (UID: \"9f8e71e3-8cd2-4213-952e-e9db1194df27\") " pod="openshift-ingress/e2e-extmod-a-98b389-openshift-default-57c5bc4c69-t5g6x"
Oct 30 19:39:19.374329 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:19.374297    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-a-98b389-openshift-default-57c5bc4c69-t5g6x"]
Oct 30 19:39:19.459488 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:19.459451    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f8e71e3-8cd2-4213-952e-e9db1194df27-credential-socket\") pod \"e2e-extmod-a-98b389-openshift-default-57c5bc4c69-t5g6x\" (UID: \"9f8e71e3-8cd2-4213-952e-e9db1194df27\") " pod="openshift-ingress/e2e-extmod-a-98b389-openshift-default-57c5bc4c69-t5g6x"
Oct 30 19:39:19.459488 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:19.459492    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9f8e71e3-8cd2-4213-952e-e9db1194df27-workload-certs\") pod \"e2e-extmod-a-98b389-openshift-default-57c5bc4c69-t5g6x\" (UID: \"9f8e71e3-8cd2-4213-952e-e9db1194df27\") " pod="openshift-ingress/e2e-extmod-a-98b389-openshift-default-57c5bc4c69-t5g6x"
Oct 30 19:39:19.459740 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:19.459516    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4blg2\" (UniqueName: \"kubernetes.io/projected/9f8e71e3-8cd2-4213-952e-e9db1194df27-kube-api-access-4blg2\") pod \"e2e-extmod-a-98b389-openshift-default-57c5bc4c69-t5g6x\" (UID: \"9f8e71e3-8cd2-4213-952e-e9db1194df27\") " pod="openshift-ingress/e2e-extmod-a-98b389-openshift-default-57c5bc4c69-t5g6x"
Oct 30 19:39:19.459740 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:19.459550    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9f8e71e3-8cd2-4213-952e-e9db1194df27-istio-token\") pod \"e2e-extmod-a-98b389-openshift-default-57c5bc4c69-t5g6x\" (UID: \"9f8e71e3-8cd2-4213-952e-e9db1194df27\") " pod="openshift-ingress/e2e-extmod-a-98b389-openshift-default-57c5bc4c69-t5g6x"
Oct 30 19:39:19.459740 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:19.459580    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f8e71e3-8cd2-4213-952e-e9db1194df27-workload-socket\") pod \"e2e-extmod-a-98b389-openshift-default-57c5bc4c69-t5g6x\" (UID: \"9f8e71e3-8cd2-4213-952e-e9db1194df27\") " pod="openshift-ingress/e2e-extmod-a-98b389-openshift-default-57c5bc4c69-t5g6x"
Oct 30 19:39:19.459872 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:19.459738    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9f8e71e3-8cd2-4213-952e-e9db1194df27-istio-envoy\") pod \"e2e-extmod-a-98b389-openshift-default-57c5bc4c69-t5g6x\" (UID: \"9f8e71e3-8cd2-4213-952e-e9db1194df27\") " pod="openshift-ingress/e2e-extmod-a-98b389-openshift-default-57c5bc4c69-t5g6x"
Oct 30 19:39:19.459872 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:19.459801    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9f8e71e3-8cd2-4213-952e-e9db1194df27-istiod-ca-cert\") pod \"e2e-extmod-a-98b389-openshift-default-57c5bc4c69-t5g6x\" (UID: \"9f8e71e3-8cd2-4213-952e-e9db1194df27\") " pod="openshift-ingress/e2e-extmod-a-98b389-openshift-default-57c5bc4c69-t5g6x"
Oct 30 19:39:19.459872 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:19.459867    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9f8e71e3-8cd2-4213-952e-e9db1194df27-istio-data\") pod \"e2e-extmod-a-98b389-openshift-default-57c5bc4c69-t5g6x\" (UID: \"9f8e71e3-8cd2-4213-952e-e9db1194df27\") " pod="openshift-ingress/e2e-extmod-a-98b389-openshift-default-57c5bc4c69-t5g6x"
Oct 30 19:39:19.460032 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:19.459964    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f8e71e3-8cd2-4213-952e-e9db1194df27-credential-socket\") pod \"e2e-extmod-a-98b389-openshift-default-57c5bc4c69-t5g6x\" (UID: \"9f8e71e3-8cd2-4213-952e-e9db1194df27\") " pod="openshift-ingress/e2e-extmod-a-98b389-openshift-default-57c5bc4c69-t5g6x"
Oct 30 19:39:19.460032 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:19.459978    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9f8e71e3-8cd2-4213-952e-e9db1194df27-istio-podinfo\") pod \"e2e-extmod-a-98b389-openshift-default-57c5bc4c69-t5g6x\" (UID: \"9f8e71e3-8cd2-4213-952e-e9db1194df27\") " pod="openshift-ingress/e2e-extmod-a-98b389-openshift-default-57c5bc4c69-t5g6x"
Oct 30 19:39:19.460145 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:19.460003    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9f8e71e3-8cd2-4213-952e-e9db1194df27-workload-certs\") pod \"e2e-extmod-a-98b389-openshift-default-57c5bc4c69-t5g6x\" (UID: \"9f8e71e3-8cd2-4213-952e-e9db1194df27\") " pod="openshift-ingress/e2e-extmod-a-98b389-openshift-default-57c5bc4c69-t5g6x"
Oct 30 19:39:19.460145 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:19.460072    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f8e71e3-8cd2-4213-952e-e9db1194df27-workload-socket\") pod \"e2e-extmod-a-98b389-openshift-default-57c5bc4c69-t5g6x\" (UID: \"9f8e71e3-8cd2-4213-952e-e9db1194df27\") " pod="openshift-ingress/e2e-extmod-a-98b389-openshift-default-57c5bc4c69-t5g6x"
Oct 30 19:39:19.460272 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:19.460250    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9f8e71e3-8cd2-4213-952e-e9db1194df27-istio-data\") pod \"e2e-extmod-a-98b389-openshift-default-57c5bc4c69-t5g6x\" (UID: \"9f8e71e3-8cd2-4213-952e-e9db1194df27\") " pod="openshift-ingress/e2e-extmod-a-98b389-openshift-default-57c5bc4c69-t5g6x"
Oct 30 19:39:19.460697 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:19.460673    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9f8e71e3-8cd2-4213-952e-e9db1194df27-istiod-ca-cert\") pod \"e2e-extmod-a-98b389-openshift-default-57c5bc4c69-t5g6x\" (UID: \"9f8e71e3-8cd2-4213-952e-e9db1194df27\") " pod="openshift-ingress/e2e-extmod-a-98b389-openshift-default-57c5bc4c69-t5g6x"
Oct 30 19:39:19.462327 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:19.462307    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9f8e71e3-8cd2-4213-952e-e9db1194df27-istio-envoy\") pod \"e2e-extmod-a-98b389-openshift-default-57c5bc4c69-t5g6x\" (UID: \"9f8e71e3-8cd2-4213-952e-e9db1194df27\") " pod="openshift-ingress/e2e-extmod-a-98b389-openshift-default-57c5bc4c69-t5g6x"
Oct 30 19:39:19.462394 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:19.462378    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9f8e71e3-8cd2-4213-952e-e9db1194df27-istio-podinfo\") pod \"e2e-extmod-a-98b389-openshift-default-57c5bc4c69-t5g6x\" (UID: \"9f8e71e3-8cd2-4213-952e-e9db1194df27\") " pod="openshift-ingress/e2e-extmod-a-98b389-openshift-default-57c5bc4c69-t5g6x"
Oct 30 19:39:19.525426 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:19.525353    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9f8e71e3-8cd2-4213-952e-e9db1194df27-istio-token\") pod \"e2e-extmod-a-98b389-openshift-default-57c5bc4c69-t5g6x\" (UID: \"9f8e71e3-8cd2-4213-952e-e9db1194df27\") " pod="openshift-ingress/e2e-extmod-a-98b389-openshift-default-57c5bc4c69-t5g6x"
Oct 30 19:39:19.526132 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:19.526071    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4blg2\" (UniqueName: \"kubernetes.io/projected/9f8e71e3-8cd2-4213-952e-e9db1194df27-kube-api-access-4blg2\") pod \"e2e-extmod-a-98b389-openshift-default-57c5bc4c69-t5g6x\" (UID: \"9f8e71e3-8cd2-4213-952e-e9db1194df27\") " pod="openshift-ingress/e2e-extmod-a-98b389-openshift-default-57c5bc4c69-t5g6x"
Oct 30 19:39:19.578841 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:19.578815    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-98b389-openshift-default-57c5bc4c69-t5g6x"
Oct 30 19:39:20.014909 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:20.014883    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-a-98b389-openshift-default-57c5bc4c69-t5g6x"]
Oct 30 19:39:20.016880 ip-10-0-134-46 kubenswrapper[4888]: W1030 19:39:20.016853    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f8e71e3_8cd2_4213_952e_e9db1194df27.slice/crio-24b18465771fe148707f05ae67bd794970309b1ba8e4e0b843e99383b611f0b4 WatchSource:0}: Error finding container 24b18465771fe148707f05ae67bd794970309b1ba8e4e0b843e99383b611f0b4: Status 404 returned error can't find the container with id 24b18465771fe148707f05ae67bd794970309b1ba8e4e0b843e99383b611f0b4
Oct 30 19:39:20.019691 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:20.019655    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:39:20.019769 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:20.019725    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:39:20.019769 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:20.019753    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:39:20.913991 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:20.913959    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-98b389-openshift-default-57c5bc4c69-t5g6x" event={"ID":"9f8e71e3-8cd2-4213-952e-e9db1194df27","Type":"ContainerStarted","Data":"dfe03ddaf9ba723ebdb7d76350970820fb9dd9e8f07360d24e20420d2a1936b9"}
Oct 30 19:39:20.914431 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:20.914012    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-98b389-openshift-default-57c5bc4c69-t5g6x" event={"ID":"9f8e71e3-8cd2-4213-952e-e9db1194df27","Type":"ContainerStarted","Data":"24b18465771fe148707f05ae67bd794970309b1ba8e4e0b843e99383b611f0b4"}
Oct 30 19:39:21.045819 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:21.045767    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-extmod-a-98b389-openshift-default-57c5bc4c69-t5g6x" podStartSLOduration=2.045753624 podStartE2EDuration="2.045753624s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:39:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:39:21.045349161 +0000 UTC m=+3925.292755189" watchObservedRunningTime="2026-10-30 19:39:21.045753624 +0000 UTC m=+3925.293159649"
Oct 30 19:39:21.579768 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:21.579727    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-extmod-a-98b389-openshift-default-57c5bc4c69-t5g6x"
Oct 30 19:39:21.586150 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:21.586122    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-extmod-a-98b389-openshift-default-57c5bc4c69-t5g6x"
Oct 30 19:39:21.923132 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:21.923045    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-extmod-a-98b389-openshift-default-57c5bc4c69-t5g6x"
Oct 30 19:39:21.924240 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:21.924221    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-extmod-a-98b389-openshift-default-57c5bc4c69-t5g6x"
Oct 30 19:39:22.570639 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:22.570602    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-185e8b-5b56c5cf98-snzjb"]
Oct 30 19:39:22.570903 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:22.570853    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-185e8b-5b56c5cf98-snzjb" podUID="e5d89a86-c9d8-41bd-9152-a2d8e284d9e6" containerName="maas-api" containerID="cri-o://2606d91761abd9c2de0d24cdfde4b048b36c5f8eab65f866be9efa31a43fef65" gracePeriod=30
Oct 30 19:39:22.859979 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:22.859954    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-185e8b-5b56c5cf98-snzjb"
Oct 30 19:39:22.930178 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:22.930147    4888 generic.go:356] "Generic (PLEG): container finished" podID="e5d89a86-c9d8-41bd-9152-a2d8e284d9e6" containerID="2606d91761abd9c2de0d24cdfde4b048b36c5f8eab65f866be9efa31a43fef65" exitCode=0
Oct 30 19:39:22.930687 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:22.930195    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-185e8b-5b56c5cf98-snzjb"
Oct 30 19:39:22.930687 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:22.930234    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-185e8b-5b56c5cf98-snzjb" event={"ID":"e5d89a86-c9d8-41bd-9152-a2d8e284d9e6","Type":"ContainerDied","Data":"2606d91761abd9c2de0d24cdfde4b048b36c5f8eab65f866be9efa31a43fef65"}
Oct 30 19:39:22.930687 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:22.930325    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-185e8b-5b56c5cf98-snzjb" event={"ID":"e5d89a86-c9d8-41bd-9152-a2d8e284d9e6","Type":"ContainerDied","Data":"736256c27bc3c3c4353e3fe92a1bd25ed8e1f762e8eda7d9450f2625f2c0687f"}
Oct 30 19:39:22.930687 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:22.930361    4888 scope.go:122] "RemoveContainer" containerID="2606d91761abd9c2de0d24cdfde4b048b36c5f8eab65f866be9efa31a43fef65"
Oct 30 19:39:22.940737 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:22.940712    4888 scope.go:122] "RemoveContainer" containerID="2606d91761abd9c2de0d24cdfde4b048b36c5f8eab65f866be9efa31a43fef65"
Oct 30 19:39:22.941022 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:39:22.940999    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2606d91761abd9c2de0d24cdfde4b048b36c5f8eab65f866be9efa31a43fef65\": container with ID starting with 2606d91761abd9c2de0d24cdfde4b048b36c5f8eab65f866be9efa31a43fef65 not found: ID does not exist" containerID="2606d91761abd9c2de0d24cdfde4b048b36c5f8eab65f866be9efa31a43fef65"
Oct 30 19:39:22.941087 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:22.941030    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2606d91761abd9c2de0d24cdfde4b048b36c5f8eab65f866be9efa31a43fef65"} err="failed to get container status \"2606d91761abd9c2de0d24cdfde4b048b36c5f8eab65f866be9efa31a43fef65\": rpc error: code = NotFound desc = could not find container \"2606d91761abd9c2de0d24cdfde4b048b36c5f8eab65f866be9efa31a43fef65\": container with ID starting with 2606d91761abd9c2de0d24cdfde4b048b36c5f8eab65f866be9efa31a43fef65 not found: ID does not exist"
Oct 30 19:39:23.003079 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:23.003045    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e5d89a86-c9d8-41bd-9152-a2d8e284d9e6-metrics-certs\" (UniqueName: \"kubernetes.io/secret/e5d89a86-c9d8-41bd-9152-a2d8e284d9e6-metrics-certs\") pod \"e5d89a86-c9d8-41bd-9152-a2d8e284d9e6\" (UID: \"e5d89a86-c9d8-41bd-9152-a2d8e284d9e6\") "
Oct 30 19:39:23.003307 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:23.003147    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e5d89a86-c9d8-41bd-9152-a2d8e284d9e6-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e5d89a86-c9d8-41bd-9152-a2d8e284d9e6-openshift-service-ca\") pod \"e5d89a86-c9d8-41bd-9152-a2d8e284d9e6\" (UID: \"e5d89a86-c9d8-41bd-9152-a2d8e284d9e6\") "
Oct 30 19:39:23.003307 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:23.003243    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e5d89a86-c9d8-41bd-9152-a2d8e284d9e6-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e5d89a86-c9d8-41bd-9152-a2d8e284d9e6-maas-api-tls\") pod \"e5d89a86-c9d8-41bd-9152-a2d8e284d9e6\" (UID: \"e5d89a86-c9d8-41bd-9152-a2d8e284d9e6\") "
Oct 30 19:39:23.003458 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:23.003343    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e5d89a86-c9d8-41bd-9152-a2d8e284d9e6-kube-api-access-stccw\" (UniqueName: \"kubernetes.io/projected/e5d89a86-c9d8-41bd-9152-a2d8e284d9e6-kube-api-access-stccw\") pod \"e5d89a86-c9d8-41bd-9152-a2d8e284d9e6\" (UID: \"e5d89a86-c9d8-41bd-9152-a2d8e284d9e6\") "
Oct 30 19:39:23.004135 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:23.004107    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e5d89a86-c9d8-41bd-9152-a2d8e284d9e6-openshift-service-ca" pod "e5d89a86-c9d8-41bd-9152-a2d8e284d9e6" (UID: "e5d89a86-c9d8-41bd-9152-a2d8e284d9e6"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:39:23.006237 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:23.006200    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e5d89a86-c9d8-41bd-9152-a2d8e284d9e6-kube-api-access-stccw" pod "e5d89a86-c9d8-41bd-9152-a2d8e284d9e6" (UID: "e5d89a86-c9d8-41bd-9152-a2d8e284d9e6"). InnerVolumeSpecName "kube-api-access-stccw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:39:23.006679 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:23.006655    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e5d89a86-c9d8-41bd-9152-a2d8e284d9e6-metrics-certs" pod "e5d89a86-c9d8-41bd-9152-a2d8e284d9e6" (UID: "e5d89a86-c9d8-41bd-9152-a2d8e284d9e6"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:39:23.006933 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:23.006911    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e5d89a86-c9d8-41bd-9152-a2d8e284d9e6-maas-api-tls" pod "e5d89a86-c9d8-41bd-9152-a2d8e284d9e6" (UID: "e5d89a86-c9d8-41bd-9152-a2d8e284d9e6"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:39:23.104933 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:23.104846    4888 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e5d89a86-c9d8-41bd-9152-a2d8e284d9e6-metrics-certs\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:39:23.104933 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:23.104882    4888 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e5d89a86-c9d8-41bd-9152-a2d8e284d9e6-openshift-service-ca\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:39:23.104933 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:23.104899    4888 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e5d89a86-c9d8-41bd-9152-a2d8e284d9e6-maas-api-tls\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:39:23.104933 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:23.104915    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-stccw\" (UniqueName: \"kubernetes.io/projected/e5d89a86-c9d8-41bd-9152-a2d8e284d9e6-kube-api-access-stccw\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:39:23.355149 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:23.355070    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-185e8b-5b56c5cf98-snzjb"]
Oct 30 19:39:23.359668 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:23.359641    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-185e8b-5b56c5cf98-snzjb"]
Oct 30 19:39:23.895803 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:23.895763    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e5d89a86-c9d8-41bd-9152-a2d8e284d9e6" path="/var/lib/kubelet/pods/e5d89a86-c9d8-41bd-9152-a2d8e284d9e6/volumes"
Oct 30 19:39:27.882166 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:27.882137    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-98b389-7bc5cb55bc-vp4tb"]
Oct 30 19:39:27.883013 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:27.882996    4888 cpu_manager.go:414] "containerMap: removing container" podUID="e5d89a86-c9d8-41bd-9152-a2d8e284d9e6" containerName="maas-api"
Oct 30 19:39:27.883076 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:27.883016    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e5d89a86-c9d8-41bd-9152-a2d8e284d9e6" containerName="maas-api"
Oct 30 19:39:27.883196 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:27.883184    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="e5d89a86-c9d8-41bd-9152-a2d8e284d9e6" containerName="maas-api"
Oct 30 19:39:27.888529 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:27.888501    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-98b389-7bc5cb55bc-vp4tb"
Oct 30 19:39:27.942561 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:27.942533    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-98b389-7bc5cb55bc-vp4tb"]
Oct 30 19:39:27.964779 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:27.964751    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p5snp\" (UniqueName: \"kubernetes.io/projected/a49754b1-9b01-48d6-8146-a4354554e8b3-kube-api-access-p5snp\") pod \"payload-pre-processing-e2e-extmod-a-98b389-7bc5cb55bc-vp4tb\" (UID: \"a49754b1-9b01-48d6-8146-a4354554e8b3\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-98b389-7bc5cb55bc-vp4tb"
Oct 30 19:39:27.964998 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:27.964984    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a49754b1-9b01-48d6-8146-a4354554e8b3-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-a-98b389-7bc5cb55bc-vp4tb\" (UID: \"a49754b1-9b01-48d6-8146-a4354554e8b3\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-98b389-7bc5cb55bc-vp4tb"
Oct 30 19:39:27.990099 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:27.990072    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-extmod-a-98b389-dockercfg-cfltb\""
Oct 30 19:39:27.990099 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:27.990072    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-extmod-a-98b389\""
Oct 30 19:39:28.059099 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:28.059031    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-98b389-c94bf75bb-2xl4p"]
Oct 30 19:39:28.065441 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:28.065420    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-a-98b389-c94bf75bb-2xl4p"
Oct 30 19:39:28.066363 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:28.066329    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p5snp\" (UniqueName: \"kubernetes.io/projected/a49754b1-9b01-48d6-8146-a4354554e8b3-kube-api-access-p5snp\") pod \"payload-pre-processing-e2e-extmod-a-98b389-7bc5cb55bc-vp4tb\" (UID: \"a49754b1-9b01-48d6-8146-a4354554e8b3\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-98b389-7bc5cb55bc-vp4tb"
Oct 30 19:39:28.066625 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:28.066605    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a49754b1-9b01-48d6-8146-a4354554e8b3-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-a-98b389-7bc5cb55bc-vp4tb\" (UID: \"a49754b1-9b01-48d6-8146-a4354554e8b3\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-98b389-7bc5cb55bc-vp4tb"
Oct 30 19:39:28.067346 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:28.067327    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a49754b1-9b01-48d6-8146-a4354554e8b3-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-a-98b389-7bc5cb55bc-vp4tb\" (UID: \"a49754b1-9b01-48d6-8146-a4354554e8b3\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-98b389-7bc5cb55bc-vp4tb"
Oct 30 19:39:28.089944 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:28.089912    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-98b389-c94bf75bb-2xl4p"]
Oct 30 19:39:28.129084 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:28.129056    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p5snp\" (UniqueName: \"kubernetes.io/projected/a49754b1-9b01-48d6-8146-a4354554e8b3-kube-api-access-p5snp\") pod \"payload-pre-processing-e2e-extmod-a-98b389-7bc5cb55bc-vp4tb\" (UID: \"a49754b1-9b01-48d6-8146-a4354554e8b3\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-98b389-7bc5cb55bc-vp4tb"
Oct 30 19:39:28.168212 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:28.168133    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5d128a48-cd8d-4a3f-ab1b-1f007ae62e5d-plugins-config-volume\") pod \"payload-processing-e2e-extmod-a-98b389-c94bf75bb-2xl4p\" (UID: \"5d128a48-cd8d-4a3f-ab1b-1f007ae62e5d\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-98b389-c94bf75bb-2xl4p"
Oct 30 19:39:28.168388 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:28.168270    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t4mzc\" (UniqueName: \"kubernetes.io/projected/5d128a48-cd8d-4a3f-ab1b-1f007ae62e5d-kube-api-access-t4mzc\") pod \"payload-processing-e2e-extmod-a-98b389-c94bf75bb-2xl4p\" (UID: \"5d128a48-cd8d-4a3f-ab1b-1f007ae62e5d\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-98b389-c94bf75bb-2xl4p"
Oct 30 19:39:28.205593 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:28.205564    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-98b389-7bc5cb55bc-vp4tb"
Oct 30 19:39:28.269371 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:28.269339    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5d128a48-cd8d-4a3f-ab1b-1f007ae62e5d-plugins-config-volume\") pod \"payload-processing-e2e-extmod-a-98b389-c94bf75bb-2xl4p\" (UID: \"5d128a48-cd8d-4a3f-ab1b-1f007ae62e5d\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-98b389-c94bf75bb-2xl4p"
Oct 30 19:39:28.269469 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:28.269451    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t4mzc\" (UniqueName: \"kubernetes.io/projected/5d128a48-cd8d-4a3f-ab1b-1f007ae62e5d-kube-api-access-t4mzc\") pod \"payload-processing-e2e-extmod-a-98b389-c94bf75bb-2xl4p\" (UID: \"5d128a48-cd8d-4a3f-ab1b-1f007ae62e5d\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-98b389-c94bf75bb-2xl4p"
Oct 30 19:39:28.270251 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:28.270226    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5d128a48-cd8d-4a3f-ab1b-1f007ae62e5d-plugins-config-volume\") pod \"payload-processing-e2e-extmod-a-98b389-c94bf75bb-2xl4p\" (UID: \"5d128a48-cd8d-4a3f-ab1b-1f007ae62e5d\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-98b389-c94bf75bb-2xl4p"
Oct 30 19:39:28.334848 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:28.334812    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t4mzc\" (UniqueName: \"kubernetes.io/projected/5d128a48-cd8d-4a3f-ab1b-1f007ae62e5d-kube-api-access-t4mzc\") pod \"payload-processing-e2e-extmod-a-98b389-c94bf75bb-2xl4p\" (UID: \"5d128a48-cd8d-4a3f-ab1b-1f007ae62e5d\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-98b389-c94bf75bb-2xl4p"
Oct 30 19:39:28.391754 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:28.391725    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-a-98b389-c94bf75bb-2xl4p"
Oct 30 19:39:28.589189 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:28.589162    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-98b389-7bc5cb55bc-vp4tb"]
Oct 30 19:39:28.589718 ip-10-0-134-46 kubenswrapper[4888]: W1030 19:39:28.589695    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda49754b1_9b01_48d6_8146_a4354554e8b3.slice/crio-979484f63dfc5df86452ac64b8eafb685e1369defee771abb85fda08e1c61ff4 WatchSource:0}: Error finding container 979484f63dfc5df86452ac64b8eafb685e1369defee771abb85fda08e1c61ff4: Status 404 returned error can't find the container with id 979484f63dfc5df86452ac64b8eafb685e1369defee771abb85fda08e1c61ff4
Oct 30 19:39:28.773400 ip-10-0-134-46 kubenswrapper[4888]: W1030 19:39:28.773368    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d128a48_cd8d_4a3f_ab1b_1f007ae62e5d.slice/crio-8400fc569db191b09bb4318085f49fe4a1f51f8a1cd46857c5646e2f7fffb7fb WatchSource:0}: Error finding container 8400fc569db191b09bb4318085f49fe4a1f51f8a1cd46857c5646e2f7fffb7fb: Status 404 returned error can't find the container with id 8400fc569db191b09bb4318085f49fe4a1f51f8a1cd46857c5646e2f7fffb7fb
Oct 30 19:39:28.773546 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:28.773449    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-98b389-c94bf75bb-2xl4p"]
Oct 30 19:39:28.974751 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:28.974715    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-a-98b389-c94bf75bb-2xl4p" event={"ID":"5d128a48-cd8d-4a3f-ab1b-1f007ae62e5d","Type":"ContainerStarted","Data":"019e2c88716d2a5c3511d9be297a7e979ee631e3791f8afe313a085ba2a64ae2"}
Oct 30 19:39:28.975203 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:28.974771    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-a-98b389-c94bf75bb-2xl4p" event={"ID":"5d128a48-cd8d-4a3f-ab1b-1f007ae62e5d","Type":"ContainerStarted","Data":"8400fc569db191b09bb4318085f49fe4a1f51f8a1cd46857c5646e2f7fffb7fb"}
Oct 30 19:39:28.975203 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:28.974839    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-extmod-a-98b389-c94bf75bb-2xl4p"
Oct 30 19:39:28.976483 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:28.976459    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-98b389-7bc5cb55bc-vp4tb" event={"ID":"a49754b1-9b01-48d6-8146-a4354554e8b3","Type":"ContainerStarted","Data":"c746d54e45340b6fd510cb4f900004d43d494e8ba1fbea6ca8f63aa51aaa7fbd"}
Oct 30 19:39:28.976610 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:28.976501    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-98b389-7bc5cb55bc-vp4tb" event={"ID":"a49754b1-9b01-48d6-8146-a4354554e8b3","Type":"ContainerStarted","Data":"979484f63dfc5df86452ac64b8eafb685e1369defee771abb85fda08e1c61ff4"}
Oct 30 19:39:28.976610 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:28.976546    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-98b389-7bc5cb55bc-vp4tb"
Oct 30 19:39:29.095605 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:29.095560    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-extmod-a-98b389-c94bf75bb-2xl4p" podStartSLOduration=2.095546649 podStartE2EDuration="2.095546649s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:39:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:39:29.09380369 +0000 UTC m=+3933.341209714" watchObservedRunningTime="2026-10-30 19:39:29.095546649 +0000 UTC m=+3933.342952691"
Oct 30 19:39:29.224413 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:29.224351    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-98b389-7bc5cb55bc-vp4tb" podStartSLOduration=2.224333315 podStartE2EDuration="2.224333315s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:39:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:39:29.220091397 +0000 UTC m=+3933.467497422" watchObservedRunningTime="2026-10-30 19:39:29.224333315 +0000 UTC m=+3933.471739340"
Oct 30 19:39:33.082809 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:33.082769    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-b-185e8b-openshift-default-59ff49bb5c-tflbd"]
Oct 30 19:39:33.083341 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:33.083046    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-extmod-b-185e8b-openshift-default-59ff49bb5c-tflbd" podUID="6381d3a5-bf3a-454a-b888-3aaea17c86f4" containerName="istio-proxy" containerID="cri-o://d3476df7b0f7c76866f269dd6ab139d7678635163f3a6846d00651b96a1cd556" gracePeriod=30
Oct 30 19:39:35.617766 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:39:35.617732    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-jbsw6" containerName="extractor"
Oct 30 19:39:38.353520 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:38.353461    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-185e8b-openshift-default-59ff49bb5c-tflbd"
Oct 30 19:39:38.381664 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:38.381638    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6381d3a5-bf3a-454a-b888-3aaea17c86f4-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6381d3a5-bf3a-454a-b888-3aaea17c86f4-istio-envoy\") pod \"6381d3a5-bf3a-454a-b888-3aaea17c86f4\" (UID: \"6381d3a5-bf3a-454a-b888-3aaea17c86f4\") "
Oct 30 19:39:38.381795 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:38.381680    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6381d3a5-bf3a-454a-b888-3aaea17c86f4-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6381d3a5-bf3a-454a-b888-3aaea17c86f4-credential-socket\") pod \"6381d3a5-bf3a-454a-b888-3aaea17c86f4\" (UID: \"6381d3a5-bf3a-454a-b888-3aaea17c86f4\") "
Oct 30 19:39:38.381795 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:38.381709    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6381d3a5-bf3a-454a-b888-3aaea17c86f4-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6381d3a5-bf3a-454a-b888-3aaea17c86f4-istio-data\") pod \"6381d3a5-bf3a-454a-b888-3aaea17c86f4\" (UID: \"6381d3a5-bf3a-454a-b888-3aaea17c86f4\") "
Oct 30 19:39:38.381795 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:38.381728    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6381d3a5-bf3a-454a-b888-3aaea17c86f4-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6381d3a5-bf3a-454a-b888-3aaea17c86f4-istiod-ca-cert\") pod \"6381d3a5-bf3a-454a-b888-3aaea17c86f4\" (UID: \"6381d3a5-bf3a-454a-b888-3aaea17c86f4\") "
Oct 30 19:39:38.381795 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:38.381749    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6381d3a5-bf3a-454a-b888-3aaea17c86f4-kube-api-access-xz9g9\" (UniqueName: \"kubernetes.io/projected/6381d3a5-bf3a-454a-b888-3aaea17c86f4-kube-api-access-xz9g9\") pod \"6381d3a5-bf3a-454a-b888-3aaea17c86f4\" (UID: \"6381d3a5-bf3a-454a-b888-3aaea17c86f4\") "
Oct 30 19:39:38.381795 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:38.381780    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6381d3a5-bf3a-454a-b888-3aaea17c86f4-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6381d3a5-bf3a-454a-b888-3aaea17c86f4-workload-certs\") pod \"6381d3a5-bf3a-454a-b888-3aaea17c86f4\" (UID: \"6381d3a5-bf3a-454a-b888-3aaea17c86f4\") "
Oct 30 19:39:38.382125 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:38.381821    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/6381d3a5-bf3a-454a-b888-3aaea17c86f4-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6381d3a5-bf3a-454a-b888-3aaea17c86f4-istio-podinfo\") pod \"6381d3a5-bf3a-454a-b888-3aaea17c86f4\" (UID: \"6381d3a5-bf3a-454a-b888-3aaea17c86f4\") "
Oct 30 19:39:38.382125 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:38.381922    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6381d3a5-bf3a-454a-b888-3aaea17c86f4-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6381d3a5-bf3a-454a-b888-3aaea17c86f4-workload-socket\") pod \"6381d3a5-bf3a-454a-b888-3aaea17c86f4\" (UID: \"6381d3a5-bf3a-454a-b888-3aaea17c86f4\") "
Oct 30 19:39:38.382125 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:38.381954    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6381d3a5-bf3a-454a-b888-3aaea17c86f4-istio-token\" (UniqueName: \"kubernetes.io/projected/6381d3a5-bf3a-454a-b888-3aaea17c86f4-istio-token\") pod \"6381d3a5-bf3a-454a-b888-3aaea17c86f4\" (UID: \"6381d3a5-bf3a-454a-b888-3aaea17c86f4\") "
Oct 30 19:39:38.382125 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:38.382053    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6381d3a5-bf3a-454a-b888-3aaea17c86f4-credential-socket" pod "6381d3a5-bf3a-454a-b888-3aaea17c86f4" (UID: "6381d3a5-bf3a-454a-b888-3aaea17c86f4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:39:38.382400 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:38.382162    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6381d3a5-bf3a-454a-b888-3aaea17c86f4-workload-certs" pod "6381d3a5-bf3a-454a-b888-3aaea17c86f4" (UID: "6381d3a5-bf3a-454a-b888-3aaea17c86f4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:39:38.382400 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:38.382260    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6381d3a5-bf3a-454a-b888-3aaea17c86f4-istiod-ca-cert" pod "6381d3a5-bf3a-454a-b888-3aaea17c86f4" (UID: "6381d3a5-bf3a-454a-b888-3aaea17c86f4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:39:38.382400 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:38.382297    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6381d3a5-bf3a-454a-b888-3aaea17c86f4-workload-socket" pod "6381d3a5-bf3a-454a-b888-3aaea17c86f4" (UID: "6381d3a5-bf3a-454a-b888-3aaea17c86f4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:39:38.382400 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:38.382334    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6381d3a5-bf3a-454a-b888-3aaea17c86f4-istio-data" pod "6381d3a5-bf3a-454a-b888-3aaea17c86f4" (UID: "6381d3a5-bf3a-454a-b888-3aaea17c86f4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:39:38.382832 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:38.382795    4888 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6381d3a5-bf3a-454a-b888-3aaea17c86f4-credential-socket\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:39:38.382832 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:38.382823    4888 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6381d3a5-bf3a-454a-b888-3aaea17c86f4-istio-data\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:39:38.382832 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:38.382839    4888 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6381d3a5-bf3a-454a-b888-3aaea17c86f4-istiod-ca-cert\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:39:38.382832 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:38.382853    4888 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6381d3a5-bf3a-454a-b888-3aaea17c86f4-workload-certs\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:39:38.383113 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:38.382867    4888 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6381d3a5-bf3a-454a-b888-3aaea17c86f4-workload-socket\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:39:38.384202 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:38.384176    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6381d3a5-bf3a-454a-b888-3aaea17c86f4-istio-envoy" pod "6381d3a5-bf3a-454a-b888-3aaea17c86f4" (UID: "6381d3a5-bf3a-454a-b888-3aaea17c86f4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:39:38.384324 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:38.384235    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6381d3a5-bf3a-454a-b888-3aaea17c86f4-kube-api-access-xz9g9" pod "6381d3a5-bf3a-454a-b888-3aaea17c86f4" (UID: "6381d3a5-bf3a-454a-b888-3aaea17c86f4"). InnerVolumeSpecName "kube-api-access-xz9g9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:39:38.384324 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:38.384309    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6381d3a5-bf3a-454a-b888-3aaea17c86f4-istio-token" pod "6381d3a5-bf3a-454a-b888-3aaea17c86f4" (UID: "6381d3a5-bf3a-454a-b888-3aaea17c86f4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:39:38.384450 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:38.384428    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6381d3a5-bf3a-454a-b888-3aaea17c86f4-istio-podinfo" pod "6381d3a5-bf3a-454a-b888-3aaea17c86f4" (UID: "6381d3a5-bf3a-454a-b888-3aaea17c86f4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:39:38.483676 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:38.483647    4888 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6381d3a5-bf3a-454a-b888-3aaea17c86f4-istio-token\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:39:38.483676 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:38.483673    4888 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6381d3a5-bf3a-454a-b888-3aaea17c86f4-istio-envoy\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:39:38.483855 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:38.483683    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xz9g9\" (UniqueName: \"kubernetes.io/projected/6381d3a5-bf3a-454a-b888-3aaea17c86f4-kube-api-access-xz9g9\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:39:38.483855 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:38.483692    4888 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6381d3a5-bf3a-454a-b888-3aaea17c86f4-istio-podinfo\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:39:39.043436 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:39.043399    4888 generic.go:356] "Generic (PLEG): container finished" podID="6381d3a5-bf3a-454a-b888-3aaea17c86f4" containerID="d3476df7b0f7c76866f269dd6ab139d7678635163f3a6846d00651b96a1cd556" exitCode=0
Oct 30 19:39:39.043604 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:39.043468    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-185e8b-openshift-default-59ff49bb5c-tflbd"
Oct 30 19:39:39.043604 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:39.043477    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-185e8b-openshift-default-59ff49bb5c-tflbd" event={"ID":"6381d3a5-bf3a-454a-b888-3aaea17c86f4","Type":"ContainerDied","Data":"d3476df7b0f7c76866f269dd6ab139d7678635163f3a6846d00651b96a1cd556"}
Oct 30 19:39:39.043604 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:39.043533    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-185e8b-openshift-default-59ff49bb5c-tflbd" event={"ID":"6381d3a5-bf3a-454a-b888-3aaea17c86f4","Type":"ContainerDied","Data":"bbeddbfa3f43f67071e7f0148010beda68d53cf101dbe41b7d494e5bbf748400"}
Oct 30 19:39:39.043604 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:39.043556    4888 scope.go:122] "RemoveContainer" containerID="d3476df7b0f7c76866f269dd6ab139d7678635163f3a6846d00651b96a1cd556"
Oct 30 19:39:39.055266 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:39.055250    4888 scope.go:122] "RemoveContainer" containerID="fc7ae7314397c53e7e758a8ca85c7cdea891a75a12d8f00e1610b36d04b6cc28"
Oct 30 19:39:39.065241 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:39.065226    4888 scope.go:122] "RemoveContainer" containerID="d3476df7b0f7c76866f269dd6ab139d7678635163f3a6846d00651b96a1cd556"
Oct 30 19:39:39.065566 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:39:39.065543    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d3476df7b0f7c76866f269dd6ab139d7678635163f3a6846d00651b96a1cd556\": container with ID starting with d3476df7b0f7c76866f269dd6ab139d7678635163f3a6846d00651b96a1cd556 not found: ID does not exist" containerID="d3476df7b0f7c76866f269dd6ab139d7678635163f3a6846d00651b96a1cd556"
Oct 30 19:39:39.065635 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:39.065578    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d3476df7b0f7c76866f269dd6ab139d7678635163f3a6846d00651b96a1cd556"} err="failed to get container status \"d3476df7b0f7c76866f269dd6ab139d7678635163f3a6846d00651b96a1cd556\": rpc error: code = NotFound desc = could not find container \"d3476df7b0f7c76866f269dd6ab139d7678635163f3a6846d00651b96a1cd556\": container with ID starting with d3476df7b0f7c76866f269dd6ab139d7678635163f3a6846d00651b96a1cd556 not found: ID does not exist"
Oct 30 19:39:39.065635 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:39.065602    4888 scope.go:122] "RemoveContainer" containerID="fc7ae7314397c53e7e758a8ca85c7cdea891a75a12d8f00e1610b36d04b6cc28"
Oct 30 19:39:39.065890 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:39:39.065876    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fc7ae7314397c53e7e758a8ca85c7cdea891a75a12d8f00e1610b36d04b6cc28\": container with ID starting with fc7ae7314397c53e7e758a8ca85c7cdea891a75a12d8f00e1610b36d04b6cc28 not found: ID does not exist" containerID="fc7ae7314397c53e7e758a8ca85c7cdea891a75a12d8f00e1610b36d04b6cc28"
Oct 30 19:39:39.065931 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:39.065896    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fc7ae7314397c53e7e758a8ca85c7cdea891a75a12d8f00e1610b36d04b6cc28"} err="failed to get container status \"fc7ae7314397c53e7e758a8ca85c7cdea891a75a12d8f00e1610b36d04b6cc28\": rpc error: code = NotFound desc = could not find container \"fc7ae7314397c53e7e758a8ca85c7cdea891a75a12d8f00e1610b36d04b6cc28\": container with ID starting with fc7ae7314397c53e7e758a8ca85c7cdea891a75a12d8f00e1610b36d04b6cc28 not found: ID does not exist"
Oct 30 19:39:39.215024 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:39.214990    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-b-185e8b-openshift-default-59ff49bb5c-tflbd"]
Oct 30 19:39:39.266075 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:39.266046    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-extmod-b-185e8b-openshift-default-59ff49bb5c-tflbd"]
Oct 30 19:39:39.892061 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:39.892026    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6381d3a5-bf3a-454a-b888-3aaea17c86f4" path="/var/lib/kubelet/pods/6381d3a5-bf3a-454a-b888-3aaea17c86f4/volumes"
Oct 30 19:39:39.986068 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:39.986043    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-98b389-7bc5cb55bc-vp4tb"
Oct 30 19:39:39.986347 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:39.986324    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-extmod-a-98b389-c94bf75bb-2xl4p"
Oct 30 19:39:45.289258 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:45.289222    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-rjmdz"]
Oct 30 19:39:59.248302 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:59.248251    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-9e4e63-b858ffbf4-z7whl"]
Oct 30 19:39:59.249203 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:59.249142    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-9e4e63-b858ffbf4-z7whl" podUID="d8169ec1-dc9b-44a4-a5ba-09ee2ccf7a66" containerName="maas-api" containerID="cri-o://b40de078a3db8c7e46fffcf3594f00302235928942de68e5bd2a9d222513da8e" gracePeriod=30
Oct 30 19:39:59.540043 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:59.540017    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-9e4e63-b858ffbf4-z7whl"
Oct 30 19:39:59.629454 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:59.629421    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d8169ec1-dc9b-44a4-a5ba-09ee2ccf7a66-kube-api-access-hpsr4\" (UniqueName: \"kubernetes.io/projected/d8169ec1-dc9b-44a4-a5ba-09ee2ccf7a66-kube-api-access-hpsr4\") pod \"d8169ec1-dc9b-44a4-a5ba-09ee2ccf7a66\" (UID: \"d8169ec1-dc9b-44a4-a5ba-09ee2ccf7a66\") "
Oct 30 19:39:59.629667 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:59.629503    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d8169ec1-dc9b-44a4-a5ba-09ee2ccf7a66-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d8169ec1-dc9b-44a4-a5ba-09ee2ccf7a66-openshift-service-ca\") pod \"d8169ec1-dc9b-44a4-a5ba-09ee2ccf7a66\" (UID: \"d8169ec1-dc9b-44a4-a5ba-09ee2ccf7a66\") "
Oct 30 19:39:59.629667 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:59.629548    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d8169ec1-dc9b-44a4-a5ba-09ee2ccf7a66-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d8169ec1-dc9b-44a4-a5ba-09ee2ccf7a66-maas-api-tls\") pod \"d8169ec1-dc9b-44a4-a5ba-09ee2ccf7a66\" (UID: \"d8169ec1-dc9b-44a4-a5ba-09ee2ccf7a66\") "
Oct 30 19:39:59.629667 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:59.629580    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d8169ec1-dc9b-44a4-a5ba-09ee2ccf7a66-metrics-certs\" (UniqueName: \"kubernetes.io/secret/d8169ec1-dc9b-44a4-a5ba-09ee2ccf7a66-metrics-certs\") pod \"d8169ec1-dc9b-44a4-a5ba-09ee2ccf7a66\" (UID: \"d8169ec1-dc9b-44a4-a5ba-09ee2ccf7a66\") "
Oct 30 19:39:59.630180 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:59.629922    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d8169ec1-dc9b-44a4-a5ba-09ee2ccf7a66-openshift-service-ca" pod "d8169ec1-dc9b-44a4-a5ba-09ee2ccf7a66" (UID: "d8169ec1-dc9b-44a4-a5ba-09ee2ccf7a66"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:39:59.630180 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:59.630097    4888 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d8169ec1-dc9b-44a4-a5ba-09ee2ccf7a66-openshift-service-ca\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:39:59.633745 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:59.633700    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d8169ec1-dc9b-44a4-a5ba-09ee2ccf7a66-metrics-certs" pod "d8169ec1-dc9b-44a4-a5ba-09ee2ccf7a66" (UID: "d8169ec1-dc9b-44a4-a5ba-09ee2ccf7a66"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:39:59.644305 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:59.636436    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d8169ec1-dc9b-44a4-a5ba-09ee2ccf7a66-maas-api-tls" pod "d8169ec1-dc9b-44a4-a5ba-09ee2ccf7a66" (UID: "d8169ec1-dc9b-44a4-a5ba-09ee2ccf7a66"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:39:59.644305 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:59.643436    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d8169ec1-dc9b-44a4-a5ba-09ee2ccf7a66-kube-api-access-hpsr4" pod "d8169ec1-dc9b-44a4-a5ba-09ee2ccf7a66" (UID: "d8169ec1-dc9b-44a4-a5ba-09ee2ccf7a66"). InnerVolumeSpecName "kube-api-access-hpsr4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:39:59.730954 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:59.730921    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hpsr4\" (UniqueName: \"kubernetes.io/projected/d8169ec1-dc9b-44a4-a5ba-09ee2ccf7a66-kube-api-access-hpsr4\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:39:59.730954 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:59.730950    4888 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d8169ec1-dc9b-44a4-a5ba-09ee2ccf7a66-maas-api-tls\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:39:59.730954 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:39:59.730960    4888 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d8169ec1-dc9b-44a4-a5ba-09ee2ccf7a66-metrics-certs\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:40:00.249028 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:00.248933    4888 generic.go:356] "Generic (PLEG): container finished" podID="d8169ec1-dc9b-44a4-a5ba-09ee2ccf7a66" containerID="b40de078a3db8c7e46fffcf3594f00302235928942de68e5bd2a9d222513da8e" exitCode=0
Oct 30 19:40:00.249744 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:00.249711    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-9e4e63-b858ffbf4-z7whl" event={"ID":"d8169ec1-dc9b-44a4-a5ba-09ee2ccf7a66","Type":"ContainerDied","Data":"b40de078a3db8c7e46fffcf3594f00302235928942de68e5bd2a9d222513da8e"}
Oct 30 19:40:00.249935 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:00.249911    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-9e4e63-b858ffbf4-z7whl" event={"ID":"d8169ec1-dc9b-44a4-a5ba-09ee2ccf7a66","Type":"ContainerDied","Data":"1f598a05b02be50f5a3125eef54bf188d1162bb66309662542db452adf7fba4c"}
Oct 30 19:40:00.250075 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:00.250062    4888 scope.go:122] "RemoveContainer" containerID="b40de078a3db8c7e46fffcf3594f00302235928942de68e5bd2a9d222513da8e"
Oct 30 19:40:00.250451 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:00.250419    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-9e4e63-b858ffbf4-z7whl"
Oct 30 19:40:00.265494 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:00.265468    4888 scope.go:122] "RemoveContainer" containerID="b40de078a3db8c7e46fffcf3594f00302235928942de68e5bd2a9d222513da8e"
Oct 30 19:40:00.265804 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:40:00.265780    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b40de078a3db8c7e46fffcf3594f00302235928942de68e5bd2a9d222513da8e\": container with ID starting with b40de078a3db8c7e46fffcf3594f00302235928942de68e5bd2a9d222513da8e not found: ID does not exist" containerID="b40de078a3db8c7e46fffcf3594f00302235928942de68e5bd2a9d222513da8e"
Oct 30 19:40:00.265887 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:00.265806    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b40de078a3db8c7e46fffcf3594f00302235928942de68e5bd2a9d222513da8e"} err="failed to get container status \"b40de078a3db8c7e46fffcf3594f00302235928942de68e5bd2a9d222513da8e\": rpc error: code = NotFound desc = could not find container \"b40de078a3db8c7e46fffcf3594f00302235928942de68e5bd2a9d222513da8e\": container with ID starting with b40de078a3db8c7e46fffcf3594f00302235928942de68e5bd2a9d222513da8e not found: ID does not exist"
Oct 30 19:40:00.424064 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:00.424023    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-9e4e63-b858ffbf4-z7whl"]
Oct 30 19:40:00.475064 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:00.475030    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-9e4e63-b858ffbf4-z7whl"]
Oct 30 19:40:01.890543 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:01.890512    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d8169ec1-dc9b-44a4-a5ba-09ee2ccf7a66" path="/var/lib/kubelet/pods/d8169ec1-dc9b-44a4-a5ba-09ee2ccf7a66/volumes"
Oct 30 19:40:05.619086 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:40:05.619051    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-jbsw6" containerName="extractor"
Oct 30 19:40:08.776535 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:08.776504    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-a-9e4e63-openshift-default-5785d6dfd9-nbk9s"]
Oct 30 19:40:08.776966 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:08.776747    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-extmod-a-9e4e63-openshift-default-5785d6dfd9-nbk9s" podUID="5cc67731-e338-42cc-8076-5806861336d9" containerName="istio-proxy" containerID="cri-o://0fc2404f38518785f2d400650e7750467f59274faf3386fb9c3629a631a2062f" gracePeriod=30
Oct 30 19:40:10.539774 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:10.539738    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-3b6013-f879fcb57-dx298"]
Oct 30 19:40:10.540230 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:10.539967    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-3b6013-f879fcb57-dx298" podUID="1598af0f-006e-4459-9508-75aaa889964d" containerName="maas-api" containerID="cri-o://4573630803e53259ab91a8724d49972edf0628652ee2a03d23e4d8254001d0b6" gracePeriod=30
Oct 30 19:40:10.824563 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:10.824540    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-3b6013-f879fcb57-dx298"
Oct 30 19:40:10.860690 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:10.860661    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1598af0f-006e-4459-9508-75aaa889964d-metrics-certs\" (UniqueName: \"kubernetes.io/secret/1598af0f-006e-4459-9508-75aaa889964d-metrics-certs\") pod \"1598af0f-006e-4459-9508-75aaa889964d\" (UID: \"1598af0f-006e-4459-9508-75aaa889964d\") "
Oct 30 19:40:10.860881 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:10.860713    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1598af0f-006e-4459-9508-75aaa889964d-kube-api-access-hlwhk\" (UniqueName: \"kubernetes.io/projected/1598af0f-006e-4459-9508-75aaa889964d-kube-api-access-hlwhk\") pod \"1598af0f-006e-4459-9508-75aaa889964d\" (UID: \"1598af0f-006e-4459-9508-75aaa889964d\") "
Oct 30 19:40:10.860881 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:10.860849    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1598af0f-006e-4459-9508-75aaa889964d-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1598af0f-006e-4459-9508-75aaa889964d-maas-api-tls\") pod \"1598af0f-006e-4459-9508-75aaa889964d\" (UID: \"1598af0f-006e-4459-9508-75aaa889964d\") "
Oct 30 19:40:10.861014 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:10.860889    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1598af0f-006e-4459-9508-75aaa889964d-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1598af0f-006e-4459-9508-75aaa889964d-openshift-service-ca\") pod \"1598af0f-006e-4459-9508-75aaa889964d\" (UID: \"1598af0f-006e-4459-9508-75aaa889964d\") "
Oct 30 19:40:10.861838 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:10.861652    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1598af0f-006e-4459-9508-75aaa889964d-openshift-service-ca" pod "1598af0f-006e-4459-9508-75aaa889964d" (UID: "1598af0f-006e-4459-9508-75aaa889964d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:40:10.863377 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:10.863351    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1598af0f-006e-4459-9508-75aaa889964d-kube-api-access-hlwhk" pod "1598af0f-006e-4459-9508-75aaa889964d" (UID: "1598af0f-006e-4459-9508-75aaa889964d"). InnerVolumeSpecName "kube-api-access-hlwhk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:40:10.863580 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:10.863556    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1598af0f-006e-4459-9508-75aaa889964d-maas-api-tls" pod "1598af0f-006e-4459-9508-75aaa889964d" (UID: "1598af0f-006e-4459-9508-75aaa889964d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:40:10.863681 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:10.863579    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1598af0f-006e-4459-9508-75aaa889964d-metrics-certs" pod "1598af0f-006e-4459-9508-75aaa889964d" (UID: "1598af0f-006e-4459-9508-75aaa889964d"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:40:10.984327 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:10.984250    4888 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1598af0f-006e-4459-9508-75aaa889964d-metrics-certs\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:40:10.984327 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:10.984310    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hlwhk\" (UniqueName: \"kubernetes.io/projected/1598af0f-006e-4459-9508-75aaa889964d-kube-api-access-hlwhk\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:40:10.984327 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:10.984329    4888 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1598af0f-006e-4459-9508-75aaa889964d-maas-api-tls\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:40:10.984653 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:10.984344    4888 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1598af0f-006e-4459-9508-75aaa889964d-openshift-service-ca\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:40:11.333461 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:11.333427    4888 generic.go:356] "Generic (PLEG): container finished" podID="1598af0f-006e-4459-9508-75aaa889964d" containerID="4573630803e53259ab91a8724d49972edf0628652ee2a03d23e4d8254001d0b6" exitCode=0
Oct 30 19:40:11.333652 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:11.333476    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-3b6013-f879fcb57-dx298"
Oct 30 19:40:11.333652 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:11.333505    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-3b6013-f879fcb57-dx298" event={"ID":"1598af0f-006e-4459-9508-75aaa889964d","Type":"ContainerDied","Data":"4573630803e53259ab91a8724d49972edf0628652ee2a03d23e4d8254001d0b6"}
Oct 30 19:40:11.333652 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:11.333560    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-3b6013-f879fcb57-dx298" event={"ID":"1598af0f-006e-4459-9508-75aaa889964d","Type":"ContainerDied","Data":"0ba924fdb28ab90637deca8de429f5046210824f0ddae92f76f02fbd41bca26b"}
Oct 30 19:40:11.333652 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:11.333584    4888 scope.go:122] "RemoveContainer" containerID="4573630803e53259ab91a8724d49972edf0628652ee2a03d23e4d8254001d0b6"
Oct 30 19:40:11.344278 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:11.344260    4888 scope.go:122] "RemoveContainer" containerID="4573630803e53259ab91a8724d49972edf0628652ee2a03d23e4d8254001d0b6"
Oct 30 19:40:11.344611 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:40:11.344590    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4573630803e53259ab91a8724d49972edf0628652ee2a03d23e4d8254001d0b6\": container with ID starting with 4573630803e53259ab91a8724d49972edf0628652ee2a03d23e4d8254001d0b6 not found: ID does not exist" containerID="4573630803e53259ab91a8724d49972edf0628652ee2a03d23e4d8254001d0b6"
Oct 30 19:40:11.344681 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:11.344620    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4573630803e53259ab91a8724d49972edf0628652ee2a03d23e4d8254001d0b6"} err="failed to get container status \"4573630803e53259ab91a8724d49972edf0628652ee2a03d23e4d8254001d0b6\": rpc error: code = NotFound desc = could not find container \"4573630803e53259ab91a8724d49972edf0628652ee2a03d23e4d8254001d0b6\": container with ID starting with 4573630803e53259ab91a8724d49972edf0628652ee2a03d23e4d8254001d0b6 not found: ID does not exist"
Oct 30 19:40:11.504548 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:11.504513    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-3b6013-f879fcb57-dx298"]
Oct 30 19:40:11.507416 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:11.507389    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-3b6013-f879fcb57-dx298"]
Oct 30 19:40:11.891846 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:11.891804    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1598af0f-006e-4459-9508-75aaa889964d" path="/var/lib/kubelet/pods/1598af0f-006e-4459-9508-75aaa889964d/volumes"
Oct 30 19:40:14.053988 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:14.053930    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-9e4e63-openshift-default-5785d6dfd9-nbk9s"
Oct 30 19:40:14.120866 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:14.120839    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5cc67731-e338-42cc-8076-5806861336d9-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5cc67731-e338-42cc-8076-5806861336d9-workload-socket\") pod \"5cc67731-e338-42cc-8076-5806861336d9\" (UID: \"5cc67731-e338-42cc-8076-5806861336d9\") "
Oct 30 19:40:14.120999 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:14.120907    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5cc67731-e338-42cc-8076-5806861336d9-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5cc67731-e338-42cc-8076-5806861336d9-istiod-ca-cert\") pod \"5cc67731-e338-42cc-8076-5806861336d9\" (UID: \"5cc67731-e338-42cc-8076-5806861336d9\") "
Oct 30 19:40:14.120999 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:14.120981    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/5cc67731-e338-42cc-8076-5806861336d9-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5cc67731-e338-42cc-8076-5806861336d9-istio-podinfo\") pod \"5cc67731-e338-42cc-8076-5806861336d9\" (UID: \"5cc67731-e338-42cc-8076-5806861336d9\") "
Oct 30 19:40:14.121104 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:14.121017    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5cc67731-e338-42cc-8076-5806861336d9-istio-token\" (UniqueName: \"kubernetes.io/projected/5cc67731-e338-42cc-8076-5806861336d9-istio-token\") pod \"5cc67731-e338-42cc-8076-5806861336d9\" (UID: \"5cc67731-e338-42cc-8076-5806861336d9\") "
Oct 30 19:40:14.121104 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:14.121051    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5cc67731-e338-42cc-8076-5806861336d9-kube-api-access-2vghx\" (UniqueName: \"kubernetes.io/projected/5cc67731-e338-42cc-8076-5806861336d9-kube-api-access-2vghx\") pod \"5cc67731-e338-42cc-8076-5806861336d9\" (UID: \"5cc67731-e338-42cc-8076-5806861336d9\") "
Oct 30 19:40:14.121104 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:14.121093    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5cc67731-e338-42cc-8076-5806861336d9-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5cc67731-e338-42cc-8076-5806861336d9-workload-certs\") pod \"5cc67731-e338-42cc-8076-5806861336d9\" (UID: \"5cc67731-e338-42cc-8076-5806861336d9\") "
Oct 30 19:40:14.121273 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:14.121122    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5cc67731-e338-42cc-8076-5806861336d9-workload-socket" pod "5cc67731-e338-42cc-8076-5806861336d9" (UID: "5cc67731-e338-42cc-8076-5806861336d9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:40:14.121273 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:14.121139    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5cc67731-e338-42cc-8076-5806861336d9-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5cc67731-e338-42cc-8076-5806861336d9-istio-envoy\") pod \"5cc67731-e338-42cc-8076-5806861336d9\" (UID: \"5cc67731-e338-42cc-8076-5806861336d9\") "
Oct 30 19:40:14.121273 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:14.121223    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5cc67731-e338-42cc-8076-5806861336d9-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5cc67731-e338-42cc-8076-5806861336d9-istio-data\") pod \"5cc67731-e338-42cc-8076-5806861336d9\" (UID: \"5cc67731-e338-42cc-8076-5806861336d9\") "
Oct 30 19:40:14.121273 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:14.121267    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5cc67731-e338-42cc-8076-5806861336d9-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5cc67731-e338-42cc-8076-5806861336d9-credential-socket\") pod \"5cc67731-e338-42cc-8076-5806861336d9\" (UID: \"5cc67731-e338-42cc-8076-5806861336d9\") "
Oct 30 19:40:14.121581 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:14.121274    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5cc67731-e338-42cc-8076-5806861336d9-istiod-ca-cert" pod "5cc67731-e338-42cc-8076-5806861336d9" (UID: "5cc67731-e338-42cc-8076-5806861336d9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:40:14.121581 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:14.121462    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5cc67731-e338-42cc-8076-5806861336d9-workload-certs" pod "5cc67731-e338-42cc-8076-5806861336d9" (UID: "5cc67731-e338-42cc-8076-5806861336d9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:40:14.121581 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:14.121470    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5cc67731-e338-42cc-8076-5806861336d9-istio-data" pod "5cc67731-e338-42cc-8076-5806861336d9" (UID: "5cc67731-e338-42cc-8076-5806861336d9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:40:14.121770 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:14.121725    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5cc67731-e338-42cc-8076-5806861336d9-credential-socket" pod "5cc67731-e338-42cc-8076-5806861336d9" (UID: "5cc67731-e338-42cc-8076-5806861336d9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:40:14.122028 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:14.122009    4888 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5cc67731-e338-42cc-8076-5806861336d9-istiod-ca-cert\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:40:14.122096 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:14.122035    4888 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5cc67731-e338-42cc-8076-5806861336d9-workload-certs\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:40:14.122096 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:14.122051    4888 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5cc67731-e338-42cc-8076-5806861336d9-istio-data\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:40:14.122096 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:14.122064    4888 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5cc67731-e338-42cc-8076-5806861336d9-credential-socket\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:40:14.122096 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:14.122078    4888 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5cc67731-e338-42cc-8076-5806861336d9-workload-socket\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:40:14.123569 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:14.123542    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5cc67731-e338-42cc-8076-5806861336d9-istio-podinfo" pod "5cc67731-e338-42cc-8076-5806861336d9" (UID: "5cc67731-e338-42cc-8076-5806861336d9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:40:14.123682 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:14.123604    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5cc67731-e338-42cc-8076-5806861336d9-kube-api-access-2vghx" pod "5cc67731-e338-42cc-8076-5806861336d9" (UID: "5cc67731-e338-42cc-8076-5806861336d9"). InnerVolumeSpecName "kube-api-access-2vghx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:40:14.123849 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:14.123825    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5cc67731-e338-42cc-8076-5806861336d9-istio-envoy" pod "5cc67731-e338-42cc-8076-5806861336d9" (UID: "5cc67731-e338-42cc-8076-5806861336d9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:40:14.123949 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:14.123887    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5cc67731-e338-42cc-8076-5806861336d9-istio-token" pod "5cc67731-e338-42cc-8076-5806861336d9" (UID: "5cc67731-e338-42cc-8076-5806861336d9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:40:14.223234 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:14.223200    4888 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5cc67731-e338-42cc-8076-5806861336d9-istio-envoy\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:40:14.223234 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:14.223228    4888 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5cc67731-e338-42cc-8076-5806861336d9-istio-podinfo\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:40:14.223234 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:14.223237    4888 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5cc67731-e338-42cc-8076-5806861336d9-istio-token\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:40:14.223479 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:14.223245    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2vghx\" (UniqueName: \"kubernetes.io/projected/5cc67731-e338-42cc-8076-5806861336d9-kube-api-access-2vghx\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:40:14.356636 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:14.356606    4888 generic.go:356] "Generic (PLEG): container finished" podID="5cc67731-e338-42cc-8076-5806861336d9" containerID="0fc2404f38518785f2d400650e7750467f59274faf3386fb9c3629a631a2062f" exitCode=0
Oct 30 19:40:14.356849 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:14.356676    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-9e4e63-openshift-default-5785d6dfd9-nbk9s"
Oct 30 19:40:14.356849 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:14.356693    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-9e4e63-openshift-default-5785d6dfd9-nbk9s" event={"ID":"5cc67731-e338-42cc-8076-5806861336d9","Type":"ContainerDied","Data":"0fc2404f38518785f2d400650e7750467f59274faf3386fb9c3629a631a2062f"}
Oct 30 19:40:14.356849 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:14.356766    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-9e4e63-openshift-default-5785d6dfd9-nbk9s" event={"ID":"5cc67731-e338-42cc-8076-5806861336d9","Type":"ContainerDied","Data":"e454f1305396fb56b63bdcfaa72cce98a5b53d630e9ce97f03d828dda8966b93"}
Oct 30 19:40:14.356849 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:14.356793    4888 scope.go:122] "RemoveContainer" containerID="0fc2404f38518785f2d400650e7750467f59274faf3386fb9c3629a631a2062f"
Oct 30 19:40:14.368553 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:14.368439    4888 scope.go:122] "RemoveContainer" containerID="48dd6a1cb16f3c9c13e8f9359ee07c0da725624613312a0452443c8bc3383aec"
Oct 30 19:40:14.379442 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:14.379421    4888 scope.go:122] "RemoveContainer" containerID="0fc2404f38518785f2d400650e7750467f59274faf3386fb9c3629a631a2062f"
Oct 30 19:40:14.379788 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:40:14.379762    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0fc2404f38518785f2d400650e7750467f59274faf3386fb9c3629a631a2062f\": container with ID starting with 0fc2404f38518785f2d400650e7750467f59274faf3386fb9c3629a631a2062f not found: ID does not exist" containerID="0fc2404f38518785f2d400650e7750467f59274faf3386fb9c3629a631a2062f"
Oct 30 19:40:14.379876 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:14.379802    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0fc2404f38518785f2d400650e7750467f59274faf3386fb9c3629a631a2062f"} err="failed to get container status \"0fc2404f38518785f2d400650e7750467f59274faf3386fb9c3629a631a2062f\": rpc error: code = NotFound desc = could not find container \"0fc2404f38518785f2d400650e7750467f59274faf3386fb9c3629a631a2062f\": container with ID starting with 0fc2404f38518785f2d400650e7750467f59274faf3386fb9c3629a631a2062f not found: ID does not exist"
Oct 30 19:40:14.379876 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:14.379828    4888 scope.go:122] "RemoveContainer" containerID="48dd6a1cb16f3c9c13e8f9359ee07c0da725624613312a0452443c8bc3383aec"
Oct 30 19:40:14.380100 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:40:14.380081    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"48dd6a1cb16f3c9c13e8f9359ee07c0da725624613312a0452443c8bc3383aec\": container with ID starting with 48dd6a1cb16f3c9c13e8f9359ee07c0da725624613312a0452443c8bc3383aec not found: ID does not exist" containerID="48dd6a1cb16f3c9c13e8f9359ee07c0da725624613312a0452443c8bc3383aec"
Oct 30 19:40:14.380160 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:14.380111    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"48dd6a1cb16f3c9c13e8f9359ee07c0da725624613312a0452443c8bc3383aec"} err="failed to get container status \"48dd6a1cb16f3c9c13e8f9359ee07c0da725624613312a0452443c8bc3383aec\": rpc error: code = NotFound desc = could not find container \"48dd6a1cb16f3c9c13e8f9359ee07c0da725624613312a0452443c8bc3383aec\": container with ID starting with 48dd6a1cb16f3c9c13e8f9359ee07c0da725624613312a0452443c8bc3383aec not found: ID does not exist"
Oct 30 19:40:14.530143 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:14.530110    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-a-9e4e63-openshift-default-5785d6dfd9-nbk9s"]
Oct 30 19:40:14.532996 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:14.532975    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-extmod-a-9e4e63-openshift-default-5785d6dfd9-nbk9s"]
Oct 30 19:40:15.893988 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:15.893955    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5cc67731-e338-42cc-8076-5806861336d9" path="/var/lib/kubelet/pods/5cc67731-e338-42cc-8076-5806861336d9/volumes"
Oct 30 19:40:19.099616 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:19.099583    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-rjmdz"]
Oct 30 19:40:21.323008 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:21.322977    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-q8jkp"]
Oct 30 19:40:21.323448 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:21.323216    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-q8jkp" podUID="eb4ed8c8-8816-4b42-803e-3dfc9a504893" containerName="istio-proxy" containerID="cri-o://b698322c35d5de0c2a1f0ad67eb051a2d1d880ec311800ba99961c38aa77a24f" gracePeriod=30
Oct 30 19:40:21.327013 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:21.326989    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-345670-5dd96f489f-wvj5b"]
Oct 30 19:40:26.327460 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:26.327405    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-345670-5dd96f489f-wvj5b" podUID="bba5ec16-efe1-4a7a-a966-ef948a7c2b2e" containerName="payload-pre-processing" containerID="cri-o://72665270c956a269f8bebf9089be209a1a3ce4573d1eca33e2daf8d690928e64" gracePeriod=25
Oct 30 19:40:26.532961 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:26.532931    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-b-25e5ef-openshift-default-7855d5bc49-wnnkh"]
Oct 30 19:40:26.533543 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:26.533495    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-extmod-b-25e5ef-openshift-default-7855d5bc49-wnnkh" podUID="65f89a7e-6069-4aa0-a285-f83cba89116a" containerName="istio-proxy" containerID="cri-o://e2003ca119c1e926a822f1e0b67efe0913e99234f3e985458688a30576e51657" gracePeriod=30
Oct 30 19:40:26.692413 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:26.692394    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-q8jkp"
Oct 30 19:40:26.707868 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:26.707849    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-345670-5dd96f489f-wvj5b"
Oct 30 19:40:26.781668 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:26.781646    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/eb4ed8c8-8816-4b42-803e-3dfc9a504893-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eb4ed8c8-8816-4b42-803e-3dfc9a504893-istiod-ca-cert\") pod \"eb4ed8c8-8816-4b42-803e-3dfc9a504893\" (UID: \"eb4ed8c8-8816-4b42-803e-3dfc9a504893\") "
Oct 30 19:40:26.781794 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:26.781689    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/eb4ed8c8-8816-4b42-803e-3dfc9a504893-kube-api-access-j8rnj\" (UniqueName: \"kubernetes.io/projected/eb4ed8c8-8816-4b42-803e-3dfc9a504893-kube-api-access-j8rnj\") pod \"eb4ed8c8-8816-4b42-803e-3dfc9a504893\" (UID: \"eb4ed8c8-8816-4b42-803e-3dfc9a504893\") "
Oct 30 19:40:26.781794 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:26.781713    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/eb4ed8c8-8816-4b42-803e-3dfc9a504893-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eb4ed8c8-8816-4b42-803e-3dfc9a504893-istio-podinfo\") pod \"eb4ed8c8-8816-4b42-803e-3dfc9a504893\" (UID: \"eb4ed8c8-8816-4b42-803e-3dfc9a504893\") "
Oct 30 19:40:26.781989 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:26.781969    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/eb4ed8c8-8816-4b42-803e-3dfc9a504893-istio-token\" (UniqueName: \"kubernetes.io/projected/eb4ed8c8-8816-4b42-803e-3dfc9a504893-istio-token\") pod \"eb4ed8c8-8816-4b42-803e-3dfc9a504893\" (UID: \"eb4ed8c8-8816-4b42-803e-3dfc9a504893\") "
Oct 30 19:40:26.782077 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:26.782043    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eb4ed8c8-8816-4b42-803e-3dfc9a504893-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eb4ed8c8-8816-4b42-803e-3dfc9a504893-istio-data\") pod \"eb4ed8c8-8816-4b42-803e-3dfc9a504893\" (UID: \"eb4ed8c8-8816-4b42-803e-3dfc9a504893\") "
Oct 30 19:40:26.782220 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:26.782089    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eb4ed8c8-8816-4b42-803e-3dfc9a504893-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb4ed8c8-8816-4b42-803e-3dfc9a504893-credential-socket\") pod \"eb4ed8c8-8816-4b42-803e-3dfc9a504893\" (UID: \"eb4ed8c8-8816-4b42-803e-3dfc9a504893\") "
Oct 30 19:40:26.782220 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:26.782131    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eb4ed8c8-8816-4b42-803e-3dfc9a504893-istiod-ca-cert" pod "eb4ed8c8-8816-4b42-803e-3dfc9a504893" (UID: "eb4ed8c8-8816-4b42-803e-3dfc9a504893"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:40:26.782220 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:26.782155    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bba5ec16-efe1-4a7a-a966-ef948a7c2b2e-kube-api-access-hnghl\" (UniqueName: \"kubernetes.io/projected/bba5ec16-efe1-4a7a-a966-ef948a7c2b2e-kube-api-access-hnghl\") pod \"bba5ec16-efe1-4a7a-a966-ef948a7c2b2e\" (UID: \"bba5ec16-efe1-4a7a-a966-ef948a7c2b2e\") "
Oct 30 19:40:26.782421 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:26.782234    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/bba5ec16-efe1-4a7a-a966-ef948a7c2b2e-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bba5ec16-efe1-4a7a-a966-ef948a7c2b2e-plugins-config-volume\") pod \"bba5ec16-efe1-4a7a-a966-ef948a7c2b2e\" (UID: \"bba5ec16-efe1-4a7a-a966-ef948a7c2b2e\") "
Oct 30 19:40:26.782421 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:26.782322    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eb4ed8c8-8816-4b42-803e-3dfc9a504893-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eb4ed8c8-8816-4b42-803e-3dfc9a504893-workload-certs\") pod \"eb4ed8c8-8816-4b42-803e-3dfc9a504893\" (UID: \"eb4ed8c8-8816-4b42-803e-3dfc9a504893\") "
Oct 30 19:40:26.782421 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:26.782353    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eb4ed8c8-8816-4b42-803e-3dfc9a504893-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eb4ed8c8-8816-4b42-803e-3dfc9a504893-istio-envoy\") pod \"eb4ed8c8-8816-4b42-803e-3dfc9a504893\" (UID: \"eb4ed8c8-8816-4b42-803e-3dfc9a504893\") "
Oct 30 19:40:26.782421 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:26.782416    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eb4ed8c8-8816-4b42-803e-3dfc9a504893-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb4ed8c8-8816-4b42-803e-3dfc9a504893-workload-socket\") pod \"eb4ed8c8-8816-4b42-803e-3dfc9a504893\" (UID: \"eb4ed8c8-8816-4b42-803e-3dfc9a504893\") "
Oct 30 19:40:26.782653 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:26.782557    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb4ed8c8-8816-4b42-803e-3dfc9a504893-credential-socket" pod "eb4ed8c8-8816-4b42-803e-3dfc9a504893" (UID: "eb4ed8c8-8816-4b42-803e-3dfc9a504893"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:40:26.783149 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:26.782939    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb4ed8c8-8816-4b42-803e-3dfc9a504893-istio-data" pod "eb4ed8c8-8816-4b42-803e-3dfc9a504893" (UID: "eb4ed8c8-8816-4b42-803e-3dfc9a504893"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:40:26.783149 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:26.782949    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb4ed8c8-8816-4b42-803e-3dfc9a504893-workload-socket" pod "eb4ed8c8-8816-4b42-803e-3dfc9a504893" (UID: "eb4ed8c8-8816-4b42-803e-3dfc9a504893"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:40:26.783149 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:26.783000    4888 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eb4ed8c8-8816-4b42-803e-3dfc9a504893-istiod-ca-cert\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:40:26.783149 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:26.783010    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb4ed8c8-8816-4b42-803e-3dfc9a504893-workload-certs" pod "eb4ed8c8-8816-4b42-803e-3dfc9a504893" (UID: "eb4ed8c8-8816-4b42-803e-3dfc9a504893"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:40:26.783149 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:26.783021    4888 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb4ed8c8-8816-4b42-803e-3dfc9a504893-credential-socket\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:40:26.783649 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:26.783306    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bba5ec16-efe1-4a7a-a966-ef948a7c2b2e-plugins-config-volume" pod "bba5ec16-efe1-4a7a-a966-ef948a7c2b2e" (UID: "bba5ec16-efe1-4a7a-a966-ef948a7c2b2e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:40:26.784708 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:26.784683    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eb4ed8c8-8816-4b42-803e-3dfc9a504893-kube-api-access-j8rnj" pod "eb4ed8c8-8816-4b42-803e-3dfc9a504893" (UID: "eb4ed8c8-8816-4b42-803e-3dfc9a504893"). InnerVolumeSpecName "kube-api-access-j8rnj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:40:26.784832 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:26.784820    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eb4ed8c8-8816-4b42-803e-3dfc9a504893-istio-token" pod "eb4ed8c8-8816-4b42-803e-3dfc9a504893" (UID: "eb4ed8c8-8816-4b42-803e-3dfc9a504893"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:40:26.785186 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:26.785163    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb4ed8c8-8816-4b42-803e-3dfc9a504893-istio-envoy" pod "eb4ed8c8-8816-4b42-803e-3dfc9a504893" (UID: "eb4ed8c8-8816-4b42-803e-3dfc9a504893"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:40:26.785321 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:26.785278    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bba5ec16-efe1-4a7a-a966-ef948a7c2b2e-kube-api-access-hnghl" pod "bba5ec16-efe1-4a7a-a966-ef948a7c2b2e" (UID: "bba5ec16-efe1-4a7a-a966-ef948a7c2b2e"). InnerVolumeSpecName "kube-api-access-hnghl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:40:26.785370 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:26.785354    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/eb4ed8c8-8816-4b42-803e-3dfc9a504893-istio-podinfo" pod "eb4ed8c8-8816-4b42-803e-3dfc9a504893" (UID: "eb4ed8c8-8816-4b42-803e-3dfc9a504893"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:40:26.884117 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:26.884026    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hnghl\" (UniqueName: \"kubernetes.io/projected/bba5ec16-efe1-4a7a-a966-ef948a7c2b2e-kube-api-access-hnghl\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:40:26.884117 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:26.884052    4888 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bba5ec16-efe1-4a7a-a966-ef948a7c2b2e-plugins-config-volume\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:40:26.884117 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:26.884064    4888 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eb4ed8c8-8816-4b42-803e-3dfc9a504893-workload-certs\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:40:26.884117 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:26.884073    4888 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eb4ed8c8-8816-4b42-803e-3dfc9a504893-istio-envoy\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:40:26.884117 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:26.884081    4888 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb4ed8c8-8816-4b42-803e-3dfc9a504893-workload-socket\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:40:26.884117 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:26.884089    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j8rnj\" (UniqueName: \"kubernetes.io/projected/eb4ed8c8-8816-4b42-803e-3dfc9a504893-kube-api-access-j8rnj\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:40:26.884117 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:26.884097    4888 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eb4ed8c8-8816-4b42-803e-3dfc9a504893-istio-podinfo\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:40:26.884117 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:26.884106    4888 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eb4ed8c8-8816-4b42-803e-3dfc9a504893-istio-token\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:40:26.884117 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:26.884115    4888 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eb4ed8c8-8816-4b42-803e-3dfc9a504893-istio-data\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:40:27.290119 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:27.290023    4888 patch_prober.go:29] interesting pod/e2e-extmod-b-25e5ef-openshift-default-7855d5bc49-wnnkh container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 19:40:27.290119 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:27.290088    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-extmod-b-25e5ef-openshift-default-7855d5bc49-wnnkh" podUID="65f89a7e-6069-4aa0-a285-f83cba89116a" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 19:40:27.446545 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:27.446510    4888 generic.go:356] "Generic (PLEG): container finished" podID="bba5ec16-efe1-4a7a-a966-ef948a7c2b2e" containerID="72665270c956a269f8bebf9089be209a1a3ce4573d1eca33e2daf8d690928e64" exitCode=0
Oct 30 19:40:27.446986 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:27.446565    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-345670-5dd96f489f-wvj5b"
Oct 30 19:40:27.446986 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:27.446598    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-345670-5dd96f489f-wvj5b" event={"ID":"bba5ec16-efe1-4a7a-a966-ef948a7c2b2e","Type":"ContainerDied","Data":"72665270c956a269f8bebf9089be209a1a3ce4573d1eca33e2daf8d690928e64"}
Oct 30 19:40:27.446986 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:27.446657    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-345670-5dd96f489f-wvj5b" event={"ID":"bba5ec16-efe1-4a7a-a966-ef948a7c2b2e","Type":"ContainerDied","Data":"e798642adfe71d623916c51034b95fc675a81f0a2b7a05612144473d0d82d8a5"}
Oct 30 19:40:27.446986 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:27.446686    4888 scope.go:122] "RemoveContainer" containerID="72665270c956a269f8bebf9089be209a1a3ce4573d1eca33e2daf8d690928e64"
Oct 30 19:40:27.448342 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:27.448301    4888 generic.go:356] "Generic (PLEG): container finished" podID="eb4ed8c8-8816-4b42-803e-3dfc9a504893" containerID="b698322c35d5de0c2a1f0ad67eb051a2d1d880ec311800ba99961c38aa77a24f" exitCode=0
Oct 30 19:40:27.448459 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:27.448353    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-q8jkp" event={"ID":"eb4ed8c8-8816-4b42-803e-3dfc9a504893","Type":"ContainerDied","Data":"b698322c35d5de0c2a1f0ad67eb051a2d1d880ec311800ba99961c38aa77a24f"}
Oct 30 19:40:27.448459 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:27.448370    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-q8jkp"
Oct 30 19:40:27.448459 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:27.448406    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-q8jkp" event={"ID":"eb4ed8c8-8816-4b42-803e-3dfc9a504893","Type":"ContainerDied","Data":"ac919feb8d66f4360bc602358f72ab2abae196663ff6d61d7207f4601c86330d"}
Oct 30 19:40:27.457603 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:27.457583    4888 scope.go:122] "RemoveContainer" containerID="72665270c956a269f8bebf9089be209a1a3ce4573d1eca33e2daf8d690928e64"
Oct 30 19:40:27.457947 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:40:27.457885    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"72665270c956a269f8bebf9089be209a1a3ce4573d1eca33e2daf8d690928e64\": container with ID starting with 72665270c956a269f8bebf9089be209a1a3ce4573d1eca33e2daf8d690928e64 not found: ID does not exist" containerID="72665270c956a269f8bebf9089be209a1a3ce4573d1eca33e2daf8d690928e64"
Oct 30 19:40:27.457947 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:27.457911    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"72665270c956a269f8bebf9089be209a1a3ce4573d1eca33e2daf8d690928e64"} err="failed to get container status \"72665270c956a269f8bebf9089be209a1a3ce4573d1eca33e2daf8d690928e64\": rpc error: code = NotFound desc = could not find container \"72665270c956a269f8bebf9089be209a1a3ce4573d1eca33e2daf8d690928e64\": container with ID starting with 72665270c956a269f8bebf9089be209a1a3ce4573d1eca33e2daf8d690928e64 not found: ID does not exist"
Oct 30 19:40:27.457947 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:27.457931    4888 scope.go:122] "RemoveContainer" containerID="b698322c35d5de0c2a1f0ad67eb051a2d1d880ec311800ba99961c38aa77a24f"
Oct 30 19:40:27.467128 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:27.467111    4888 scope.go:122] "RemoveContainer" containerID="b698322c35d5de0c2a1f0ad67eb051a2d1d880ec311800ba99961c38aa77a24f"
Oct 30 19:40:27.467473 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:40:27.467451    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b698322c35d5de0c2a1f0ad67eb051a2d1d880ec311800ba99961c38aa77a24f\": container with ID starting with b698322c35d5de0c2a1f0ad67eb051a2d1d880ec311800ba99961c38aa77a24f not found: ID does not exist" containerID="b698322c35d5de0c2a1f0ad67eb051a2d1d880ec311800ba99961c38aa77a24f"
Oct 30 19:40:27.467541 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:27.467478    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b698322c35d5de0c2a1f0ad67eb051a2d1d880ec311800ba99961c38aa77a24f"} err="failed to get container status \"b698322c35d5de0c2a1f0ad67eb051a2d1d880ec311800ba99961c38aa77a24f\": rpc error: code = NotFound desc = could not find container \"b698322c35d5de0c2a1f0ad67eb051a2d1d880ec311800ba99961c38aa77a24f\": container with ID starting with b698322c35d5de0c2a1f0ad67eb051a2d1d880ec311800ba99961c38aa77a24f not found: ID does not exist"
Oct 30 19:40:27.615976 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:27.615949    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-345670-5dd96f489f-wvj5b"]
Oct 30 19:40:27.620868 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:27.620845    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-345670-5dd96f489f-wvj5b"]
Oct 30 19:40:27.733631 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:27.733595    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-q8jkp"]
Oct 30 19:40:27.737455 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:27.737431    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-3b6013-openshift-default-85cf856477-q8jkp"]
Oct 30 19:40:27.892583 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:27.892503    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="bba5ec16-efe1-4a7a-a966-ef948a7c2b2e" path="/var/lib/kubelet/pods/bba5ec16-efe1-4a7a-a966-ef948a7c2b2e/volumes"
Oct 30 19:40:27.893079 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:27.893057    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="eb4ed8c8-8816-4b42-803e-3dfc9a504893" path="/var/lib/kubelet/pods/eb4ed8c8-8816-4b42-803e-3dfc9a504893/volumes"
Oct 30 19:40:29.333349 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:29.333317    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-98b389-7c5dd76cdf-tbchn"]
Oct 30 19:40:29.334131 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:29.333834    4888 cpu_manager.go:414] "containerMap: removing container" podUID="eb4ed8c8-8816-4b42-803e-3dfc9a504893" containerName="istio-proxy"
Oct 30 19:40:29.334131 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:29.333845    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="eb4ed8c8-8816-4b42-803e-3dfc9a504893" containerName="istio-proxy"
Oct 30 19:40:29.334131 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:29.333864    4888 cpu_manager.go:414] "containerMap: removing container" podUID="5cc67731-e338-42cc-8076-5806861336d9" containerName="istio-proxy"
Oct 30 19:40:29.334131 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:29.333870    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5cc67731-e338-42cc-8076-5806861336d9" containerName="istio-proxy"
Oct 30 19:40:29.334131 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:29.333881    4888 cpu_manager.go:414] "containerMap: removing container" podUID="6381d3a5-bf3a-454a-b888-3aaea17c86f4" containerName="istio-proxy"
Oct 30 19:40:29.334131 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:29.333887    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6381d3a5-bf3a-454a-b888-3aaea17c86f4" containerName="istio-proxy"
Oct 30 19:40:29.334131 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:29.333895    4888 cpu_manager.go:414] "containerMap: removing container" podUID="1598af0f-006e-4459-9508-75aaa889964d" containerName="maas-api"
Oct 30 19:40:29.334131 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:29.333900    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1598af0f-006e-4459-9508-75aaa889964d" containerName="maas-api"
Oct 30 19:40:29.334131 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:29.333905    4888 cpu_manager.go:414] "containerMap: removing container" podUID="d8169ec1-dc9b-44a4-a5ba-09ee2ccf7a66" containerName="maas-api"
Oct 30 19:40:29.334131 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:29.333911    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d8169ec1-dc9b-44a4-a5ba-09ee2ccf7a66" containerName="maas-api"
Oct 30 19:40:29.334131 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:29.333917    4888 cpu_manager.go:414] "containerMap: removing container" podUID="5cc67731-e338-42cc-8076-5806861336d9" containerName="istio-proxy"
Oct 30 19:40:29.334131 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:29.333922    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5cc67731-e338-42cc-8076-5806861336d9" containerName="istio-proxy"
Oct 30 19:40:29.334131 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:29.333931    4888 cpu_manager.go:414] "containerMap: removing container" podUID="bba5ec16-efe1-4a7a-a966-ef948a7c2b2e" containerName="payload-pre-processing"
Oct 30 19:40:29.334131 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:29.333935    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bba5ec16-efe1-4a7a-a966-ef948a7c2b2e" containerName="payload-pre-processing"
Oct 30 19:40:29.334131 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:29.333953    4888 cpu_manager.go:414] "containerMap: removing container" podUID="6381d3a5-bf3a-454a-b888-3aaea17c86f4" containerName="istio-proxy"
Oct 30 19:40:29.334131 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:29.333958    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6381d3a5-bf3a-454a-b888-3aaea17c86f4" containerName="istio-proxy"
Oct 30 19:40:29.334131 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:29.334037    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="6381d3a5-bf3a-454a-b888-3aaea17c86f4" containerName="istio-proxy"
Oct 30 19:40:29.334131 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:29.334050    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="eb4ed8c8-8816-4b42-803e-3dfc9a504893" containerName="istio-proxy"
Oct 30 19:40:29.334131 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:29.334057    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="5cc67731-e338-42cc-8076-5806861336d9" containerName="istio-proxy"
Oct 30 19:40:29.334131 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:29.334063    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="1598af0f-006e-4459-9508-75aaa889964d" containerName="maas-api"
Oct 30 19:40:29.334131 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:29.334070    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="d8169ec1-dc9b-44a4-a5ba-09ee2ccf7a66" containerName="maas-api"
Oct 30 19:40:29.334131 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:29.334076    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="bba5ec16-efe1-4a7a-a966-ef948a7c2b2e" containerName="payload-pre-processing"
Oct 30 19:40:29.338970 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:29.338950    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-98b389-7c5dd76cdf-tbchn"
Oct 30 19:40:29.392356 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:29.392330    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-extmod-a-98b389-serving-cert\""
Oct 30 19:40:29.446372 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:29.446345    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-98b389-7c5dd76cdf-tbchn"]
Oct 30 19:40:29.514904 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:29.514869    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f1743252-096e-4c09-85a4-e8d60e0709c0-openshift-service-ca\") pod \"maas-api-e2e-extmod-a-98b389-7c5dd76cdf-tbchn\" (UID: \"f1743252-096e-4c09-85a4-e8d60e0709c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-98b389-7c5dd76cdf-tbchn"
Oct 30 19:40:29.515073 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:29.514923    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8w6hg\" (UniqueName: \"kubernetes.io/projected/f1743252-096e-4c09-85a4-e8d60e0709c0-kube-api-access-8w6hg\") pod \"maas-api-e2e-extmod-a-98b389-7c5dd76cdf-tbchn\" (UID: \"f1743252-096e-4c09-85a4-e8d60e0709c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-98b389-7c5dd76cdf-tbchn"
Oct 30 19:40:29.515127 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:29.515100    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f1743252-096e-4c09-85a4-e8d60e0709c0-metrics-certs\") pod \"maas-api-e2e-extmod-a-98b389-7c5dd76cdf-tbchn\" (UID: \"f1743252-096e-4c09-85a4-e8d60e0709c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-98b389-7c5dd76cdf-tbchn"
Oct 30 19:40:29.515202 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:29.515187    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f1743252-096e-4c09-85a4-e8d60e0709c0-maas-api-tls\") pod \"maas-api-e2e-extmod-a-98b389-7c5dd76cdf-tbchn\" (UID: \"f1743252-096e-4c09-85a4-e8d60e0709c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-98b389-7c5dd76cdf-tbchn"
Oct 30 19:40:29.616665 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:29.616568    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f1743252-096e-4c09-85a4-e8d60e0709c0-metrics-certs\") pod \"maas-api-e2e-extmod-a-98b389-7c5dd76cdf-tbchn\" (UID: \"f1743252-096e-4c09-85a4-e8d60e0709c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-98b389-7c5dd76cdf-tbchn"
Oct 30 19:40:29.616665 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:29.616659    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f1743252-096e-4c09-85a4-e8d60e0709c0-maas-api-tls\") pod \"maas-api-e2e-extmod-a-98b389-7c5dd76cdf-tbchn\" (UID: \"f1743252-096e-4c09-85a4-e8d60e0709c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-98b389-7c5dd76cdf-tbchn"
Oct 30 19:40:29.616922 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:29.616734    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f1743252-096e-4c09-85a4-e8d60e0709c0-openshift-service-ca\") pod \"maas-api-e2e-extmod-a-98b389-7c5dd76cdf-tbchn\" (UID: \"f1743252-096e-4c09-85a4-e8d60e0709c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-98b389-7c5dd76cdf-tbchn"
Oct 30 19:40:29.616922 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:29.616786    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8w6hg\" (UniqueName: \"kubernetes.io/projected/f1743252-096e-4c09-85a4-e8d60e0709c0-kube-api-access-8w6hg\") pod \"maas-api-e2e-extmod-a-98b389-7c5dd76cdf-tbchn\" (UID: \"f1743252-096e-4c09-85a4-e8d60e0709c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-98b389-7c5dd76cdf-tbchn"
Oct 30 19:40:29.616922 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:40:29.616789    4888 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-extmod-a-98b389-serving-cert: secret "maas-api-e2e-extmod-a-98b389-serving-cert" not found
Oct 30 19:40:29.617103 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:40:29.616927    4888 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f1743252-096e-4c09-85a4-e8d60e0709c0-maas-api-tls podName:f1743252-096e-4c09-85a4-e8d60e0709c0 nodeName:}" failed. No retries permitted until 2026-10-30 19:40:30.116904697 +0000 UTC m=+3994.364310701 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/f1743252-096e-4c09-85a4-e8d60e0709c0-maas-api-tls") pod "maas-api-e2e-extmod-a-98b389-7c5dd76cdf-tbchn" (UID: "f1743252-096e-4c09-85a4-e8d60e0709c0") : secret "maas-api-e2e-extmod-a-98b389-serving-cert" not found
Oct 30 19:40:29.617406 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:29.617386    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f1743252-096e-4c09-85a4-e8d60e0709c0-openshift-service-ca\") pod \"maas-api-e2e-extmod-a-98b389-7c5dd76cdf-tbchn\" (UID: \"f1743252-096e-4c09-85a4-e8d60e0709c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-98b389-7c5dd76cdf-tbchn"
Oct 30 19:40:29.619146 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:29.619128    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f1743252-096e-4c09-85a4-e8d60e0709c0-metrics-certs\") pod \"maas-api-e2e-extmod-a-98b389-7c5dd76cdf-tbchn\" (UID: \"f1743252-096e-4c09-85a4-e8d60e0709c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-98b389-7c5dd76cdf-tbchn"
Oct 30 19:40:29.679931 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:29.679899    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8w6hg\" (UniqueName: \"kubernetes.io/projected/f1743252-096e-4c09-85a4-e8d60e0709c0-kube-api-access-8w6hg\") pod \"maas-api-e2e-extmod-a-98b389-7c5dd76cdf-tbchn\" (UID: \"f1743252-096e-4c09-85a4-e8d60e0709c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-98b389-7c5dd76cdf-tbchn"
Oct 30 19:40:30.123772 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:30.123740    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f1743252-096e-4c09-85a4-e8d60e0709c0-maas-api-tls\") pod \"maas-api-e2e-extmod-a-98b389-7c5dd76cdf-tbchn\" (UID: \"f1743252-096e-4c09-85a4-e8d60e0709c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-98b389-7c5dd76cdf-tbchn"
Oct 30 19:40:30.126357 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:30.126333    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f1743252-096e-4c09-85a4-e8d60e0709c0-maas-api-tls\") pod \"maas-api-e2e-extmod-a-98b389-7c5dd76cdf-tbchn\" (UID: \"f1743252-096e-4c09-85a4-e8d60e0709c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-98b389-7c5dd76cdf-tbchn"
Oct 30 19:40:30.256195 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:30.256154    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-98b389-7c5dd76cdf-tbchn"
Oct 30 19:40:30.639071 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:30.639040    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-98b389-7c5dd76cdf-tbchn"]
Oct 30 19:40:30.640598 ip-10-0-134-46 kubenswrapper[4888]: W1030 19:40:30.640571    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1743252_096e_4c09_85a4_e8d60e0709c0.slice/crio-4820f9a76a6d0a80951c69637d56a5674fe101a6b06c9a36554ef3d770e6e788 WatchSource:0}: Error finding container 4820f9a76a6d0a80951c69637d56a5674fe101a6b06c9a36554ef3d770e6e788: Status 404 returned error can't find the container with id 4820f9a76a6d0a80951c69637d56a5674fe101a6b06c9a36554ef3d770e6e788
Oct 30 19:40:30.642557 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:30.642536    4888 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 19:40:31.478212 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:31.478179    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-98b389-7c5dd76cdf-tbchn" event={"ID":"f1743252-096e-4c09-85a4-e8d60e0709c0","Type":"ContainerStarted","Data":"4820f9a76a6d0a80951c69637d56a5674fe101a6b06c9a36554ef3d770e6e788"}
Oct 30 19:40:31.805383 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:31.805356    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-25e5ef-openshift-default-7855d5bc49-wnnkh"
Oct 30 19:40:31.946772 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:31.946745    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/65f89a7e-6069-4aa0-a285-f83cba89116a-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/65f89a7e-6069-4aa0-a285-f83cba89116a-workload-certs\") pod \"65f89a7e-6069-4aa0-a285-f83cba89116a\" (UID: \"65f89a7e-6069-4aa0-a285-f83cba89116a\") "
Oct 30 19:40:31.946988 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:31.946852    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/65f89a7e-6069-4aa0-a285-f83cba89116a-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/65f89a7e-6069-4aa0-a285-f83cba89116a-istiod-ca-cert\") pod \"65f89a7e-6069-4aa0-a285-f83cba89116a\" (UID: \"65f89a7e-6069-4aa0-a285-f83cba89116a\") "
Oct 30 19:40:31.946988 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:31.946962    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/65f89a7e-6069-4aa0-a285-f83cba89116a-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/65f89a7e-6069-4aa0-a285-f83cba89116a-credential-socket\") pod \"65f89a7e-6069-4aa0-a285-f83cba89116a\" (UID: \"65f89a7e-6069-4aa0-a285-f83cba89116a\") "
Oct 30 19:40:31.947127 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:31.947037    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/65f89a7e-6069-4aa0-a285-f83cba89116a-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/65f89a7e-6069-4aa0-a285-f83cba89116a-istio-envoy\") pod \"65f89a7e-6069-4aa0-a285-f83cba89116a\" (UID: \"65f89a7e-6069-4aa0-a285-f83cba89116a\") "
Oct 30 19:40:31.947127 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:31.947036    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/65f89a7e-6069-4aa0-a285-f83cba89116a-workload-certs" pod "65f89a7e-6069-4aa0-a285-f83cba89116a" (UID: "65f89a7e-6069-4aa0-a285-f83cba89116a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:40:31.947127 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:31.947086    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/65f89a7e-6069-4aa0-a285-f83cba89116a-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/65f89a7e-6069-4aa0-a285-f83cba89116a-istio-podinfo\") pod \"65f89a7e-6069-4aa0-a285-f83cba89116a\" (UID: \"65f89a7e-6069-4aa0-a285-f83cba89116a\") "
Oct 30 19:40:31.947325 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:31.947132    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/65f89a7e-6069-4aa0-a285-f83cba89116a-istio-token\" (UniqueName: \"kubernetes.io/projected/65f89a7e-6069-4aa0-a285-f83cba89116a-istio-token\") pod \"65f89a7e-6069-4aa0-a285-f83cba89116a\" (UID: \"65f89a7e-6069-4aa0-a285-f83cba89116a\") "
Oct 30 19:40:31.947325 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:31.947215    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/65f89a7e-6069-4aa0-a285-f83cba89116a-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/65f89a7e-6069-4aa0-a285-f83cba89116a-istio-data\") pod \"65f89a7e-6069-4aa0-a285-f83cba89116a\" (UID: \"65f89a7e-6069-4aa0-a285-f83cba89116a\") "
Oct 30 19:40:31.947325 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:31.947235    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/65f89a7e-6069-4aa0-a285-f83cba89116a-credential-socket" pod "65f89a7e-6069-4aa0-a285-f83cba89116a" (UID: "65f89a7e-6069-4aa0-a285-f83cba89116a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:40:31.947325 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:31.947259    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/65f89a7e-6069-4aa0-a285-f83cba89116a-kube-api-access-sk2bh\" (UniqueName: \"kubernetes.io/projected/65f89a7e-6069-4aa0-a285-f83cba89116a-kube-api-access-sk2bh\") pod \"65f89a7e-6069-4aa0-a285-f83cba89116a\" (UID: \"65f89a7e-6069-4aa0-a285-f83cba89116a\") "
Oct 30 19:40:31.947556 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:31.947340    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/65f89a7e-6069-4aa0-a285-f83cba89116a-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/65f89a7e-6069-4aa0-a285-f83cba89116a-workload-socket\") pod \"65f89a7e-6069-4aa0-a285-f83cba89116a\" (UID: \"65f89a7e-6069-4aa0-a285-f83cba89116a\") "
Oct 30 19:40:31.947556 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:31.947532    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/65f89a7e-6069-4aa0-a285-f83cba89116a-istiod-ca-cert" pod "65f89a7e-6069-4aa0-a285-f83cba89116a" (UID: "65f89a7e-6069-4aa0-a285-f83cba89116a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:40:31.947737 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:31.947716    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/65f89a7e-6069-4aa0-a285-f83cba89116a-istio-data" pod "65f89a7e-6069-4aa0-a285-f83cba89116a" (UID: "65f89a7e-6069-4aa0-a285-f83cba89116a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:40:31.947957 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:31.947936    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/65f89a7e-6069-4aa0-a285-f83cba89116a-workload-socket" pod "65f89a7e-6069-4aa0-a285-f83cba89116a" (UID: "65f89a7e-6069-4aa0-a285-f83cba89116a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:40:31.948125 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:31.948013    4888 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/65f89a7e-6069-4aa0-a285-f83cba89116a-istiod-ca-cert\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:40:31.948199 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:31.948133    4888 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/65f89a7e-6069-4aa0-a285-f83cba89116a-credential-socket\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:40:31.948199 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:31.948148    4888 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/65f89a7e-6069-4aa0-a285-f83cba89116a-istio-data\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:40:31.948199 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:31.948162    4888 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/65f89a7e-6069-4aa0-a285-f83cba89116a-workload-certs\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:40:31.950331 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:31.950259    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/65f89a7e-6069-4aa0-a285-f83cba89116a-istio-podinfo" pod "65f89a7e-6069-4aa0-a285-f83cba89116a" (UID: "65f89a7e-6069-4aa0-a285-f83cba89116a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:40:31.951992 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:31.951963    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/65f89a7e-6069-4aa0-a285-f83cba89116a-kube-api-access-sk2bh" pod "65f89a7e-6069-4aa0-a285-f83cba89116a" (UID: "65f89a7e-6069-4aa0-a285-f83cba89116a"). InnerVolumeSpecName "kube-api-access-sk2bh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:40:31.952406 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:31.952372    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/65f89a7e-6069-4aa0-a285-f83cba89116a-istio-token" pod "65f89a7e-6069-4aa0-a285-f83cba89116a" (UID: "65f89a7e-6069-4aa0-a285-f83cba89116a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:40:31.960413 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:31.960384    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/65f89a7e-6069-4aa0-a285-f83cba89116a-istio-envoy" pod "65f89a7e-6069-4aa0-a285-f83cba89116a" (UID: "65f89a7e-6069-4aa0-a285-f83cba89116a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:40:32.049784 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:32.049718    4888 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/65f89a7e-6069-4aa0-a285-f83cba89116a-istio-envoy\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:40:32.049784 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:32.049745    4888 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/65f89a7e-6069-4aa0-a285-f83cba89116a-istio-podinfo\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:40:32.049784 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:32.049755    4888 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/65f89a7e-6069-4aa0-a285-f83cba89116a-istio-token\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:40:32.049784 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:32.049764    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sk2bh\" (UniqueName: \"kubernetes.io/projected/65f89a7e-6069-4aa0-a285-f83cba89116a-kube-api-access-sk2bh\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:40:32.049784 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:32.049775    4888 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/65f89a7e-6069-4aa0-a285-f83cba89116a-workload-socket\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:40:32.487103 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:32.487067    4888 generic.go:356] "Generic (PLEG): container finished" podID="65f89a7e-6069-4aa0-a285-f83cba89116a" containerID="e2003ca119c1e926a822f1e0b67efe0913e99234f3e985458688a30576e51657" exitCode=0
Oct 30 19:40:32.487328 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:32.487114    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-25e5ef-openshift-default-7855d5bc49-wnnkh"
Oct 30 19:40:32.487328 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:32.487148    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-25e5ef-openshift-default-7855d5bc49-wnnkh" event={"ID":"65f89a7e-6069-4aa0-a285-f83cba89116a","Type":"ContainerDied","Data":"e2003ca119c1e926a822f1e0b67efe0913e99234f3e985458688a30576e51657"}
Oct 30 19:40:32.487328 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:32.487218    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-25e5ef-openshift-default-7855d5bc49-wnnkh" event={"ID":"65f89a7e-6069-4aa0-a285-f83cba89116a","Type":"ContainerDied","Data":"c6d3ddc6b17570ce5c0ba6506a3511137980be44c2ca37efd0302c41b5aef651"}
Oct 30 19:40:32.487328 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:32.487252    4888 scope.go:122] "RemoveContainer" containerID="e2003ca119c1e926a822f1e0b67efe0913e99234f3e985458688a30576e51657"
Oct 30 19:40:32.489104 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:32.489046    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-98b389-7c5dd76cdf-tbchn" event={"ID":"f1743252-096e-4c09-85a4-e8d60e0709c0","Type":"ContainerStarted","Data":"ef5833ed11ef556be277bfcbebd71a1cbbe6be942029ebe08d7e1b32de9984bc"}
Oct 30 19:40:32.489232 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:32.489182    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-98b389-7c5dd76cdf-tbchn"
Oct 30 19:40:32.498786 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:32.498767    4888 scope.go:122] "RemoveContainer" containerID="8e290a9bd38221aa30b5588024ed4367f8b85deb9e6c6a8026410431c1c29d3d"
Oct 30 19:40:32.508138 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:32.508122    4888 scope.go:122] "RemoveContainer" containerID="e2003ca119c1e926a822f1e0b67efe0913e99234f3e985458688a30576e51657"
Oct 30 19:40:32.508430 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:40:32.508408    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e2003ca119c1e926a822f1e0b67efe0913e99234f3e985458688a30576e51657\": container with ID starting with e2003ca119c1e926a822f1e0b67efe0913e99234f3e985458688a30576e51657 not found: ID does not exist" containerID="e2003ca119c1e926a822f1e0b67efe0913e99234f3e985458688a30576e51657"
Oct 30 19:40:32.508493 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:32.508442    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e2003ca119c1e926a822f1e0b67efe0913e99234f3e985458688a30576e51657"} err="failed to get container status \"e2003ca119c1e926a822f1e0b67efe0913e99234f3e985458688a30576e51657\": rpc error: code = NotFound desc = could not find container \"e2003ca119c1e926a822f1e0b67efe0913e99234f3e985458688a30576e51657\": container with ID starting with e2003ca119c1e926a822f1e0b67efe0913e99234f3e985458688a30576e51657 not found: ID does not exist"
Oct 30 19:40:32.508493 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:32.508466    4888 scope.go:122] "RemoveContainer" containerID="8e290a9bd38221aa30b5588024ed4367f8b85deb9e6c6a8026410431c1c29d3d"
Oct 30 19:40:32.508731 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:40:32.508710    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8e290a9bd38221aa30b5588024ed4367f8b85deb9e6c6a8026410431c1c29d3d\": container with ID starting with 8e290a9bd38221aa30b5588024ed4367f8b85deb9e6c6a8026410431c1c29d3d not found: ID does not exist" containerID="8e290a9bd38221aa30b5588024ed4367f8b85deb9e6c6a8026410431c1c29d3d"
Oct 30 19:40:32.508823 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:32.508735    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8e290a9bd38221aa30b5588024ed4367f8b85deb9e6c6a8026410431c1c29d3d"} err="failed to get container status \"8e290a9bd38221aa30b5588024ed4367f8b85deb9e6c6a8026410431c1c29d3d\": rpc error: code = NotFound desc = could not find container \"8e290a9bd38221aa30b5588024ed4367f8b85deb9e6c6a8026410431c1c29d3d\": container with ID starting with 8e290a9bd38221aa30b5588024ed4367f8b85deb9e6c6a8026410431c1c29d3d not found: ID does not exist"
Oct 30 19:40:32.663648 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:32.663604    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-98b389-7c5dd76cdf-tbchn" podStartSLOduration=2.716470295 podStartE2EDuration="3.663588932s" totalImagesPullingTime="947.118637ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:40:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:40:32.661528846 +0000 UTC m=+3996.908934881" watchObservedRunningTime="2026-10-30 19:40:32.663588932 +0000 UTC m=+3996.910994958"
Oct 30 19:40:32.778241 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:32.778138    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-b-25e5ef-openshift-default-7855d5bc49-wnnkh"]
Oct 30 19:40:32.785858 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:32.785823    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-extmod-b-25e5ef-openshift-default-7855d5bc49-wnnkh"]
Oct 30 19:40:33.890376 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:33.890347    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="65f89a7e-6069-4aa0-a285-f83cba89116a" path="/var/lib/kubelet/pods/65f89a7e-6069-4aa0-a285-f83cba89116a/volumes"
Oct 30 19:40:35.618070 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:40:35.617990    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-jbsw6" containerName="extractor"
Oct 30 19:40:38.503123 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:38.503097    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-98b389-7c5dd76cdf-tbchn"
Oct 30 19:40:38.925756 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:38.925716    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w1-23302c-7854f56b76-vc7z7"]
Oct 30 19:40:43.926721 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:43.926680    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-23302c-7854f56b76-vc7z7" podUID="546e6862-d08b-48ad-bb55-3fd290c8ce37" containerName="payload-processing" containerID="cri-o://99a792cb2652cd87a8111bafd5dcde3fdb2cbc29b4048b1c31fc0ada6dcd34e0" gracePeriod=25
Oct 30 19:40:45.561927 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:45.561890    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-23302c-7854f56b76-vc7z7" podUID="546e6862-d08b-48ad-bb55-3fd290c8ce37" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:40:46.792060 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:46.792021    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-rjmdz"]
Oct 30 19:40:55.561995 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:55.561955    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-23302c-7854f56b76-vc7z7" podUID="546e6862-d08b-48ad-bb55-3fd290c8ce37" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:40:56.667262 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:56.667184    4888 generic.go:356] "Generic (PLEG): container finished" podID="9f8e71e3-8cd2-4213-952e-e9db1194df27" containerID="dfe03ddaf9ba723ebdb7d76350970820fb9dd9e8f07360d24e20420d2a1936b9" exitCode=0
Oct 30 19:40:56.667724 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:56.667255    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-98b389-openshift-default-57c5bc4c69-t5g6x" event={"ID":"9f8e71e3-8cd2-4213-952e-e9db1194df27","Type":"ContainerDied","Data":"dfe03ddaf9ba723ebdb7d76350970820fb9dd9e8f07360d24e20420d2a1936b9"}
Oct 30 19:40:56.667724 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:56.667674    4888 scope.go:122] "RemoveContainer" containerID="dfe03ddaf9ba723ebdb7d76350970820fb9dd9e8f07360d24e20420d2a1936b9"
Oct 30 19:40:56.669126 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:56.669099    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:40:56.669198 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:56.669175    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:40:56.669235 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:56.669210    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:40:57.699747 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:57.699713    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-98b389-openshift-default-57c5bc4c69-t5g6x" event={"ID":"9f8e71e3-8cd2-4213-952e-e9db1194df27","Type":"ContainerStarted","Data":"c86359bd4f06848e4ea92cc55c6e73074f2feadbc0f5b617b62e908c4a1ac03e"}
Oct 30 19:40:58.581628 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:58.581598    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-extmod-a-98b389-openshift-default-57c5bc4c69-t5g6x"
Oct 30 19:40:58.588309 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:58.584370    4888 patch_prober.go:29] interesting pod/e2e-extmod-a-98b389-openshift-default-57c5bc4c69-t5g6x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.177:15021/healthz/ready\": dial tcp 10.132.0.177:15021: connect: connection refused" start-of-body=
Oct 30 19:40:58.588309 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:58.584439    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-98b389-openshift-default-57c5bc4c69-t5g6x" podUID="9f8e71e3-8cd2-4213-952e-e9db1194df27" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.177:15021/healthz/ready\": dial tcp 10.132.0.177:15021: connect: connection refused"
Oct 30 19:40:59.584311 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:59.584239    4888 patch_prober.go:29] interesting pod/e2e-extmod-a-98b389-openshift-default-57c5bc4c69-t5g6x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.177:15021/healthz/ready\": dial tcp 10.132.0.177:15021: connect: connection refused" start-of-body=
Oct 30 19:40:59.584847 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:40:59.584321    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-98b389-openshift-default-57c5bc4c69-t5g6x" podUID="9f8e71e3-8cd2-4213-952e-e9db1194df27" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.177:15021/healthz/ready\": dial tcp 10.132.0.177:15021: connect: connection refused"
Oct 30 19:41:00.583308 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:41:00.580643    4888 patch_prober.go:29] interesting pod/e2e-extmod-a-98b389-openshift-default-57c5bc4c69-t5g6x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.177:15021/healthz/ready\": dial tcp 10.132.0.177:15021: connect: connection refused" start-of-body=
Oct 30 19:41:00.583308 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:41:00.580713    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-98b389-openshift-default-57c5bc4c69-t5g6x" podUID="9f8e71e3-8cd2-4213-952e-e9db1194df27" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.177:15021/healthz/ready\": dial tcp 10.132.0.177:15021: connect: connection refused"
Oct 30 19:41:01.585312 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:41:01.581653    4888 patch_prober.go:29] interesting pod/e2e-extmod-a-98b389-openshift-default-57c5bc4c69-t5g6x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.177:15021/healthz/ready\": dial tcp 10.132.0.177:15021: connect: connection refused" start-of-body=
Oct 30 19:41:01.585312 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:41:01.581729    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-98b389-openshift-default-57c5bc4c69-t5g6x" podUID="9f8e71e3-8cd2-4213-952e-e9db1194df27" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.177:15021/healthz/ready\": dial tcp 10.132.0.177:15021: connect: connection refused"
Oct 30 19:41:02.579666 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:41:02.579629    4888 patch_prober.go:29] interesting pod/e2e-extmod-a-98b389-openshift-default-57c5bc4c69-t5g6x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.177:15021/healthz/ready\": dial tcp 10.132.0.177:15021: connect: connection refused" start-of-body=
Oct 30 19:41:02.579871 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:41:02.579716    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-98b389-openshift-default-57c5bc4c69-t5g6x" podUID="9f8e71e3-8cd2-4213-952e-e9db1194df27" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.177:15021/healthz/ready\": dial tcp 10.132.0.177:15021: connect: connection refused"
Oct 30 19:41:03.584240 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:41:03.584205    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-extmod-a-98b389-openshift-default-57c5bc4c69-t5g6x"
Oct 30 19:41:03.584728 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:41:03.584707    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-extmod-a-98b389-openshift-default-57c5bc4c69-t5g6x"
Oct 30 19:41:03.585547 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:41:03.585530    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-extmod-a-98b389-openshift-default-57c5bc4c69-t5g6x"
Oct 30 19:41:04.202658 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:41:04.202629    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-23302c-7854f56b76-vc7z7"
Oct 30 19:41:04.304296 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:41:04.304262    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/546e6862-d08b-48ad-bb55-3fd290c8ce37-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/546e6862-d08b-48ad-bb55-3fd290c8ce37-plugins-config-volume\") pod \"546e6862-d08b-48ad-bb55-3fd290c8ce37\" (UID: \"546e6862-d08b-48ad-bb55-3fd290c8ce37\") "
Oct 30 19:41:04.304488 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:41:04.304412    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/546e6862-d08b-48ad-bb55-3fd290c8ce37-kube-api-access-4vx45\" (UniqueName: \"kubernetes.io/projected/546e6862-d08b-48ad-bb55-3fd290c8ce37-kube-api-access-4vx45\") pod \"546e6862-d08b-48ad-bb55-3fd290c8ce37\" (UID: \"546e6862-d08b-48ad-bb55-3fd290c8ce37\") "
Oct 30 19:41:04.304654 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:41:04.304631    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/546e6862-d08b-48ad-bb55-3fd290c8ce37-plugins-config-volume" pod "546e6862-d08b-48ad-bb55-3fd290c8ce37" (UID: "546e6862-d08b-48ad-bb55-3fd290c8ce37"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:41:04.306655 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:41:04.306629    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/546e6862-d08b-48ad-bb55-3fd290c8ce37-kube-api-access-4vx45" pod "546e6862-d08b-48ad-bb55-3fd290c8ce37" (UID: "546e6862-d08b-48ad-bb55-3fd290c8ce37"). InnerVolumeSpecName "kube-api-access-4vx45". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:41:04.405910 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:41:04.405879    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4vx45\" (UniqueName: \"kubernetes.io/projected/546e6862-d08b-48ad-bb55-3fd290c8ce37-kube-api-access-4vx45\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:41:04.405910 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:41:04.405904    4888 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/546e6862-d08b-48ad-bb55-3fd290c8ce37-plugins-config-volume\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:41:04.749027 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:41:04.748994    4888 generic.go:356] "Generic (PLEG): container finished" podID="546e6862-d08b-48ad-bb55-3fd290c8ce37" containerID="99a792cb2652cd87a8111bafd5dcde3fdb2cbc29b4048b1c31fc0ada6dcd34e0" exitCode=0
Oct 30 19:41:04.749568 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:41:04.749039    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-23302c-7854f56b76-vc7z7"
Oct 30 19:41:04.749568 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:41:04.749088    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-23302c-7854f56b76-vc7z7" event={"ID":"546e6862-d08b-48ad-bb55-3fd290c8ce37","Type":"ContainerDied","Data":"99a792cb2652cd87a8111bafd5dcde3fdb2cbc29b4048b1c31fc0ada6dcd34e0"}
Oct 30 19:41:04.749568 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:41:04.749151    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-23302c-7854f56b76-vc7z7" event={"ID":"546e6862-d08b-48ad-bb55-3fd290c8ce37","Type":"ContainerDied","Data":"888688a4129c00b1da5213002694b7354003080b536938e3a4ec6e1cdf7157c9"}
Oct 30 19:41:04.749568 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:41:04.749183    4888 scope.go:122] "RemoveContainer" containerID="99a792cb2652cd87a8111bafd5dcde3fdb2cbc29b4048b1c31fc0ada6dcd34e0"
Oct 30 19:41:04.759480 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:41:04.759465    4888 scope.go:122] "RemoveContainer" containerID="99a792cb2652cd87a8111bafd5dcde3fdb2cbc29b4048b1c31fc0ada6dcd34e0"
Oct 30 19:41:04.759720 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:41:04.759699    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"99a792cb2652cd87a8111bafd5dcde3fdb2cbc29b4048b1c31fc0ada6dcd34e0\": container with ID starting with 99a792cb2652cd87a8111bafd5dcde3fdb2cbc29b4048b1c31fc0ada6dcd34e0 not found: ID does not exist" containerID="99a792cb2652cd87a8111bafd5dcde3fdb2cbc29b4048b1c31fc0ada6dcd34e0"
Oct 30 19:41:04.759784 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:41:04.759729    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"99a792cb2652cd87a8111bafd5dcde3fdb2cbc29b4048b1c31fc0ada6dcd34e0"} err="failed to get container status \"99a792cb2652cd87a8111bafd5dcde3fdb2cbc29b4048b1c31fc0ada6dcd34e0\": rpc error: code = NotFound desc = could not find container \"99a792cb2652cd87a8111bafd5dcde3fdb2cbc29b4048b1c31fc0ada6dcd34e0\": container with ID starting with 99a792cb2652cd87a8111bafd5dcde3fdb2cbc29b4048b1c31fc0ada6dcd34e0 not found: ID does not exist"
Oct 30 19:41:04.932821 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:41:04.932779    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w1-23302c-7854f56b76-vc7z7"]
Oct 30 19:41:04.935372 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:41:04.935346    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w1-23302c-7854f56b76-vc7z7"]
Oct 30 19:41:05.619023 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:41:05.618984    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-jbsw6" containerName="extractor"
Oct 30 19:41:05.890725 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:41:05.890648    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="546e6862-d08b-48ad-bb55-3fd290c8ce37" path="/var/lib/kubelet/pods/546e6862-d08b-48ad-bb55-3fd290c8ce37/volumes"
Oct 30 19:41:25.196031 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:41:25.195997    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-rjmdz"]
Oct 30 19:41:34.779055 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:41:34.779019    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w0-19f6ea-55f66dc48d-jgnn8"]
Oct 30 19:41:35.617468 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:41:35.617436    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-jbsw6" containerName="extractor"
Oct 30 19:41:39.195126 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:41:39.195072    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-extmod-b-5da1ae-openshift-default-77c657bb8f-nng6m"]
Oct 30 19:41:39.195912 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:41:39.195892    4888 cpu_manager.go:414] "containerMap: removing container" podUID="65f89a7e-6069-4aa0-a285-f83cba89116a" containerName="istio-proxy"
Oct 30 19:41:39.195968 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:41:39.195916    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="65f89a7e-6069-4aa0-a285-f83cba89116a" containerName="istio-proxy"
Oct 30 19:41:39.195968 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:41:39.195936    4888 cpu_manager.go:414] "containerMap: removing container" podUID="65f89a7e-6069-4aa0-a285-f83cba89116a" containerName="istio-proxy"
Oct 30 19:41:39.195968 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:41:39.195944    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="65f89a7e-6069-4aa0-a285-f83cba89116a" containerName="istio-proxy"
Oct 30 19:41:39.196068 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:41:39.195982    4888 cpu_manager.go:414] "containerMap: removing container" podUID="546e6862-d08b-48ad-bb55-3fd290c8ce37" containerName="payload-processing"
Oct 30 19:41:39.196068 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:41:39.195990    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="546e6862-d08b-48ad-bb55-3fd290c8ce37" containerName="payload-processing"
Oct 30 19:41:39.196143 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:41:39.196125    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="65f89a7e-6069-4aa0-a285-f83cba89116a" containerName="istio-proxy"
Oct 30 19:41:39.196143 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:41:39.196140    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="65f89a7e-6069-4aa0-a285-f83cba89116a" containerName="istio-proxy"
Oct 30 19:41:39.196210 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:41:39.196155    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="5cc67731-e338-42cc-8076-5806861336d9" containerName="istio-proxy"
Oct 30 19:41:39.196210 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:41:39.196167    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="546e6862-d08b-48ad-bb55-3fd290c8ce37" containerName="payload-processing"
Oct 30 19:41:39.196210 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:41:39.196182    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="6381d3a5-bf3a-454a-b888-3aaea17c86f4" containerName="istio-proxy"
Oct 30 19:41:39.199691 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:41:39.199670    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-5da1ae-openshift-default-77c657bb8f-nng6m"
Oct 30 19:41:39.266615 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:41:39.266587    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-extmod-b-5da1ae-openshift-default-dockercfg-2ll9s\""
Oct 30 19:41:39.299215 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:41:39.299177    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/242ace3b-f2c6-4fd5-abe5-31ce3ae9a12c-istiod-ca-cert\") pod \"e2e-extmod-b-5da1ae-openshift-default-77c657bb8f-nng6m\" (UID: \"242ace3b-f2c6-4fd5-abe5-31ce3ae9a12c\") " pod="openshift-ingress/e2e-extmod-b-5da1ae-openshift-default-77c657bb8f-nng6m"
Oct 30 19:41:39.299432 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:41:39.299321    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/242ace3b-f2c6-4fd5-abe5-31ce3ae9a12c-workload-certs\") pod \"e2e-extmod-b-5da1ae-openshift-default-77c657bb8f-nng6m\" (UID: \"242ace3b-f2c6-4fd5-abe5-31ce3ae9a12c\") " pod="openshift-ingress/e2e-extmod-b-5da1ae-openshift-default-77c657bb8f-nng6m"
Oct 30 19:41:39.299503 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:41:39.299441    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/242ace3b-f2c6-4fd5-abe5-31ce3ae9a12c-istio-data\") pod \"e2e-extmod-b-5da1ae-openshift-default-77c657bb8f-nng6m\" (UID: \"242ace3b-f2c6-4fd5-abe5-31ce3ae9a12c\") " pod="openshift-ingress/e2e-extmod-b-5da1ae-openshift-default-77c657bb8f-nng6m"
Oct 30 19:41:39.299543 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:41:39.299501    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/242ace3b-f2c6-4fd5-abe5-31ce3ae9a12c-credential-socket\") pod \"e2e-extmod-b-5da1ae-openshift-default-77c657bb8f-nng6m\" (UID: \"242ace3b-f2c6-4fd5-abe5-31ce3ae9a12c\") " pod="openshift-ingress/e2e-extmod-b-5da1ae-openshift-default-77c657bb8f-nng6m"
Oct 30 19:41:39.299588 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:41:39.299558    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/242ace3b-f2c6-4fd5-abe5-31ce3ae9a12c-istio-token\") pod \"e2e-extmod-b-5da1ae-openshift-default-77c657bb8f-nng6m\" (UID: \"242ace3b-f2c6-4fd5-abe5-31ce3ae9a12c\") " pod="openshift-ingress/e2e-extmod-b-5da1ae-openshift-default-77c657bb8f-nng6m"
Oct 30 19:41:39.299643 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:41:39.299589    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/242ace3b-f2c6-4fd5-abe5-31ce3ae9a12c-istio-envoy\") pod \"e2e-extmod-b-5da1ae-openshift-default-77c657bb8f-nng6m\" (UID: \"242ace3b-f2c6-4fd5-abe5-31ce3ae9a12c\") " pod="openshift-ingress/e2e-extmod-b-5da1ae-openshift-default-77c657bb8f-nng6m"
Oct 30 19:41:39.299643 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:41:39.299626    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/242ace3b-f2c6-4fd5-abe5-31ce3ae9a12c-istio-podinfo\") pod \"e2e-extmod-b-5da1ae-openshift-default-77c657bb8f-nng6m\" (UID: \"242ace3b-f2c6-4fd5-abe5-31ce3ae9a12c\") " pod="openshift-ingress/e2e-extmod-b-5da1ae-openshift-default-77c657bb8f-nng6m"
Oct 30 19:41:39.299777 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:41:39.299759    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b5shf\" (UniqueName: \"kubernetes.io/projected/242ace3b-f2c6-4fd5-abe5-31ce3ae9a12c-kube-api-access-b5shf\") pod \"e2e-extmod-b-5da1ae-openshift-default-77c657bb8f-nng6m\" (UID: \"242ace3b-f2c6-4fd5-abe5-31ce3ae9a12c\") " pod="openshift-ingress/e2e-extmod-b-5da1ae-openshift-default-77c657bb8f-nng6m"
Oct 30 19:41:39.299870 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:41:39.299846    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/242ace3b-f2c6-4fd5-abe5-31ce3ae9a12c-workload-socket\") pod \"e2e-extmod-b-5da1ae-openshift-default-77c657bb8f-nng6m\" (UID: \"242ace3b-f2c6-4fd5-abe5-31ce3ae9a12c\") " pod="openshift-ingress/e2e-extmod-b-5da1ae-openshift-default-77c657bb8f-nng6m"
Oct 30 19:41:39.320789 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:41:39.320761    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-b-5da1ae-openshift-default-77c657bb8f-nng6m"]
Oct 30 19:41:39.400870 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:41:39.400833    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/242ace3b-f2c6-4fd5-abe5-31ce3ae9a12c-istio-podinfo\") pod \"e2e-extmod-b-5da1ae-openshift-default-77c657bb8f-nng6m\" (UID: \"242ace3b-f2c6-4fd5-abe5-31ce3ae9a12c\") " pod="openshift-ingress/e2e-extmod-b-5da1ae-openshift-default-77c657bb8f-nng6m"
Oct 30 19:41:39.401102 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:41:39.400889    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b5shf\" (UniqueName: \"kubernetes.io/projected/242ace3b-f2c6-4fd5-abe5-31ce3ae9a12c-kube-api-access-b5shf\") pod \"e2e-extmod-b-5da1ae-openshift-default-77c657bb8f-nng6m\" (UID: \"242ace3b-f2c6-4fd5-abe5-31ce3ae9a12c\") " pod="openshift-ingress/e2e-extmod-b-5da1ae-openshift-default-77c657bb8f-nng6m"
Oct 30 19:41:39.401102 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:41:39.400922    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/242ace3b-f2c6-4fd5-abe5-31ce3ae9a12c-workload-socket\") pod \"e2e-extmod-b-5da1ae-openshift-default-77c657bb8f-nng6m\" (UID: \"242ace3b-f2c6-4fd5-abe5-31ce3ae9a12c\") " pod="openshift-ingress/e2e-extmod-b-5da1ae-openshift-default-77c657bb8f-nng6m"
Oct 30 19:41:39.401102 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:41:39.400972    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/242ace3b-f2c6-4fd5-abe5-31ce3ae9a12c-istiod-ca-cert\") pod \"e2e-extmod-b-5da1ae-openshift-default-77c657bb8f-nng6m\" (UID: \"242ace3b-f2c6-4fd5-abe5-31ce3ae9a12c\") " pod="openshift-ingress/e2e-extmod-b-5da1ae-openshift-default-77c657bb8f-nng6m"
Oct 30 19:41:39.401102 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:41:39.401030    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/242ace3b-f2c6-4fd5-abe5-31ce3ae9a12c-workload-certs\") pod \"e2e-extmod-b-5da1ae-openshift-default-77c657bb8f-nng6m\" (UID: \"242ace3b-f2c6-4fd5-abe5-31ce3ae9a12c\") " pod="openshift-ingress/e2e-extmod-b-5da1ae-openshift-default-77c657bb8f-nng6m"
Oct 30 19:41:39.401396 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:41:39.401141    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/242ace3b-f2c6-4fd5-abe5-31ce3ae9a12c-istio-data\") pod \"e2e-extmod-b-5da1ae-openshift-default-77c657bb8f-nng6m\" (UID: \"242ace3b-f2c6-4fd5-abe5-31ce3ae9a12c\") " pod="openshift-ingress/e2e-extmod-b-5da1ae-openshift-default-77c657bb8f-nng6m"
Oct 30 19:41:39.401396 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:41:39.401217    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/242ace3b-f2c6-4fd5-abe5-31ce3ae9a12c-credential-socket\") pod \"e2e-extmod-b-5da1ae-openshift-default-77c657bb8f-nng6m\" (UID: \"242ace3b-f2c6-4fd5-abe5-31ce3ae9a12c\") " pod="openshift-ingress/e2e-extmod-b-5da1ae-openshift-default-77c657bb8f-nng6m"
Oct 30 19:41:39.401396 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:41:39.401252    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/242ace3b-f2c6-4fd5-abe5-31ce3ae9a12c-istio-token\") pod \"e2e-extmod-b-5da1ae-openshift-default-77c657bb8f-nng6m\" (UID: \"242ace3b-f2c6-4fd5-abe5-31ce3ae9a12c\") " pod="openshift-ingress/e2e-extmod-b-5da1ae-openshift-default-77c657bb8f-nng6m"
Oct 30 19:41:39.401396 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:41:39.401295    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/242ace3b-f2c6-4fd5-abe5-31ce3ae9a12c-istio-envoy\") pod \"e2e-extmod-b-5da1ae-openshift-default-77c657bb8f-nng6m\" (UID: \"242ace3b-f2c6-4fd5-abe5-31ce3ae9a12c\") " pod="openshift-ingress/e2e-extmod-b-5da1ae-openshift-default-77c657bb8f-nng6m"
Oct 30 19:41:39.401643 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:41:39.401466    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/242ace3b-f2c6-4fd5-abe5-31ce3ae9a12c-workload-socket\") pod \"e2e-extmod-b-5da1ae-openshift-default-77c657bb8f-nng6m\" (UID: \"242ace3b-f2c6-4fd5-abe5-31ce3ae9a12c\") " pod="openshift-ingress/e2e-extmod-b-5da1ae-openshift-default-77c657bb8f-nng6m"
Oct 30 19:41:39.401643 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:41:39.401483    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/242ace3b-f2c6-4fd5-abe5-31ce3ae9a12c-workload-certs\") pod \"e2e-extmod-b-5da1ae-openshift-default-77c657bb8f-nng6m\" (UID: \"242ace3b-f2c6-4fd5-abe5-31ce3ae9a12c\") " pod="openshift-ingress/e2e-extmod-b-5da1ae-openshift-default-77c657bb8f-nng6m"
Oct 30 19:41:39.401643 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:41:39.401560    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/242ace3b-f2c6-4fd5-abe5-31ce3ae9a12c-istio-data\") pod \"e2e-extmod-b-5da1ae-openshift-default-77c657bb8f-nng6m\" (UID: \"242ace3b-f2c6-4fd5-abe5-31ce3ae9a12c\") " pod="openshift-ingress/e2e-extmod-b-5da1ae-openshift-default-77c657bb8f-nng6m"
Oct 30 19:41:39.401805 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:41:39.401726    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/242ace3b-f2c6-4fd5-abe5-31ce3ae9a12c-credential-socket\") pod \"e2e-extmod-b-5da1ae-openshift-default-77c657bb8f-nng6m\" (UID: \"242ace3b-f2c6-4fd5-abe5-31ce3ae9a12c\") " pod="openshift-ingress/e2e-extmod-b-5da1ae-openshift-default-77c657bb8f-nng6m"
Oct 30 19:41:39.402190 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:41:39.402169    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/242ace3b-f2c6-4fd5-abe5-31ce3ae9a12c-istiod-ca-cert\") pod \"e2e-extmod-b-5da1ae-openshift-default-77c657bb8f-nng6m\" (UID: \"242ace3b-f2c6-4fd5-abe5-31ce3ae9a12c\") " pod="openshift-ingress/e2e-extmod-b-5da1ae-openshift-default-77c657bb8f-nng6m"
Oct 30 19:41:39.403747 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:41:39.403721    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/242ace3b-f2c6-4fd5-abe5-31ce3ae9a12c-istio-envoy\") pod \"e2e-extmod-b-5da1ae-openshift-default-77c657bb8f-nng6m\" (UID: \"242ace3b-f2c6-4fd5-abe5-31ce3ae9a12c\") " pod="openshift-ingress/e2e-extmod-b-5da1ae-openshift-default-77c657bb8f-nng6m"
Oct 30 19:41:39.404226 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:41:39.404207    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/242ace3b-f2c6-4fd5-abe5-31ce3ae9a12c-istio-podinfo\") pod \"e2e-extmod-b-5da1ae-openshift-default-77c657bb8f-nng6m\" (UID: \"242ace3b-f2c6-4fd5-abe5-31ce3ae9a12c\") " pod="openshift-ingress/e2e-extmod-b-5da1ae-openshift-default-77c657bb8f-nng6m"
Oct 30 19:41:39.465976 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:41:39.465906    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/242ace3b-f2c6-4fd5-abe5-31ce3ae9a12c-istio-token\") pod \"e2e-extmod-b-5da1ae-openshift-default-77c657bb8f-nng6m\" (UID: \"242ace3b-f2c6-4fd5-abe5-31ce3ae9a12c\") " pod="openshift-ingress/e2e-extmod-b-5da1ae-openshift-default-77c657bb8f-nng6m"
Oct 30 19:41:39.466254 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:41:39.466232    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b5shf\" (UniqueName: \"kubernetes.io/projected/242ace3b-f2c6-4fd5-abe5-31ce3ae9a12c-kube-api-access-b5shf\") pod \"e2e-extmod-b-5da1ae-openshift-default-77c657bb8f-nng6m\" (UID: \"242ace3b-f2c6-4fd5-abe5-31ce3ae9a12c\") " pod="openshift-ingress/e2e-extmod-b-5da1ae-openshift-default-77c657bb8f-nng6m"
Oct 30 19:41:39.516083 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:41:39.516048    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-5da1ae-openshift-default-77c657bb8f-nng6m"
Oct 30 19:41:39.780306 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:41:39.780172    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-19f6ea-55f66dc48d-jgnn8" podUID="50a86a24-23eb-4bd4-abbd-9327c64cc99a" containerName="payload-processing" containerID="cri-o://cce12db065b94049db00267c524af8becc40b545dca5b23f61f45de44e324684" gracePeriod=25
Oct 30 19:41:39.952845 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:41:39.952813    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-b-5da1ae-openshift-default-77c657bb8f-nng6m"]
Oct 30 19:41:39.953355 ip-10-0-134-46 kubenswrapper[4888]: W1030 19:41:39.953327    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod242ace3b_f2c6_4fd5_abe5_31ce3ae9a12c.slice/crio-d535ec0f9ea74d8f02efb67dd2a3a94918453d74492a1045fb1184af68d4b11c WatchSource:0}: Error finding container d535ec0f9ea74d8f02efb67dd2a3a94918453d74492a1045fb1184af68d4b11c: Status 404 returned error can't find the container with id d535ec0f9ea74d8f02efb67dd2a3a94918453d74492a1045fb1184af68d4b11c
Oct 30 19:41:39.955875 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:41:39.955842    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:41:39.955952 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:41:39.955909    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:41:39.955952 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:41:39.955945    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:41:39.992210 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:41:39.992179    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-5da1ae-openshift-default-77c657bb8f-nng6m" event={"ID":"242ace3b-f2c6-4fd5-abe5-31ce3ae9a12c","Type":"ContainerStarted","Data":"d535ec0f9ea74d8f02efb67dd2a3a94918453d74492a1045fb1184af68d4b11c"}
Oct 30 19:41:41.001135 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:41:41.001099    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-5da1ae-openshift-default-77c657bb8f-nng6m" event={"ID":"242ace3b-f2c6-4fd5-abe5-31ce3ae9a12c","Type":"ContainerStarted","Data":"296603c2add4596d5de87bdb74d1cb67d542d22cc7ec6581922a41b57a855676"}
Oct 30 19:41:41.177295 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:41:41.177225    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-extmod-b-5da1ae-openshift-default-77c657bb8f-nng6m" podStartSLOduration=2.177210005 podStartE2EDuration="2.177210005s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:41:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:41:41.174891974 +0000 UTC m=+4065.422298041" watchObservedRunningTime="2026-10-30 19:41:41.177210005 +0000 UTC m=+4065.424616030"
Oct 30 19:41:41.521112 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:41:41.521022    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-extmod-b-5da1ae-openshift-default-77c657bb8f-nng6m"
Oct 30 19:41:41.533690 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:41:41.533654    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-extmod-b-5da1ae-openshift-default-77c657bb8f-nng6m"
Oct 30 19:41:42.007549 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:41:42.007523    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-extmod-b-5da1ae-openshift-default-77c657bb8f-nng6m"
Oct 30 19:41:42.008408 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:41:42.008386    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-extmod-b-5da1ae-openshift-default-77c657bb8f-nng6m"
Oct 30 19:41:49.278932 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:41:49.278897    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-4a7a52/premium-w2-4a7a52-kserve-746d9579f6-hn7hl"]
Oct 30 19:41:49.690821 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:41:49.690782    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-19f6ea-55f66dc48d-jgnn8" podUID="50a86a24-23eb-4bd4-abbd-9327c64cc99a" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:41:59.691536 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:41:59.691499    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-19f6ea-55f66dc48d-jgnn8" podUID="50a86a24-23eb-4bd4-abbd-9327c64cc99a" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:42:00.062520 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:42:00.062490    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-19f6ea-55f66dc48d-jgnn8"
Oct 30 19:42:00.142598 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:42:00.142556    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/50a86a24-23eb-4bd4-abbd-9327c64cc99a-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/50a86a24-23eb-4bd4-abbd-9327c64cc99a-plugins-config-volume\") pod \"50a86a24-23eb-4bd4-abbd-9327c64cc99a\" (UID: \"50a86a24-23eb-4bd4-abbd-9327c64cc99a\") "
Oct 30 19:42:00.142788 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:42:00.142681    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/50a86a24-23eb-4bd4-abbd-9327c64cc99a-kube-api-access-jhhb2\" (UniqueName: \"kubernetes.io/projected/50a86a24-23eb-4bd4-abbd-9327c64cc99a-kube-api-access-jhhb2\") pod \"50a86a24-23eb-4bd4-abbd-9327c64cc99a\" (UID: \"50a86a24-23eb-4bd4-abbd-9327c64cc99a\") "
Oct 30 19:42:00.143045 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:42:00.143021    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/50a86a24-23eb-4bd4-abbd-9327c64cc99a-plugins-config-volume" pod "50a86a24-23eb-4bd4-abbd-9327c64cc99a" (UID: "50a86a24-23eb-4bd4-abbd-9327c64cc99a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:42:00.143180 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:42:00.143163    4888 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/50a86a24-23eb-4bd4-abbd-9327c64cc99a-plugins-config-volume\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:42:00.145173 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:42:00.145142    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/50a86a24-23eb-4bd4-abbd-9327c64cc99a-kube-api-access-jhhb2" pod "50a86a24-23eb-4bd4-abbd-9327c64cc99a" (UID: "50a86a24-23eb-4bd4-abbd-9327c64cc99a"). InnerVolumeSpecName "kube-api-access-jhhb2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:42:00.152605 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:42:00.152550    4888 generic.go:356] "Generic (PLEG): container finished" podID="50a86a24-23eb-4bd4-abbd-9327c64cc99a" containerID="cce12db065b94049db00267c524af8becc40b545dca5b23f61f45de44e324684" exitCode=0
Oct 30 19:42:00.152605 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:42:00.152601    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-19f6ea-55f66dc48d-jgnn8"
Oct 30 19:42:00.152886 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:42:00.152634    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-19f6ea-55f66dc48d-jgnn8" event={"ID":"50a86a24-23eb-4bd4-abbd-9327c64cc99a","Type":"ContainerDied","Data":"cce12db065b94049db00267c524af8becc40b545dca5b23f61f45de44e324684"}
Oct 30 19:42:00.152886 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:42:00.152708    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-19f6ea-55f66dc48d-jgnn8" event={"ID":"50a86a24-23eb-4bd4-abbd-9327c64cc99a","Type":"ContainerDied","Data":"38dc2e9741bad61417f6bfc0b979b4ad9c43089dc5993632a88ce3245640b932"}
Oct 30 19:42:00.152886 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:42:00.152741    4888 scope.go:122] "RemoveContainer" containerID="cce12db065b94049db00267c524af8becc40b545dca5b23f61f45de44e324684"
Oct 30 19:42:00.170083 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:42:00.170055    4888 scope.go:122] "RemoveContainer" containerID="cce12db065b94049db00267c524af8becc40b545dca5b23f61f45de44e324684"
Oct 30 19:42:00.170491 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:42:00.170466    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cce12db065b94049db00267c524af8becc40b545dca5b23f61f45de44e324684\": container with ID starting with cce12db065b94049db00267c524af8becc40b545dca5b23f61f45de44e324684 not found: ID does not exist" containerID="cce12db065b94049db00267c524af8becc40b545dca5b23f61f45de44e324684"
Oct 30 19:42:00.170577 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:42:00.170502    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cce12db065b94049db00267c524af8becc40b545dca5b23f61f45de44e324684"} err="failed to get container status \"cce12db065b94049db00267c524af8becc40b545dca5b23f61f45de44e324684\": rpc error: code = NotFound desc = could not find container \"cce12db065b94049db00267c524af8becc40b545dca5b23f61f45de44e324684\": container with ID starting with cce12db065b94049db00267c524af8becc40b545dca5b23f61f45de44e324684 not found: ID does not exist"
Oct 30 19:42:00.245133 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:42:00.245044    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jhhb2\" (UniqueName: \"kubernetes.io/projected/50a86a24-23eb-4bd4-abbd-9327c64cc99a-kube-api-access-jhhb2\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:42:00.298613 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:42:00.298558    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w0-19f6ea-55f66dc48d-jgnn8"]
Oct 30 19:42:00.303003 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:42:00.302972    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w0-19f6ea-55f66dc48d-jgnn8"]
Oct 30 19:42:01.892418 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:42:01.892385    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="50a86a24-23eb-4bd4-abbd-9327c64cc99a" path="/var/lib/kubelet/pods/50a86a24-23eb-4bd4-abbd-9327c64cc99a/volumes"
Oct 30 19:42:03.095863 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:42:03.095824    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-rjmdz"]
Oct 30 19:42:04.300442 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:42:04.300399    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w2-4a7a52/premium-w2-4a7a52-kserve-746d9579f6-hn7hl" podUID="3160b54f-1df9-48de-80d9-8d8a91d47f56" containerName="main" containerID="cri-o://4d848c8bd5b08337a6dd1112a40e2bde01626862f7ed0a59e6f00bcd0353203d" gracePeriod=45
Oct 30 19:42:04.575210 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:42:04.575185    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-4a7a52/premium-w2-4a7a52-kserve-746d9579f6-hn7hl"
Oct 30 19:42:04.704768 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:42:04.704730    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3160b54f-1df9-48de-80d9-8d8a91d47f56-dshm\" (UniqueName: \"kubernetes.io/empty-dir/3160b54f-1df9-48de-80d9-8d8a91d47f56-dshm\") pod \"3160b54f-1df9-48de-80d9-8d8a91d47f56\" (UID: \"3160b54f-1df9-48de-80d9-8d8a91d47f56\") "
Oct 30 19:42:04.704965 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:42:04.704786    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3160b54f-1df9-48de-80d9-8d8a91d47f56-kube-api-access-rdnzg\" (UniqueName: \"kubernetes.io/projected/3160b54f-1df9-48de-80d9-8d8a91d47f56-kube-api-access-rdnzg\") pod \"3160b54f-1df9-48de-80d9-8d8a91d47f56\" (UID: \"3160b54f-1df9-48de-80d9-8d8a91d47f56\") "
Oct 30 19:42:04.704965 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:42:04.704835    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3160b54f-1df9-48de-80d9-8d8a91d47f56-tls-certs\" (UniqueName: \"kubernetes.io/secret/3160b54f-1df9-48de-80d9-8d8a91d47f56-tls-certs\") pod \"3160b54f-1df9-48de-80d9-8d8a91d47f56\" (UID: \"3160b54f-1df9-48de-80d9-8d8a91d47f56\") "
Oct 30 19:42:04.704965 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:42:04.704865    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3160b54f-1df9-48de-80d9-8d8a91d47f56-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3160b54f-1df9-48de-80d9-8d8a91d47f56-tmp-dir\") pod \"3160b54f-1df9-48de-80d9-8d8a91d47f56\" (UID: \"3160b54f-1df9-48de-80d9-8d8a91d47f56\") "
Oct 30 19:42:04.704965 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:42:04.704958    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3160b54f-1df9-48de-80d9-8d8a91d47f56-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3160b54f-1df9-48de-80d9-8d8a91d47f56-model-cache\") pod \"3160b54f-1df9-48de-80d9-8d8a91d47f56\" (UID: \"3160b54f-1df9-48de-80d9-8d8a91d47f56\") "
Oct 30 19:42:04.705236 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:42:04.705032    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3160b54f-1df9-48de-80d9-8d8a91d47f56-home\" (UniqueName: \"kubernetes.io/empty-dir/3160b54f-1df9-48de-80d9-8d8a91d47f56-home\") pod \"3160b54f-1df9-48de-80d9-8d8a91d47f56\" (UID: \"3160b54f-1df9-48de-80d9-8d8a91d47f56\") "
Oct 30 19:42:04.705754 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:42:04.705711    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3160b54f-1df9-48de-80d9-8d8a91d47f56-home" pod "3160b54f-1df9-48de-80d9-8d8a91d47f56" (UID: "3160b54f-1df9-48de-80d9-8d8a91d47f56"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:42:04.705754 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:42:04.705727    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3160b54f-1df9-48de-80d9-8d8a91d47f56-tmp-dir" pod "3160b54f-1df9-48de-80d9-8d8a91d47f56" (UID: "3160b54f-1df9-48de-80d9-8d8a91d47f56"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:42:04.705754 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:42:04.705743    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3160b54f-1df9-48de-80d9-8d8a91d47f56-model-cache" pod "3160b54f-1df9-48de-80d9-8d8a91d47f56" (UID: "3160b54f-1df9-48de-80d9-8d8a91d47f56"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:42:04.708703 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:42:04.708674    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3160b54f-1df9-48de-80d9-8d8a91d47f56-kube-api-access-rdnzg" pod "3160b54f-1df9-48de-80d9-8d8a91d47f56" (UID: "3160b54f-1df9-48de-80d9-8d8a91d47f56"). InnerVolumeSpecName "kube-api-access-rdnzg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:42:04.708835 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:42:04.708752    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3160b54f-1df9-48de-80d9-8d8a91d47f56-tls-certs" pod "3160b54f-1df9-48de-80d9-8d8a91d47f56" (UID: "3160b54f-1df9-48de-80d9-8d8a91d47f56"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:42:04.709840 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:42:04.709814    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3160b54f-1df9-48de-80d9-8d8a91d47f56-dshm" pod "3160b54f-1df9-48de-80d9-8d8a91d47f56" (UID: "3160b54f-1df9-48de-80d9-8d8a91d47f56"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:42:04.806156 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:42:04.806110    4888 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3160b54f-1df9-48de-80d9-8d8a91d47f56-dshm\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:42:04.806156 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:42:04.806147    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rdnzg\" (UniqueName: \"kubernetes.io/projected/3160b54f-1df9-48de-80d9-8d8a91d47f56-kube-api-access-rdnzg\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:42:04.806156 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:42:04.806163    4888 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3160b54f-1df9-48de-80d9-8d8a91d47f56-tls-certs\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:42:04.806508 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:42:04.806179    4888 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3160b54f-1df9-48de-80d9-8d8a91d47f56-tmp-dir\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:42:04.806508 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:42:04.806192    4888 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3160b54f-1df9-48de-80d9-8d8a91d47f56-model-cache\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:42:04.806508 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:42:04.806206    4888 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3160b54f-1df9-48de-80d9-8d8a91d47f56-home\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:42:05.185418 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:42:05.185390    4888 generic.go:356] "Generic (PLEG): container finished" podID="3160b54f-1df9-48de-80d9-8d8a91d47f56" containerID="4d848c8bd5b08337a6dd1112a40e2bde01626862f7ed0a59e6f00bcd0353203d" exitCode=0
Oct 30 19:42:05.185601 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:42:05.185443    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-4a7a52/premium-w2-4a7a52-kserve-746d9579f6-hn7hl"
Oct 30 19:42:05.185601 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:42:05.185450    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-4a7a52/premium-w2-4a7a52-kserve-746d9579f6-hn7hl" event={"ID":"3160b54f-1df9-48de-80d9-8d8a91d47f56","Type":"ContainerDied","Data":"4d848c8bd5b08337a6dd1112a40e2bde01626862f7ed0a59e6f00bcd0353203d"}
Oct 30 19:42:05.185601 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:42:05.185568    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-4a7a52/premium-w2-4a7a52-kserve-746d9579f6-hn7hl" event={"ID":"3160b54f-1df9-48de-80d9-8d8a91d47f56","Type":"ContainerDied","Data":"3af21bf02037d46de9ad04acffabe4ae29a165ac4755545840fd1b105d3c8572"}
Oct 30 19:42:05.185601 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:42:05.185598    4888 scope.go:122] "RemoveContainer" containerID="4d848c8bd5b08337a6dd1112a40e2bde01626862f7ed0a59e6f00bcd0353203d"
Oct 30 19:42:05.196558 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:42:05.196534    4888 scope.go:122] "RemoveContainer" containerID="4d848c8bd5b08337a6dd1112a40e2bde01626862f7ed0a59e6f00bcd0353203d"
Oct 30 19:42:05.198586 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:42:05.198507    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4d848c8bd5b08337a6dd1112a40e2bde01626862f7ed0a59e6f00bcd0353203d\": container with ID starting with 4d848c8bd5b08337a6dd1112a40e2bde01626862f7ed0a59e6f00bcd0353203d not found: ID does not exist" containerID="4d848c8bd5b08337a6dd1112a40e2bde01626862f7ed0a59e6f00bcd0353203d"
Oct 30 19:42:05.198586 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:42:05.198558    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4d848c8bd5b08337a6dd1112a40e2bde01626862f7ed0a59e6f00bcd0353203d"} err="failed to get container status \"4d848c8bd5b08337a6dd1112a40e2bde01626862f7ed0a59e6f00bcd0353203d\": rpc error: code = NotFound desc = could not find container \"4d848c8bd5b08337a6dd1112a40e2bde01626862f7ed0a59e6f00bcd0353203d\": container with ID starting with 4d848c8bd5b08337a6dd1112a40e2bde01626862f7ed0a59e6f00bcd0353203d not found: ID does not exist"
Oct 30 19:42:05.354103 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:42:05.354031    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-4a7a52/premium-w2-4a7a52-kserve-746d9579f6-hn7hl"]
Oct 30 19:42:05.405086 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:42:05.405050    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w2-4a7a52/premium-w2-4a7a52-kserve-746d9579f6-hn7hl"]
Oct 30 19:42:05.622190 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:42:05.622152    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-jbsw6" containerName="extractor"
Oct 30 19:42:05.890809 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:42:05.890716    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3160b54f-1df9-48de-80d9-8d8a91d47f56" path="/var/lib/kubelet/pods/3160b54f-1df9-48de-80d9-8d8a91d47f56/volumes"
Oct 30 19:42:33.760992 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:42:33.760960    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-a56c7c-696c9f7cd-qm7sl"]
Oct 30 19:42:33.761492 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:42:33.761170    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-a56c7c-696c9f7cd-qm7sl" podUID="05eefdb3-fc35-4d7a-b3be-20819a7b7aaf" containerName="maas-api" containerID="cri-o://aece5e329f9346f51fe375eecf4511e8931e00f5723e2524317d23fa9185cc19" gracePeriod=30
Oct 30 19:42:34.045503 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:42:34.045470    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-a56c7c-696c9f7cd-qm7sl"
Oct 30 19:42:34.154558 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:42:34.154516    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/05eefdb3-fc35-4d7a-b3be-20819a7b7aaf-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/05eefdb3-fc35-4d7a-b3be-20819a7b7aaf-openshift-service-ca\") pod \"05eefdb3-fc35-4d7a-b3be-20819a7b7aaf\" (UID: \"05eefdb3-fc35-4d7a-b3be-20819a7b7aaf\") "
Oct 30 19:42:34.154771 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:42:34.154576    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/05eefdb3-fc35-4d7a-b3be-20819a7b7aaf-kube-api-access-kjrfz\" (UniqueName: \"kubernetes.io/projected/05eefdb3-fc35-4d7a-b3be-20819a7b7aaf-kube-api-access-kjrfz\") pod \"05eefdb3-fc35-4d7a-b3be-20819a7b7aaf\" (UID: \"05eefdb3-fc35-4d7a-b3be-20819a7b7aaf\") "
Oct 30 19:42:34.154771 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:42:34.154682    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/05eefdb3-fc35-4d7a-b3be-20819a7b7aaf-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/05eefdb3-fc35-4d7a-b3be-20819a7b7aaf-maas-api-tls\") pod \"05eefdb3-fc35-4d7a-b3be-20819a7b7aaf\" (UID: \"05eefdb3-fc35-4d7a-b3be-20819a7b7aaf\") "
Oct 30 19:42:34.154771 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:42:34.154724    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/05eefdb3-fc35-4d7a-b3be-20819a7b7aaf-metrics-certs\" (UniqueName: \"kubernetes.io/secret/05eefdb3-fc35-4d7a-b3be-20819a7b7aaf-metrics-certs\") pod \"05eefdb3-fc35-4d7a-b3be-20819a7b7aaf\" (UID: \"05eefdb3-fc35-4d7a-b3be-20819a7b7aaf\") "
Oct 30 19:42:34.154956 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:42:34.154921    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/05eefdb3-fc35-4d7a-b3be-20819a7b7aaf-openshift-service-ca" pod "05eefdb3-fc35-4d7a-b3be-20819a7b7aaf" (UID: "05eefdb3-fc35-4d7a-b3be-20819a7b7aaf"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:42:34.155134 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:42:34.155095    4888 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/05eefdb3-fc35-4d7a-b3be-20819a7b7aaf-openshift-service-ca\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:42:34.156999 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:42:34.156968    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/05eefdb3-fc35-4d7a-b3be-20819a7b7aaf-kube-api-access-kjrfz" pod "05eefdb3-fc35-4d7a-b3be-20819a7b7aaf" (UID: "05eefdb3-fc35-4d7a-b3be-20819a7b7aaf"). InnerVolumeSpecName "kube-api-access-kjrfz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:42:34.156999 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:42:34.156973    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/05eefdb3-fc35-4d7a-b3be-20819a7b7aaf-metrics-certs" pod "05eefdb3-fc35-4d7a-b3be-20819a7b7aaf" (UID: "05eefdb3-fc35-4d7a-b3be-20819a7b7aaf"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:42:34.157160 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:42:34.157039    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/05eefdb3-fc35-4d7a-b3be-20819a7b7aaf-maas-api-tls" pod "05eefdb3-fc35-4d7a-b3be-20819a7b7aaf" (UID: "05eefdb3-fc35-4d7a-b3be-20819a7b7aaf"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:42:34.256051 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:42:34.256020    4888 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/05eefdb3-fc35-4d7a-b3be-20819a7b7aaf-metrics-certs\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:42:34.256051 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:42:34.256047    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kjrfz\" (UniqueName: \"kubernetes.io/projected/05eefdb3-fc35-4d7a-b3be-20819a7b7aaf-kube-api-access-kjrfz\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:42:34.256051 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:42:34.256056    4888 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/05eefdb3-fc35-4d7a-b3be-20819a7b7aaf-maas-api-tls\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:42:34.410633 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:42:34.410598    4888 generic.go:356] "Generic (PLEG): container finished" podID="05eefdb3-fc35-4d7a-b3be-20819a7b7aaf" containerID="aece5e329f9346f51fe375eecf4511e8931e00f5723e2524317d23fa9185cc19" exitCode=0
Oct 30 19:42:34.410853 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:42:34.410644    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-a56c7c-696c9f7cd-qm7sl"
Oct 30 19:42:34.410853 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:42:34.410675    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-a56c7c-696c9f7cd-qm7sl" event={"ID":"05eefdb3-fc35-4d7a-b3be-20819a7b7aaf","Type":"ContainerDied","Data":"aece5e329f9346f51fe375eecf4511e8931e00f5723e2524317d23fa9185cc19"}
Oct 30 19:42:34.410853 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:42:34.410740    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-a56c7c-696c9f7cd-qm7sl" event={"ID":"05eefdb3-fc35-4d7a-b3be-20819a7b7aaf","Type":"ContainerDied","Data":"93d10e6e28957c9a409fdd56a1fdbdaca846eadadbc4ab4e6a1ec2ad286cddaa"}
Oct 30 19:42:34.410853 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:42:34.410770    4888 scope.go:122] "RemoveContainer" containerID="aece5e329f9346f51fe375eecf4511e8931e00f5723e2524317d23fa9185cc19"
Oct 30 19:42:34.423098 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:42:34.423072    4888 scope.go:122] "RemoveContainer" containerID="aece5e329f9346f51fe375eecf4511e8931e00f5723e2524317d23fa9185cc19"
Oct 30 19:42:34.423451 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:42:34.423433    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aece5e329f9346f51fe375eecf4511e8931e00f5723e2524317d23fa9185cc19\": container with ID starting with aece5e329f9346f51fe375eecf4511e8931e00f5723e2524317d23fa9185cc19 not found: ID does not exist" containerID="aece5e329f9346f51fe375eecf4511e8931e00f5723e2524317d23fa9185cc19"
Oct 30 19:42:34.423518 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:42:34.423460    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aece5e329f9346f51fe375eecf4511e8931e00f5723e2524317d23fa9185cc19"} err="failed to get container status \"aece5e329f9346f51fe375eecf4511e8931e00f5723e2524317d23fa9185cc19\": rpc error: code = NotFound desc = could not find container \"aece5e329f9346f51fe375eecf4511e8931e00f5723e2524317d23fa9185cc19\": container with ID starting with aece5e329f9346f51fe375eecf4511e8931e00f5723e2524317d23fa9185cc19 not found: ID does not exist"
Oct 30 19:42:34.579519 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:42:34.579485    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-a56c7c-696c9f7cd-qm7sl"]
Oct 30 19:42:34.630261 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:42:34.630227    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-a56c7c-696c9f7cd-qm7sl"]
Oct 30 19:42:35.621049 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:42:35.621006    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-jbsw6" containerName="extractor"
Oct 30 19:42:35.893008 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:42:35.892920    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="05eefdb3-fc35-4d7a-b3be-20819a7b7aaf" path="/var/lib/kubelet/pods/05eefdb3-fc35-4d7a-b3be-20819a7b7aaf/volumes"
Oct 30 19:42:39.816610 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:42:39.816572    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-rjmdz"]
Oct 30 19:42:44.320378 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:42:44.320341    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w0-a56c7c-openshift-default-5db77d74cf-jjcdj"]
Oct 30 19:42:44.320807 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:42:44.320559    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w0-a56c7c-openshift-default-5db77d74cf-jjcdj" podUID="36712903-e111-4fb5-a600-9b666b1d1c9f" containerName="istio-proxy" containerID="cri-o://625bdab96a1a7504f8d4687c94b34e97ea397a73432468ad06cdfcfcc69fddfe" gracePeriod=30
Oct 30 19:42:44.697314 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:42:44.697252    4888 patch_prober.go:29] interesting pod/e2e-shared-b-w0-a56c7c-openshift-default-5db77d74cf-jjcdj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 19:42:44.697583 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:42:44.697364    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-w0-a56c7c-openshift-default-5db77d74cf-jjcdj" podUID="36712903-e111-4fb5-a600-9b666b1d1c9f" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 19:42:49.540474 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:42:49.540438    4888 generic.go:356] "Generic (PLEG): container finished" podID="36712903-e111-4fb5-a600-9b666b1d1c9f" containerID="625bdab96a1a7504f8d4687c94b34e97ea397a73432468ad06cdfcfcc69fddfe" exitCode=0
Oct 30 19:42:49.540936 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:42:49.540513    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w0-a56c7c-openshift-default-5db77d74cf-jjcdj" event={"ID":"36712903-e111-4fb5-a600-9b666b1d1c9f","Type":"ContainerDied","Data":"625bdab96a1a7504f8d4687c94b34e97ea397a73432468ad06cdfcfcc69fddfe"}
Oct 30 19:42:49.540936 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:42:49.540591    4888 scope.go:122] "RemoveContainer" containerID="3b56a8d77360e89020665adaeadd010c41bb43e32029d47ffb874b68bb7feb18"
Oct 30 19:42:49.596010 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:42:49.595944    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w0-a56c7c-openshift-default-5db77d74cf-jjcdj"
Oct 30 19:42:49.683111 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:42:49.683081    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/36712903-e111-4fb5-a600-9b666b1d1c9f-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/36712903-e111-4fb5-a600-9b666b1d1c9f-workload-socket\") pod \"36712903-e111-4fb5-a600-9b666b1d1c9f\" (UID: \"36712903-e111-4fb5-a600-9b666b1d1c9f\") "
Oct 30 19:42:49.683111 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:42:49.683116    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/36712903-e111-4fb5-a600-9b666b1d1c9f-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/36712903-e111-4fb5-a600-9b666b1d1c9f-workload-certs\") pod \"36712903-e111-4fb5-a600-9b666b1d1c9f\" (UID: \"36712903-e111-4fb5-a600-9b666b1d1c9f\") "
Oct 30 19:42:49.683417 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:42:49.683151    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/36712903-e111-4fb5-a600-9b666b1d1c9f-kube-api-access-psgqh\" (UniqueName: \"kubernetes.io/projected/36712903-e111-4fb5-a600-9b666b1d1c9f-kube-api-access-psgqh\") pod \"36712903-e111-4fb5-a600-9b666b1d1c9f\" (UID: \"36712903-e111-4fb5-a600-9b666b1d1c9f\") "
Oct 30 19:42:49.683417 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:42:49.683200    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/36712903-e111-4fb5-a600-9b666b1d1c9f-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/36712903-e111-4fb5-a600-9b666b1d1c9f-credential-socket\") pod \"36712903-e111-4fb5-a600-9b666b1d1c9f\" (UID: \"36712903-e111-4fb5-a600-9b666b1d1c9f\") "
Oct 30 19:42:49.683417 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:42:49.683270    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/36712903-e111-4fb5-a600-9b666b1d1c9f-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/36712903-e111-4fb5-a600-9b666b1d1c9f-istio-envoy\") pod \"36712903-e111-4fb5-a600-9b666b1d1c9f\" (UID: \"36712903-e111-4fb5-a600-9b666b1d1c9f\") "
Oct 30 19:42:49.683417 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:42:49.683333    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/36712903-e111-4fb5-a600-9b666b1d1c9f-istio-token\" (UniqueName: \"kubernetes.io/projected/36712903-e111-4fb5-a600-9b666b1d1c9f-istio-token\") pod \"36712903-e111-4fb5-a600-9b666b1d1c9f\" (UID: \"36712903-e111-4fb5-a600-9b666b1d1c9f\") "
Oct 30 19:42:49.683417 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:42:49.683380    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/36712903-e111-4fb5-a600-9b666b1d1c9f-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/36712903-e111-4fb5-a600-9b666b1d1c9f-istiod-ca-cert\") pod \"36712903-e111-4fb5-a600-9b666b1d1c9f\" (UID: \"36712903-e111-4fb5-a600-9b666b1d1c9f\") "
Oct 30 19:42:49.683705 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:42:49.683423    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/36712903-e111-4fb5-a600-9b666b1d1c9f-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/36712903-e111-4fb5-a600-9b666b1d1c9f-istio-podinfo\") pod \"36712903-e111-4fb5-a600-9b666b1d1c9f\" (UID: \"36712903-e111-4fb5-a600-9b666b1d1c9f\") "
Oct 30 19:42:49.683705 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:42:49.683433    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36712903-e111-4fb5-a600-9b666b1d1c9f-workload-socket" pod "36712903-e111-4fb5-a600-9b666b1d1c9f" (UID: "36712903-e111-4fb5-a600-9b666b1d1c9f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:42:49.683705 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:42:49.683504    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/36712903-e111-4fb5-a600-9b666b1d1c9f-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/36712903-e111-4fb5-a600-9b666b1d1c9f-istio-data\") pod \"36712903-e111-4fb5-a600-9b666b1d1c9f\" (UID: \"36712903-e111-4fb5-a600-9b666b1d1c9f\") "
Oct 30 19:42:49.683705 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:42:49.683547    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36712903-e111-4fb5-a600-9b666b1d1c9f-workload-certs" pod "36712903-e111-4fb5-a600-9b666b1d1c9f" (UID: "36712903-e111-4fb5-a600-9b666b1d1c9f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:42:49.683935 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:42:49.683858    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36712903-e111-4fb5-a600-9b666b1d1c9f-istio-data" pod "36712903-e111-4fb5-a600-9b666b1d1c9f" (UID: "36712903-e111-4fb5-a600-9b666b1d1c9f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:42:49.683935 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:42:49.683882    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/36712903-e111-4fb5-a600-9b666b1d1c9f-istiod-ca-cert" pod "36712903-e111-4fb5-a600-9b666b1d1c9f" (UID: "36712903-e111-4fb5-a600-9b666b1d1c9f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:42:49.684085 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:42:49.683956    4888 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/36712903-e111-4fb5-a600-9b666b1d1c9f-istio-data\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:42:49.684158 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:42:49.684089    4888 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/36712903-e111-4fb5-a600-9b666b1d1c9f-workload-socket\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:42:49.684158 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:42:49.684106    4888 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/36712903-e111-4fb5-a600-9b666b1d1c9f-workload-certs\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:42:49.684321 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:42:49.684274    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36712903-e111-4fb5-a600-9b666b1d1c9f-credential-socket" pod "36712903-e111-4fb5-a600-9b666b1d1c9f" (UID: "36712903-e111-4fb5-a600-9b666b1d1c9f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:42:49.686050 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:42:49.686025    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36712903-e111-4fb5-a600-9b666b1d1c9f-istio-envoy" pod "36712903-e111-4fb5-a600-9b666b1d1c9f" (UID: "36712903-e111-4fb5-a600-9b666b1d1c9f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:42:49.686775 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:42:49.686748    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/36712903-e111-4fb5-a600-9b666b1d1c9f-istio-podinfo" pod "36712903-e111-4fb5-a600-9b666b1d1c9f" (UID: "36712903-e111-4fb5-a600-9b666b1d1c9f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:42:49.686929 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:42:49.686906    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/36712903-e111-4fb5-a600-9b666b1d1c9f-kube-api-access-psgqh" pod "36712903-e111-4fb5-a600-9b666b1d1c9f" (UID: "36712903-e111-4fb5-a600-9b666b1d1c9f"). InnerVolumeSpecName "kube-api-access-psgqh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:42:49.686996 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:42:49.686943    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/36712903-e111-4fb5-a600-9b666b1d1c9f-istio-token" pod "36712903-e111-4fb5-a600-9b666b1d1c9f" (UID: "36712903-e111-4fb5-a600-9b666b1d1c9f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:42:49.785743 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:42:49.785711    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-psgqh\" (UniqueName: \"kubernetes.io/projected/36712903-e111-4fb5-a600-9b666b1d1c9f-kube-api-access-psgqh\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:42:49.785743 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:42:49.785738    4888 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/36712903-e111-4fb5-a600-9b666b1d1c9f-credential-socket\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:42:49.785743 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:42:49.785748    4888 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/36712903-e111-4fb5-a600-9b666b1d1c9f-istio-envoy\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:42:49.786085 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:42:49.785757    4888 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/36712903-e111-4fb5-a600-9b666b1d1c9f-istio-token\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:42:49.786085 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:42:49.785766    4888 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/36712903-e111-4fb5-a600-9b666b1d1c9f-istiod-ca-cert\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:42:49.786085 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:42:49.785774    4888 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/36712903-e111-4fb5-a600-9b666b1d1c9f-istio-podinfo\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:42:50.548142 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:42:50.548108    4888 generic.go:356] "Generic (PLEG): container finished" podID="242ace3b-f2c6-4fd5-abe5-31ce3ae9a12c" containerID="296603c2add4596d5de87bdb74d1cb67d542d22cc7ec6581922a41b57a855676" exitCode=0
Oct 30 19:42:50.548670 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:42:50.548184    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-5da1ae-openshift-default-77c657bb8f-nng6m" event={"ID":"242ace3b-f2c6-4fd5-abe5-31ce3ae9a12c","Type":"ContainerDied","Data":"296603c2add4596d5de87bdb74d1cb67d542d22cc7ec6581922a41b57a855676"}
Oct 30 19:42:50.548670 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:42:50.548649    4888 scope.go:122] "RemoveContainer" containerID="296603c2add4596d5de87bdb74d1cb67d542d22cc7ec6581922a41b57a855676"
Oct 30 19:42:50.549860 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:42:50.549830    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w0-a56c7c-openshift-default-5db77d74cf-jjcdj" event={"ID":"36712903-e111-4fb5-a600-9b666b1d1c9f","Type":"ContainerDied","Data":"ccf737881f458817b3dc5efb3c8992192cd2b78aaac0bb5389d0110199f08bf0"}
Oct 30 19:42:50.549939 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:42:50.549890    4888 scope.go:122] "RemoveContainer" containerID="625bdab96a1a7504f8d4687c94b34e97ea397a73432468ad06cdfcfcc69fddfe"
Oct 30 19:42:50.549939 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:42:50.549845    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w0-a56c7c-openshift-default-5db77d74cf-jjcdj"
Oct 30 19:42:50.550142 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:42:50.550114    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:42:50.550256 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:42:50.550217    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:42:50.550359 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:42:50.550266    4888 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892004Ki","pods":"250"}
Oct 30 19:42:50.838236 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:42:50.838200    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w0-a56c7c-openshift-default-5db77d74cf-jjcdj"]
Oct 30 19:42:50.842870 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:42:50.842841    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w0-a56c7c-openshift-default-5db77d74cf-jjcdj"]
Oct 30 19:42:51.559753 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:42:51.559715    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-5da1ae-openshift-default-77c657bb8f-nng6m" event={"ID":"242ace3b-f2c6-4fd5-abe5-31ce3ae9a12c","Type":"ContainerStarted","Data":"83b2c2def8c2419f26db112578fc3501a1f8ab0e41fecc3728aceff3cf05e7ae"}
Oct 30 19:42:51.900966 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:42:51.900927    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="36712903-e111-4fb5-a600-9b666b1d1c9f" path="/var/lib/kubelet/pods/36712903-e111-4fb5-a600-9b666b1d1c9f/volumes"
Oct 30 19:42:52.516523 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:42:52.516490    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-extmod-b-5da1ae-openshift-default-77c657bb8f-nng6m"
Oct 30 19:42:52.517929 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:42:52.517901    4888 patch_prober.go:29] interesting pod/e2e-extmod-b-5da1ae-openshift-default-77c657bb8f-nng6m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.181:15021/healthz/ready\": dial tcp 10.132.0.181:15021: connect: connection refused" start-of-body=
Oct 30 19:42:52.518067 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:42:52.517959    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-5da1ae-openshift-default-77c657bb8f-nng6m" podUID="242ace3b-f2c6-4fd5-abe5-31ce3ae9a12c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.181:15021/healthz/ready\": dial tcp 10.132.0.181:15021: connect: connection refused"
Oct 30 19:42:53.517224 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:42:53.517187    4888 patch_prober.go:29] interesting pod/e2e-extmod-b-5da1ae-openshift-default-77c657bb8f-nng6m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.181:15021/healthz/ready\": dial tcp 10.132.0.181:15021: connect: connection refused" start-of-body=
Oct 30 19:42:53.517766 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:42:53.517244    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-5da1ae-openshift-default-77c657bb8f-nng6m" podUID="242ace3b-f2c6-4fd5-abe5-31ce3ae9a12c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.181:15021/healthz/ready\": dial tcp 10.132.0.181:15021: connect: connection refused"
Oct 30 19:42:54.516615 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:42:54.516580    4888 patch_prober.go:29] interesting pod/e2e-extmod-b-5da1ae-openshift-default-77c657bb8f-nng6m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.181:15021/healthz/ready\": dial tcp 10.132.0.181:15021: connect: connection refused" start-of-body=
Oct 30 19:42:54.516806 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:42:54.516634    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-5da1ae-openshift-default-77c657bb8f-nng6m" podUID="242ace3b-f2c6-4fd5-abe5-31ce3ae9a12c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.181:15021/healthz/ready\": dial tcp 10.132.0.181:15021: connect: connection refused"
Oct 30 19:42:55.517492 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:42:55.517454    4888 patch_prober.go:29] interesting pod/e2e-extmod-b-5da1ae-openshift-default-77c657bb8f-nng6m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.181:15021/healthz/ready\": dial tcp 10.132.0.181:15021: connect: connection refused" start-of-body=
Oct 30 19:42:55.517936 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:42:55.517511    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-5da1ae-openshift-default-77c657bb8f-nng6m" podUID="242ace3b-f2c6-4fd5-abe5-31ce3ae9a12c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.181:15021/healthz/ready\": dial tcp 10.132.0.181:15021: connect: connection refused"
Oct 30 19:42:56.516660 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:42:56.516623    4888 patch_prober.go:29] interesting pod/e2e-extmod-b-5da1ae-openshift-default-77c657bb8f-nng6m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.181:15021/healthz/ready\": dial tcp 10.132.0.181:15021: connect: connection refused" start-of-body=
Oct 30 19:42:56.516859 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:42:56.516676    4888 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-5da1ae-openshift-default-77c657bb8f-nng6m" podUID="242ace3b-f2c6-4fd5-abe5-31ce3ae9a12c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.181:15021/healthz/ready\": dial tcp 10.132.0.181:15021: connect: connection refused"
Oct 30 19:42:57.007589 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:42:57.007560    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-extmod-b-5da1ae-openshift-default-77c657bb8f-nng6m"
Oct 30 19:42:57.035543 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:42:57.035509    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-2ade09-867957c796-stmnm"]
Oct 30 19:42:57.176813 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:42:57.176784    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-2ade09-85459998c9-cksnq"]
Oct 30 19:42:57.521992 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:42:57.521961    4888 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-extmod-b-5da1ae-openshift-default-77c657bb8f-nng6m"
Oct 30 19:42:57.625341 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:42:57.625315    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-extmod-b-5da1ae-openshift-default-77c657bb8f-nng6m"
Oct 30 19:43:02.036139 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:02.036097    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w0-2ade09-867957c796-stmnm" podUID="771fbee7-b699-4a2b-a2cd-a2f78bc8e198" containerName="payload-processing" containerID="cri-o://57f9536cea451c7dce5cf63b5237a082999a01f7b392219964ffdd7b9a156620" gracePeriod=25
Oct 30 19:43:02.177094 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:02.177023    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-2ade09-85459998c9-cksnq" podUID="e7d8237d-903d-4136-8e74-b90346f5f619" containerName="payload-pre-processing" containerID="cri-o://1d49bc4b6f14c5398972d57b8d01621d3e0c5c7cb11d650bb459c60fd18eb8b4" gracePeriod=25
Oct 30 19:43:05.618752 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:43:05.618713    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-jbsw6" containerName="extractor"
Oct 30 19:43:07.270655 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:07.270617    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w0-2ade09-867957c796-stmnm" podUID="771fbee7-b699-4a2b-a2cd-a2f78bc8e198" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:43:07.271123 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:07.270621    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-2ade09-85459998c9-cksnq" podUID="e7d8237d-903d-4136-8e74-b90346f5f619" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:43:11.227186 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:11.227146    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-cgw6d"]
Oct 30 19:43:11.227696 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:11.227414    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-cgw6d" podUID="bae588a3-e65e-4f9b-b4c4-615aa878741e" containerName="istio-proxy" containerID="cri-o://035c09ad84c4ee952cd6931ec51f499f1ed45e4ba21766a9506710ac92814e57" gracePeriod=30
Oct 30 19:43:16.484972 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:16.484940    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-cgw6d"
Oct 30 19:43:16.595900 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:16.595861    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bae588a3-e65e-4f9b-b4c4-615aa878741e-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bae588a3-e65e-4f9b-b4c4-615aa878741e-workload-certs\") pod \"bae588a3-e65e-4f9b-b4c4-615aa878741e\" (UID: \"bae588a3-e65e-4f9b-b4c4-615aa878741e\") "
Oct 30 19:43:16.596111 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:16.595911    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bae588a3-e65e-4f9b-b4c4-615aa878741e-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bae588a3-e65e-4f9b-b4c4-615aa878741e-workload-socket\") pod \"bae588a3-e65e-4f9b-b4c4-615aa878741e\" (UID: \"bae588a3-e65e-4f9b-b4c4-615aa878741e\") "
Oct 30 19:43:16.596111 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:16.595942    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bae588a3-e65e-4f9b-b4c4-615aa878741e-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bae588a3-e65e-4f9b-b4c4-615aa878741e-istio-envoy\") pod \"bae588a3-e65e-4f9b-b4c4-615aa878741e\" (UID: \"bae588a3-e65e-4f9b-b4c4-615aa878741e\") "
Oct 30 19:43:16.596111 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:16.596003    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/bae588a3-e65e-4f9b-b4c4-615aa878741e-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bae588a3-e65e-4f9b-b4c4-615aa878741e-istiod-ca-cert\") pod \"bae588a3-e65e-4f9b-b4c4-615aa878741e\" (UID: \"bae588a3-e65e-4f9b-b4c4-615aa878741e\") "
Oct 30 19:43:16.596111 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:16.596050    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bae588a3-e65e-4f9b-b4c4-615aa878741e-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bae588a3-e65e-4f9b-b4c4-615aa878741e-credential-socket\") pod \"bae588a3-e65e-4f9b-b4c4-615aa878741e\" (UID: \"bae588a3-e65e-4f9b-b4c4-615aa878741e\") "
Oct 30 19:43:16.596388 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:16.596109    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/bae588a3-e65e-4f9b-b4c4-615aa878741e-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bae588a3-e65e-4f9b-b4c4-615aa878741e-istio-podinfo\") pod \"bae588a3-e65e-4f9b-b4c4-615aa878741e\" (UID: \"bae588a3-e65e-4f9b-b4c4-615aa878741e\") "
Oct 30 19:43:16.596388 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:16.596166    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bae588a3-e65e-4f9b-b4c4-615aa878741e-istio-token\" (UniqueName: \"kubernetes.io/projected/bae588a3-e65e-4f9b-b4c4-615aa878741e-istio-token\") pod \"bae588a3-e65e-4f9b-b4c4-615aa878741e\" (UID: \"bae588a3-e65e-4f9b-b4c4-615aa878741e\") "
Oct 30 19:43:16.596388 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:16.596201    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bae588a3-e65e-4f9b-b4c4-615aa878741e-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bae588a3-e65e-4f9b-b4c4-615aa878741e-istio-data\") pod \"bae588a3-e65e-4f9b-b4c4-615aa878741e\" (UID: \"bae588a3-e65e-4f9b-b4c4-615aa878741e\") "
Oct 30 19:43:16.596388 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:16.596207    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bae588a3-e65e-4f9b-b4c4-615aa878741e-workload-certs" pod "bae588a3-e65e-4f9b-b4c4-615aa878741e" (UID: "bae588a3-e65e-4f9b-b4c4-615aa878741e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:43:16.596388 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:16.596253    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bae588a3-e65e-4f9b-b4c4-615aa878741e-kube-api-access-678m5\" (UniqueName: \"kubernetes.io/projected/bae588a3-e65e-4f9b-b4c4-615aa878741e-kube-api-access-678m5\") pod \"bae588a3-e65e-4f9b-b4c4-615aa878741e\" (UID: \"bae588a3-e65e-4f9b-b4c4-615aa878741e\") "
Oct 30 19:43:16.596388 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:16.596323    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bae588a3-e65e-4f9b-b4c4-615aa878741e-workload-socket" pod "bae588a3-e65e-4f9b-b4c4-615aa878741e" (UID: "bae588a3-e65e-4f9b-b4c4-615aa878741e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:43:16.596718 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:16.596629    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bae588a3-e65e-4f9b-b4c4-615aa878741e-istio-data" pod "bae588a3-e65e-4f9b-b4c4-615aa878741e" (UID: "bae588a3-e65e-4f9b-b4c4-615aa878741e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:43:16.596816 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:16.596789    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bae588a3-e65e-4f9b-b4c4-615aa878741e-credential-socket" pod "bae588a3-e65e-4f9b-b4c4-615aa878741e" (UID: "bae588a3-e65e-4f9b-b4c4-615aa878741e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:43:16.597125 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:16.596875    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bae588a3-e65e-4f9b-b4c4-615aa878741e-istiod-ca-cert" pod "bae588a3-e65e-4f9b-b4c4-615aa878741e" (UID: "bae588a3-e65e-4f9b-b4c4-615aa878741e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:43:16.597125 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:16.597114    4888 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bae588a3-e65e-4f9b-b4c4-615aa878741e-istio-data\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:43:16.597411 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:16.597136    4888 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bae588a3-e65e-4f9b-b4c4-615aa878741e-workload-certs\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:43:16.597411 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:16.597151    4888 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bae588a3-e65e-4f9b-b4c4-615aa878741e-workload-socket\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:43:16.598768 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:16.598712    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bae588a3-e65e-4f9b-b4c4-615aa878741e-istio-token" pod "bae588a3-e65e-4f9b-b4c4-615aa878741e" (UID: "bae588a3-e65e-4f9b-b4c4-615aa878741e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:43:16.598874 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:16.598849    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bae588a3-e65e-4f9b-b4c4-615aa878741e-kube-api-access-678m5" pod "bae588a3-e65e-4f9b-b4c4-615aa878741e" (UID: "bae588a3-e65e-4f9b-b4c4-615aa878741e"). InnerVolumeSpecName "kube-api-access-678m5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:43:16.599172 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:16.599145    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/bae588a3-e65e-4f9b-b4c4-615aa878741e-istio-podinfo" pod "bae588a3-e65e-4f9b-b4c4-615aa878741e" (UID: "bae588a3-e65e-4f9b-b4c4-615aa878741e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:43:16.599561 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:16.599539    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bae588a3-e65e-4f9b-b4c4-615aa878741e-istio-envoy" pod "bae588a3-e65e-4f9b-b4c4-615aa878741e" (UID: "bae588a3-e65e-4f9b-b4c4-615aa878741e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:43:16.698207 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:16.698171    4888 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bae588a3-e65e-4f9b-b4c4-615aa878741e-istio-token\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:43:16.698207 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:16.698201    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-678m5\" (UniqueName: \"kubernetes.io/projected/bae588a3-e65e-4f9b-b4c4-615aa878741e-kube-api-access-678m5\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:43:16.698207 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:16.698212    4888 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bae588a3-e65e-4f9b-b4c4-615aa878741e-istio-envoy\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:43:16.698591 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:16.698221    4888 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bae588a3-e65e-4f9b-b4c4-615aa878741e-istiod-ca-cert\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:43:16.698591 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:16.698232    4888 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bae588a3-e65e-4f9b-b4c4-615aa878741e-credential-socket\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:43:16.698591 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:16.698240    4888 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bae588a3-e65e-4f9b-b4c4-615aa878741e-istio-podinfo\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:43:16.756319 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:16.756266    4888 generic.go:356] "Generic (PLEG): container finished" podID="bae588a3-e65e-4f9b-b4c4-615aa878741e" containerID="035c09ad84c4ee952cd6931ec51f499f1ed45e4ba21766a9506710ac92814e57" exitCode=0
Oct 30 19:43:16.756509 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:16.756351    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-cgw6d" event={"ID":"bae588a3-e65e-4f9b-b4c4-615aa878741e","Type":"ContainerDied","Data":"035c09ad84c4ee952cd6931ec51f499f1ed45e4ba21766a9506710ac92814e57"}
Oct 30 19:43:16.756509 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:16.756408    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-cgw6d" event={"ID":"bae588a3-e65e-4f9b-b4c4-615aa878741e","Type":"ContainerDied","Data":"0fb8ed749ce9dddab6ab0e08e6200f187183a23fc5fdc5442c793b20960189f3"}
Oct 30 19:43:16.756509 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:16.756428    4888 scope.go:122] "RemoveContainer" containerID="035c09ad84c4ee952cd6931ec51f499f1ed45e4ba21766a9506710ac92814e57"
Oct 30 19:43:16.756509 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:16.756357    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-cgw6d"
Oct 30 19:43:16.767204 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:16.767186    4888 scope.go:122] "RemoveContainer" containerID="7c50df43789b05948bdbc79637a365e4bc455c0277c5f7755a9d5cff7caa60f3"
Oct 30 19:43:16.777034 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:16.777017    4888 scope.go:122] "RemoveContainer" containerID="035c09ad84c4ee952cd6931ec51f499f1ed45e4ba21766a9506710ac92814e57"
Oct 30 19:43:16.777316 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:43:16.777257    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"035c09ad84c4ee952cd6931ec51f499f1ed45e4ba21766a9506710ac92814e57\": container with ID starting with 035c09ad84c4ee952cd6931ec51f499f1ed45e4ba21766a9506710ac92814e57 not found: ID does not exist" containerID="035c09ad84c4ee952cd6931ec51f499f1ed45e4ba21766a9506710ac92814e57"
Oct 30 19:43:16.777423 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:16.777311    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"035c09ad84c4ee952cd6931ec51f499f1ed45e4ba21766a9506710ac92814e57"} err="failed to get container status \"035c09ad84c4ee952cd6931ec51f499f1ed45e4ba21766a9506710ac92814e57\": rpc error: code = NotFound desc = could not find container \"035c09ad84c4ee952cd6931ec51f499f1ed45e4ba21766a9506710ac92814e57\": container with ID starting with 035c09ad84c4ee952cd6931ec51f499f1ed45e4ba21766a9506710ac92814e57 not found: ID does not exist"
Oct 30 19:43:16.777423 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:16.777330    4888 scope.go:122] "RemoveContainer" containerID="7c50df43789b05948bdbc79637a365e4bc455c0277c5f7755a9d5cff7caa60f3"
Oct 30 19:43:16.777561 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:43:16.777543    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7c50df43789b05948bdbc79637a365e4bc455c0277c5f7755a9d5cff7caa60f3\": container with ID starting with 7c50df43789b05948bdbc79637a365e4bc455c0277c5f7755a9d5cff7caa60f3 not found: ID does not exist" containerID="7c50df43789b05948bdbc79637a365e4bc455c0277c5f7755a9d5cff7caa60f3"
Oct 30 19:43:16.777626 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:16.777566    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7c50df43789b05948bdbc79637a365e4bc455c0277c5f7755a9d5cff7caa60f3"} err="failed to get container status \"7c50df43789b05948bdbc79637a365e4bc455c0277c5f7755a9d5cff7caa60f3\": rpc error: code = NotFound desc = could not find container \"7c50df43789b05948bdbc79637a365e4bc455c0277c5f7755a9d5cff7caa60f3\": container with ID starting with 7c50df43789b05948bdbc79637a365e4bc455c0277c5f7755a9d5cff7caa60f3 not found: ID does not exist"
Oct 30 19:43:16.926745 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:16.926657    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-cgw6d"]
Oct 30 19:43:16.933600 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:16.933570    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-4a7a52-openshift-default-7cfbc88b69-cgw6d"]
Oct 30 19:43:17.196721 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:17.196629    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-rjmdz"]
Oct 30 19:43:17.270557 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:17.270523    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-2ade09-85459998c9-cksnq" podUID="e7d8237d-903d-4136-8e74-b90346f5f619" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:43:17.270557 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:17.270548    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w0-2ade09-867957c796-stmnm" podUID="771fbee7-b699-4a2b-a2cd-a2f78bc8e198" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:43:17.895764 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:17.895730    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="bae588a3-e65e-4f9b-b4c4-615aa878741e" path="/var/lib/kubelet/pods/bae588a3-e65e-4f9b-b4c4-615aa878741e/volumes"
Oct 30 19:43:22.324360 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:22.324277    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-2ade09-867957c796-stmnm"
Oct 30 19:43:22.378012 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:22.377846    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/771fbee7-b699-4a2b-a2cd-a2f78bc8e198-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/771fbee7-b699-4a2b-a2cd-a2f78bc8e198-plugins-config-volume\") pod \"771fbee7-b699-4a2b-a2cd-a2f78bc8e198\" (UID: \"771fbee7-b699-4a2b-a2cd-a2f78bc8e198\") "
Oct 30 19:43:22.379918 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:22.379409    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/771fbee7-b699-4a2b-a2cd-a2f78bc8e198-kube-api-access-tzmhf\" (UniqueName: \"kubernetes.io/projected/771fbee7-b699-4a2b-a2cd-a2f78bc8e198-kube-api-access-tzmhf\") pod \"771fbee7-b699-4a2b-a2cd-a2f78bc8e198\" (UID: \"771fbee7-b699-4a2b-a2cd-a2f78bc8e198\") "
Oct 30 19:43:22.380167 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:22.380099    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/771fbee7-b699-4a2b-a2cd-a2f78bc8e198-plugins-config-volume" pod "771fbee7-b699-4a2b-a2cd-a2f78bc8e198" (UID: "771fbee7-b699-4a2b-a2cd-a2f78bc8e198"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:43:22.388937 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:22.388872    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/771fbee7-b699-4a2b-a2cd-a2f78bc8e198-kube-api-access-tzmhf" pod "771fbee7-b699-4a2b-a2cd-a2f78bc8e198" (UID: "771fbee7-b699-4a2b-a2cd-a2f78bc8e198"). InnerVolumeSpecName "kube-api-access-tzmhf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:43:22.438582 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:22.438554    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-2ade09-85459998c9-cksnq"
Oct 30 19:43:22.480814 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:22.480777    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e7d8237d-903d-4136-8e74-b90346f5f619-kube-api-access-vctb2\" (UniqueName: \"kubernetes.io/projected/e7d8237d-903d-4136-8e74-b90346f5f619-kube-api-access-vctb2\") pod \"e7d8237d-903d-4136-8e74-b90346f5f619\" (UID: \"e7d8237d-903d-4136-8e74-b90346f5f619\") "
Oct 30 19:43:22.481020 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:22.480947    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e7d8237d-903d-4136-8e74-b90346f5f619-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e7d8237d-903d-4136-8e74-b90346f5f619-plugins-config-volume\") pod \"e7d8237d-903d-4136-8e74-b90346f5f619\" (UID: \"e7d8237d-903d-4136-8e74-b90346f5f619\") "
Oct 30 19:43:22.481430 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:22.481404    4888 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/771fbee7-b699-4a2b-a2cd-a2f78bc8e198-plugins-config-volume\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:43:22.481557 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:22.481433    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tzmhf\" (UniqueName: \"kubernetes.io/projected/771fbee7-b699-4a2b-a2cd-a2f78bc8e198-kube-api-access-tzmhf\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:43:22.481631 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:22.481581    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e7d8237d-903d-4136-8e74-b90346f5f619-plugins-config-volume" pod "e7d8237d-903d-4136-8e74-b90346f5f619" (UID: "e7d8237d-903d-4136-8e74-b90346f5f619"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:43:22.484634 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:22.484358    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e7d8237d-903d-4136-8e74-b90346f5f619-kube-api-access-vctb2" pod "e7d8237d-903d-4136-8e74-b90346f5f619" (UID: "e7d8237d-903d-4136-8e74-b90346f5f619"). InnerVolumeSpecName "kube-api-access-vctb2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:43:22.504529 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:22.504492    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-fd61e3-7647c5bb9f-h4th6"]
Oct 30 19:43:22.504782 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:22.504735    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-fd61e3-7647c5bb9f-h4th6" podUID="a4fd7ad8-cdda-470a-86bf-c0c0df93fda3" containerName="maas-api" containerID="cri-o://d212141157e251729a834d30ecbf004ac79b3a9093f6d80f1391bae628e0d560" gracePeriod=30
Oct 30 19:43:22.583089 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:22.583051    4888 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e7d8237d-903d-4136-8e74-b90346f5f619-plugins-config-volume\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:43:22.583089 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:22.583092    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vctb2\" (UniqueName: \"kubernetes.io/projected/e7d8237d-903d-4136-8e74-b90346f5f619-kube-api-access-vctb2\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:43:22.614623 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:22.614572    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-fd61e3-7647c5bb9f-h4th6" podUID="a4fd7ad8-cdda-470a-86bf-c0c0df93fda3" containerName="maas-api" probeResult="failure" output="Get \"https://10.132.0.81:8443/health\": dial tcp 10.132.0.81:8443: connect: connection refused"
Oct 30 19:43:22.773247 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:22.773224    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-fd61e3-7647c5bb9f-h4th6"
Oct 30 19:43:22.800274 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:22.800240    4888 generic.go:356] "Generic (PLEG): container finished" podID="a4fd7ad8-cdda-470a-86bf-c0c0df93fda3" containerID="d212141157e251729a834d30ecbf004ac79b3a9093f6d80f1391bae628e0d560" exitCode=0
Oct 30 19:43:22.800499 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:22.800327    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-fd61e3-7647c5bb9f-h4th6"
Oct 30 19:43:22.800499 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:22.800333    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-fd61e3-7647c5bb9f-h4th6" event={"ID":"a4fd7ad8-cdda-470a-86bf-c0c0df93fda3","Type":"ContainerDied","Data":"d212141157e251729a834d30ecbf004ac79b3a9093f6d80f1391bae628e0d560"}
Oct 30 19:43:22.800499 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:22.800391    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-fd61e3-7647c5bb9f-h4th6" event={"ID":"a4fd7ad8-cdda-470a-86bf-c0c0df93fda3","Type":"ContainerDied","Data":"8e266367c9811bbf35b7c2bf773fb1ea8889b27fdce3c433f442553d16ff4e21"}
Oct 30 19:43:22.800499 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:22.800423    4888 scope.go:122] "RemoveContainer" containerID="d212141157e251729a834d30ecbf004ac79b3a9093f6d80f1391bae628e0d560"
Oct 30 19:43:22.801880 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:22.801847    4888 generic.go:356] "Generic (PLEG): container finished" podID="e7d8237d-903d-4136-8e74-b90346f5f619" containerID="1d49bc4b6f14c5398972d57b8d01621d3e0c5c7cb11d650bb459c60fd18eb8b4" exitCode=0
Oct 30 19:43:22.801999 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:22.801910    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-2ade09-85459998c9-cksnq"
Oct 30 19:43:22.801999 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:22.801949    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-2ade09-85459998c9-cksnq" event={"ID":"e7d8237d-903d-4136-8e74-b90346f5f619","Type":"ContainerDied","Data":"1d49bc4b6f14c5398972d57b8d01621d3e0c5c7cb11d650bb459c60fd18eb8b4"}
Oct 30 19:43:22.801999 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:22.801987    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-2ade09-85459998c9-cksnq" event={"ID":"e7d8237d-903d-4136-8e74-b90346f5f619","Type":"ContainerDied","Data":"34f3b6512127594436ebf00b2394410c65f99c51a7ab17a1a7606b2f50052761"}
Oct 30 19:43:22.803775 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:22.803748    4888 generic.go:356] "Generic (PLEG): container finished" podID="771fbee7-b699-4a2b-a2cd-a2f78bc8e198" containerID="57f9536cea451c7dce5cf63b5237a082999a01f7b392219964ffdd7b9a156620" exitCode=0
Oct 30 19:43:22.803899 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:22.803801    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-2ade09-867957c796-stmnm" event={"ID":"771fbee7-b699-4a2b-a2cd-a2f78bc8e198","Type":"ContainerDied","Data":"57f9536cea451c7dce5cf63b5237a082999a01f7b392219964ffdd7b9a156620"}
Oct 30 19:43:22.803899 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:22.803840    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-2ade09-867957c796-stmnm" event={"ID":"771fbee7-b699-4a2b-a2cd-a2f78bc8e198","Type":"ContainerDied","Data":"df52c879a2bc5510ee680d79439fc92ae0e2395d6b45a9fdf83a9ef3542faf70"}
Oct 30 19:43:22.803899 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:22.803792    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-2ade09-867957c796-stmnm"
Oct 30 19:43:22.813461 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:22.813325    4888 scope.go:122] "RemoveContainer" containerID="d212141157e251729a834d30ecbf004ac79b3a9093f6d80f1391bae628e0d560"
Oct 30 19:43:22.813674 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:43:22.813653    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d212141157e251729a834d30ecbf004ac79b3a9093f6d80f1391bae628e0d560\": container with ID starting with d212141157e251729a834d30ecbf004ac79b3a9093f6d80f1391bae628e0d560 not found: ID does not exist" containerID="d212141157e251729a834d30ecbf004ac79b3a9093f6d80f1391bae628e0d560"
Oct 30 19:43:22.813779 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:22.813686    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d212141157e251729a834d30ecbf004ac79b3a9093f6d80f1391bae628e0d560"} err="failed to get container status \"d212141157e251729a834d30ecbf004ac79b3a9093f6d80f1391bae628e0d560\": rpc error: code = NotFound desc = could not find container \"d212141157e251729a834d30ecbf004ac79b3a9093f6d80f1391bae628e0d560\": container with ID starting with d212141157e251729a834d30ecbf004ac79b3a9093f6d80f1391bae628e0d560 not found: ID does not exist"
Oct 30 19:43:22.813779 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:22.813707    4888 scope.go:122] "RemoveContainer" containerID="1d49bc4b6f14c5398972d57b8d01621d3e0c5c7cb11d650bb459c60fd18eb8b4"
Oct 30 19:43:22.840464 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:22.840369    4888 scope.go:122] "RemoveContainer" containerID="1d49bc4b6f14c5398972d57b8d01621d3e0c5c7cb11d650bb459c60fd18eb8b4"
Oct 30 19:43:22.840993 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:43:22.840962    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1d49bc4b6f14c5398972d57b8d01621d3e0c5c7cb11d650bb459c60fd18eb8b4\": container with ID starting with 1d49bc4b6f14c5398972d57b8d01621d3e0c5c7cb11d650bb459c60fd18eb8b4 not found: ID does not exist" containerID="1d49bc4b6f14c5398972d57b8d01621d3e0c5c7cb11d650bb459c60fd18eb8b4"
Oct 30 19:43:22.850338 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:22.841008    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1d49bc4b6f14c5398972d57b8d01621d3e0c5c7cb11d650bb459c60fd18eb8b4"} err="failed to get container status \"1d49bc4b6f14c5398972d57b8d01621d3e0c5c7cb11d650bb459c60fd18eb8b4\": rpc error: code = NotFound desc = could not find container \"1d49bc4b6f14c5398972d57b8d01621d3e0c5c7cb11d650bb459c60fd18eb8b4\": container with ID starting with 1d49bc4b6f14c5398972d57b8d01621d3e0c5c7cb11d650bb459c60fd18eb8b4 not found: ID does not exist"
Oct 30 19:43:22.850338 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:22.841034    4888 scope.go:122] "RemoveContainer" containerID="57f9536cea451c7dce5cf63b5237a082999a01f7b392219964ffdd7b9a156620"
Oct 30 19:43:22.861511 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:22.861488    4888 scope.go:122] "RemoveContainer" containerID="57f9536cea451c7dce5cf63b5237a082999a01f7b392219964ffdd7b9a156620"
Oct 30 19:43:22.862275 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:43:22.862243    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"57f9536cea451c7dce5cf63b5237a082999a01f7b392219964ffdd7b9a156620\": container with ID starting with 57f9536cea451c7dce5cf63b5237a082999a01f7b392219964ffdd7b9a156620 not found: ID does not exist" containerID="57f9536cea451c7dce5cf63b5237a082999a01f7b392219964ffdd7b9a156620"
Oct 30 19:43:22.862405 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:22.862319    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"57f9536cea451c7dce5cf63b5237a082999a01f7b392219964ffdd7b9a156620"} err="failed to get container status \"57f9536cea451c7dce5cf63b5237a082999a01f7b392219964ffdd7b9a156620\": rpc error: code = NotFound desc = could not find container \"57f9536cea451c7dce5cf63b5237a082999a01f7b392219964ffdd7b9a156620\": container with ID starting with 57f9536cea451c7dce5cf63b5237a082999a01f7b392219964ffdd7b9a156620 not found: ID does not exist"
Oct 30 19:43:22.886486 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:22.886455    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a4fd7ad8-cdda-470a-86bf-c0c0df93fda3-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a4fd7ad8-cdda-470a-86bf-c0c0df93fda3-openshift-service-ca\") pod \"a4fd7ad8-cdda-470a-86bf-c0c0df93fda3\" (UID: \"a4fd7ad8-cdda-470a-86bf-c0c0df93fda3\") "
Oct 30 19:43:22.886654 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:22.886547    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a4fd7ad8-cdda-470a-86bf-c0c0df93fda3-metrics-certs\" (UniqueName: \"kubernetes.io/secret/a4fd7ad8-cdda-470a-86bf-c0c0df93fda3-metrics-certs\") pod \"a4fd7ad8-cdda-470a-86bf-c0c0df93fda3\" (UID: \"a4fd7ad8-cdda-470a-86bf-c0c0df93fda3\") "
Oct 30 19:43:22.886654 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:22.886579    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a4fd7ad8-cdda-470a-86bf-c0c0df93fda3-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a4fd7ad8-cdda-470a-86bf-c0c0df93fda3-maas-api-tls\") pod \"a4fd7ad8-cdda-470a-86bf-c0c0df93fda3\" (UID: \"a4fd7ad8-cdda-470a-86bf-c0c0df93fda3\") "
Oct 30 19:43:22.886777 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:22.886669    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a4fd7ad8-cdda-470a-86bf-c0c0df93fda3-kube-api-access-tllbc\" (UniqueName: \"kubernetes.io/projected/a4fd7ad8-cdda-470a-86bf-c0c0df93fda3-kube-api-access-tllbc\") pod \"a4fd7ad8-cdda-470a-86bf-c0c0df93fda3\" (UID: \"a4fd7ad8-cdda-470a-86bf-c0c0df93fda3\") "
Oct 30 19:43:22.886833 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:22.886792    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a4fd7ad8-cdda-470a-86bf-c0c0df93fda3-openshift-service-ca" pod "a4fd7ad8-cdda-470a-86bf-c0c0df93fda3" (UID: "a4fd7ad8-cdda-470a-86bf-c0c0df93fda3"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:43:22.887071 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:22.887041    4888 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a4fd7ad8-cdda-470a-86bf-c0c0df93fda3-openshift-service-ca\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:43:22.889130 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:22.889102    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a4fd7ad8-cdda-470a-86bf-c0c0df93fda3-kube-api-access-tllbc" pod "a4fd7ad8-cdda-470a-86bf-c0c0df93fda3" (UID: "a4fd7ad8-cdda-470a-86bf-c0c0df93fda3"). InnerVolumeSpecName "kube-api-access-tllbc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:43:22.889390 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:22.889370    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a4fd7ad8-cdda-470a-86bf-c0c0df93fda3-metrics-certs" pod "a4fd7ad8-cdda-470a-86bf-c0c0df93fda3" (UID: "a4fd7ad8-cdda-470a-86bf-c0c0df93fda3"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:43:22.889481 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:22.889423    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a4fd7ad8-cdda-470a-86bf-c0c0df93fda3-maas-api-tls" pod "a4fd7ad8-cdda-470a-86bf-c0c0df93fda3" (UID: "a4fd7ad8-cdda-470a-86bf-c0c0df93fda3"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:43:22.987852 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:22.987812    4888 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a4fd7ad8-cdda-470a-86bf-c0c0df93fda3-metrics-certs\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:43:22.987852 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:22.987840    4888 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a4fd7ad8-cdda-470a-86bf-c0c0df93fda3-maas-api-tls\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:43:22.987852 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:22.987850    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tllbc\" (UniqueName: \"kubernetes.io/projected/a4fd7ad8-cdda-470a-86bf-c0c0df93fda3-kube-api-access-tllbc\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:43:23.080862 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:23.080830    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-2ade09-867957c796-stmnm"]
Oct 30 19:43:23.084214 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:23.084083    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-2ade09-867957c796-stmnm"]
Oct 30 19:43:23.195266 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:23.195236    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-2ade09-85459998c9-cksnq"]
Oct 30 19:43:23.200166 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:23.200141    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-2ade09-85459998c9-cksnq"]
Oct 30 19:43:23.323612 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:23.323566    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-fd61e3-7647c5bb9f-h4th6"]
Oct 30 19:43:23.325954 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:23.325932    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-fd61e3-7647c5bb9f-h4th6"]
Oct 30 19:43:23.606445 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:23.606412    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-2ade09-d8b4fccb4-mjj8l"]
Oct 30 19:43:23.606649 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:23.606621    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-2ade09-d8b4fccb4-mjj8l" podUID="ddf8afd2-ed32-44bb-87ce-c26cde475f08" containerName="maas-api" containerID="cri-o://8108d78fc0d001365fb731a353b144c815deac503635bf95ff6d36870d902edb" gracePeriod=30
Oct 30 19:43:23.824930 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:23.824885    4888 generic.go:356] "Generic (PLEG): container finished" podID="ddf8afd2-ed32-44bb-87ce-c26cde475f08" containerID="8108d78fc0d001365fb731a353b144c815deac503635bf95ff6d36870d902edb" exitCode=0
Oct 30 19:43:23.825133 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:23.824962    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-2ade09-d8b4fccb4-mjj8l" event={"ID":"ddf8afd2-ed32-44bb-87ce-c26cde475f08","Type":"ContainerDied","Data":"8108d78fc0d001365fb731a353b144c815deac503635bf95ff6d36870d902edb"}
Oct 30 19:43:23.886206 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:23.886178    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-2ade09-d8b4fccb4-mjj8l"
Oct 30 19:43:23.890100 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:23.890073    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="771fbee7-b699-4a2b-a2cd-a2f78bc8e198" path="/var/lib/kubelet/pods/771fbee7-b699-4a2b-a2cd-a2f78bc8e198/volumes"
Oct 30 19:43:23.890537 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:23.890519    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a4fd7ad8-cdda-470a-86bf-c0c0df93fda3" path="/var/lib/kubelet/pods/a4fd7ad8-cdda-470a-86bf-c0c0df93fda3/volumes"
Oct 30 19:43:23.890864 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:23.890850    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e7d8237d-903d-4136-8e74-b90346f5f619" path="/var/lib/kubelet/pods/e7d8237d-903d-4136-8e74-b90346f5f619/volumes"
Oct 30 19:43:23.997861 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:23.997835    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ddf8afd2-ed32-44bb-87ce-c26cde475f08-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ddf8afd2-ed32-44bb-87ce-c26cde475f08-maas-api-tls\") pod \"ddf8afd2-ed32-44bb-87ce-c26cde475f08\" (UID: \"ddf8afd2-ed32-44bb-87ce-c26cde475f08\") "
Oct 30 19:43:23.998013 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:23.997876    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ddf8afd2-ed32-44bb-87ce-c26cde475f08-metrics-certs\" (UniqueName: \"kubernetes.io/secret/ddf8afd2-ed32-44bb-87ce-c26cde475f08-metrics-certs\") pod \"ddf8afd2-ed32-44bb-87ce-c26cde475f08\" (UID: \"ddf8afd2-ed32-44bb-87ce-c26cde475f08\") "
Oct 30 19:43:23.998013 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:23.997929    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ddf8afd2-ed32-44bb-87ce-c26cde475f08-kube-api-access-cx52k\" (UniqueName: \"kubernetes.io/projected/ddf8afd2-ed32-44bb-87ce-c26cde475f08-kube-api-access-cx52k\") pod \"ddf8afd2-ed32-44bb-87ce-c26cde475f08\" (UID: \"ddf8afd2-ed32-44bb-87ce-c26cde475f08\") "
Oct 30 19:43:23.998013 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:23.997990    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ddf8afd2-ed32-44bb-87ce-c26cde475f08-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ddf8afd2-ed32-44bb-87ce-c26cde475f08-openshift-service-ca\") pod \"ddf8afd2-ed32-44bb-87ce-c26cde475f08\" (UID: \"ddf8afd2-ed32-44bb-87ce-c26cde475f08\") "
Oct 30 19:43:23.998508 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:23.998477    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ddf8afd2-ed32-44bb-87ce-c26cde475f08-openshift-service-ca" pod "ddf8afd2-ed32-44bb-87ce-c26cde475f08" (UID: "ddf8afd2-ed32-44bb-87ce-c26cde475f08"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:43:24.000222 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:24.000197    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ddf8afd2-ed32-44bb-87ce-c26cde475f08-maas-api-tls" pod "ddf8afd2-ed32-44bb-87ce-c26cde475f08" (UID: "ddf8afd2-ed32-44bb-87ce-c26cde475f08"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:43:24.000222 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:24.000213    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ddf8afd2-ed32-44bb-87ce-c26cde475f08-metrics-certs" pod "ddf8afd2-ed32-44bb-87ce-c26cde475f08" (UID: "ddf8afd2-ed32-44bb-87ce-c26cde475f08"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:43:24.000463 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:24.000251    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ddf8afd2-ed32-44bb-87ce-c26cde475f08-kube-api-access-cx52k" pod "ddf8afd2-ed32-44bb-87ce-c26cde475f08" (UID: "ddf8afd2-ed32-44bb-87ce-c26cde475f08"). InnerVolumeSpecName "kube-api-access-cx52k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:43:24.099519 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:24.099483    4888 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ddf8afd2-ed32-44bb-87ce-c26cde475f08-maas-api-tls\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:43:24.099519 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:24.099512    4888 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ddf8afd2-ed32-44bb-87ce-c26cde475f08-metrics-certs\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:43:24.099753 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:24.099533    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cx52k\" (UniqueName: \"kubernetes.io/projected/ddf8afd2-ed32-44bb-87ce-c26cde475f08-kube-api-access-cx52k\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:43:24.099753 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:24.099546    4888 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ddf8afd2-ed32-44bb-87ce-c26cde475f08-openshift-service-ca\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:43:24.835642 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:24.835614    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-2ade09-d8b4fccb4-mjj8l"
Oct 30 19:43:24.835642 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:24.835623    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-2ade09-d8b4fccb4-mjj8l" event={"ID":"ddf8afd2-ed32-44bb-87ce-c26cde475f08","Type":"ContainerDied","Data":"dc6565fc2b862e5daaa7dd9e5b5e2669eccfeaf90bef568b8b5554499dcd3ea2"}
Oct 30 19:43:24.836191 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:24.835732    4888 scope.go:122] "RemoveContainer" containerID="8108d78fc0d001365fb731a353b144c815deac503635bf95ff6d36870d902edb"
Oct 30 19:43:25.010136 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:25.010093    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-2ade09-d8b4fccb4-mjj8l"]
Oct 30 19:43:25.059353 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:25.059321    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-2ade09-d8b4fccb4-mjj8l"]
Oct 30 19:43:25.889815 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:25.889784    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ddf8afd2-ed32-44bb-87ce-c26cde475f08" path="/var/lib/kubelet/pods/ddf8afd2-ed32-44bb-87ce-c26cde475f08/volumes"
Oct 30 19:43:32.377931 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:32.377897    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-msdwn"]
Oct 30 19:43:32.378402 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:32.378114    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-msdwn" podUID="f49068b0-4dea-487c-a31c-c61704119dad" containerName="istio-proxy" containerID="cri-o://77d084411a7aed534925ca3711c62beb55728f847f45ec1544e4a454432b7e1a" gracePeriod=30
Oct 30 19:43:35.619241 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:43:35.619155    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-jbsw6" containerName="extractor"
Oct 30 19:43:37.639659 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:37.639631    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-msdwn"
Oct 30 19:43:37.742684 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:37.742597    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f49068b0-4dea-487c-a31c-c61704119dad-istio-token\" (UniqueName: \"kubernetes.io/projected/f49068b0-4dea-487c-a31c-c61704119dad-istio-token\") pod \"f49068b0-4dea-487c-a31c-c61704119dad\" (UID: \"f49068b0-4dea-487c-a31c-c61704119dad\") "
Oct 30 19:43:37.742684 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:37.742652    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f49068b0-4dea-487c-a31c-c61704119dad-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f49068b0-4dea-487c-a31c-c61704119dad-istio-envoy\") pod \"f49068b0-4dea-487c-a31c-c61704119dad\" (UID: \"f49068b0-4dea-487c-a31c-c61704119dad\") "
Oct 30 19:43:37.742684 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:37.742671    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f49068b0-4dea-487c-a31c-c61704119dad-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f49068b0-4dea-487c-a31c-c61704119dad-istiod-ca-cert\") pod \"f49068b0-4dea-487c-a31c-c61704119dad\" (UID: \"f49068b0-4dea-487c-a31c-c61704119dad\") "
Oct 30 19:43:37.742939 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:37.742747    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/f49068b0-4dea-487c-a31c-c61704119dad-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f49068b0-4dea-487c-a31c-c61704119dad-istio-podinfo\") pod \"f49068b0-4dea-487c-a31c-c61704119dad\" (UID: \"f49068b0-4dea-487c-a31c-c61704119dad\") "
Oct 30 19:43:37.742939 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:37.742776    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f49068b0-4dea-487c-a31c-c61704119dad-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f49068b0-4dea-487c-a31c-c61704119dad-istio-data\") pod \"f49068b0-4dea-487c-a31c-c61704119dad\" (UID: \"f49068b0-4dea-487c-a31c-c61704119dad\") "
Oct 30 19:43:37.742939 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:37.742842    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f49068b0-4dea-487c-a31c-c61704119dad-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f49068b0-4dea-487c-a31c-c61704119dad-workload-socket\") pod \"f49068b0-4dea-487c-a31c-c61704119dad\" (UID: \"f49068b0-4dea-487c-a31c-c61704119dad\") "
Oct 30 19:43:37.742939 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:37.742929    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f49068b0-4dea-487c-a31c-c61704119dad-kube-api-access-wm52q\" (UniqueName: \"kubernetes.io/projected/f49068b0-4dea-487c-a31c-c61704119dad-kube-api-access-wm52q\") pod \"f49068b0-4dea-487c-a31c-c61704119dad\" (UID: \"f49068b0-4dea-487c-a31c-c61704119dad\") "
Oct 30 19:43:37.743175 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:37.742979    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f49068b0-4dea-487c-a31c-c61704119dad-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f49068b0-4dea-487c-a31c-c61704119dad-workload-certs\") pod \"f49068b0-4dea-487c-a31c-c61704119dad\" (UID: \"f49068b0-4dea-487c-a31c-c61704119dad\") "
Oct 30 19:43:37.743175 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:37.743012    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f49068b0-4dea-487c-a31c-c61704119dad-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f49068b0-4dea-487c-a31c-c61704119dad-credential-socket\") pod \"f49068b0-4dea-487c-a31c-c61704119dad\" (UID: \"f49068b0-4dea-487c-a31c-c61704119dad\") "
Oct 30 19:43:37.743175 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:37.743083    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f49068b0-4dea-487c-a31c-c61704119dad-istio-data" pod "f49068b0-4dea-487c-a31c-c61704119dad" (UID: "f49068b0-4dea-487c-a31c-c61704119dad"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:43:37.743371 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:37.743192    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f49068b0-4dea-487c-a31c-c61704119dad-workload-socket" pod "f49068b0-4dea-487c-a31c-c61704119dad" (UID: "f49068b0-4dea-487c-a31c-c61704119dad"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:43:37.743371 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:37.743211    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f49068b0-4dea-487c-a31c-c61704119dad-istiod-ca-cert" pod "f49068b0-4dea-487c-a31c-c61704119dad" (UID: "f49068b0-4dea-487c-a31c-c61704119dad"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:43:37.743488 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:37.743359    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f49068b0-4dea-487c-a31c-c61704119dad-credential-socket" pod "f49068b0-4dea-487c-a31c-c61704119dad" (UID: "f49068b0-4dea-487c-a31c-c61704119dad"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:43:37.743488 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:37.743424    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f49068b0-4dea-487c-a31c-c61704119dad-workload-certs" pod "f49068b0-4dea-487c-a31c-c61704119dad" (UID: "f49068b0-4dea-487c-a31c-c61704119dad"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:43:37.743622 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:37.743529    4888 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f49068b0-4dea-487c-a31c-c61704119dad-workload-certs\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:43:37.743622 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:37.743552    4888 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f49068b0-4dea-487c-a31c-c61704119dad-credential-socket\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:43:37.743622 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:37.743582    4888 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f49068b0-4dea-487c-a31c-c61704119dad-istiod-ca-cert\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:43:37.743622 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:37.743595    4888 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f49068b0-4dea-487c-a31c-c61704119dad-istio-data\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:43:37.743622 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:37.743608    4888 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f49068b0-4dea-487c-a31c-c61704119dad-workload-socket\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:43:37.745385 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:37.745339    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f49068b0-4dea-487c-a31c-c61704119dad-kube-api-access-wm52q" pod "f49068b0-4dea-487c-a31c-c61704119dad" (UID: "f49068b0-4dea-487c-a31c-c61704119dad"). InnerVolumeSpecName "kube-api-access-wm52q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:43:37.745506 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:37.745445    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f49068b0-4dea-487c-a31c-c61704119dad-istio-token" pod "f49068b0-4dea-487c-a31c-c61704119dad" (UID: "f49068b0-4dea-487c-a31c-c61704119dad"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:43:37.745697 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:37.745678    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f49068b0-4dea-487c-a31c-c61704119dad-istio-envoy" pod "f49068b0-4dea-487c-a31c-c61704119dad" (UID: "f49068b0-4dea-487c-a31c-c61704119dad"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:43:37.745756 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:37.745687    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f49068b0-4dea-487c-a31c-c61704119dad-istio-podinfo" pod "f49068b0-4dea-487c-a31c-c61704119dad" (UID: "f49068b0-4dea-487c-a31c-c61704119dad"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:43:37.844354 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:37.844317    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wm52q\" (UniqueName: \"kubernetes.io/projected/f49068b0-4dea-487c-a31c-c61704119dad-kube-api-access-wm52q\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:43:37.844354 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:37.844356    4888 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f49068b0-4dea-487c-a31c-c61704119dad-istio-token\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:43:37.844684 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:37.844373    4888 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f49068b0-4dea-487c-a31c-c61704119dad-istio-envoy\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:43:37.844684 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:37.844388    4888 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f49068b0-4dea-487c-a31c-c61704119dad-istio-podinfo\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:43:37.915319 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:37.915262    4888 generic.go:356] "Generic (PLEG): container finished" podID="f49068b0-4dea-487c-a31c-c61704119dad" containerID="77d084411a7aed534925ca3711c62beb55728f847f45ec1544e4a454432b7e1a" exitCode=0
Oct 30 19:43:37.915523 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:37.915327    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-msdwn" event={"ID":"f49068b0-4dea-487c-a31c-c61704119dad","Type":"ContainerDied","Data":"77d084411a7aed534925ca3711c62beb55728f847f45ec1544e4a454432b7e1a"}
Oct 30 19:43:37.915523 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:37.915338    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-msdwn"
Oct 30 19:43:37.915523 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:37.915382    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-msdwn" event={"ID":"f49068b0-4dea-487c-a31c-c61704119dad","Type":"ContainerDied","Data":"504fb43091830fbd5741e6969177104c225dc833e9ec65e997280cf50f627901"}
Oct 30 19:43:37.915523 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:37.915405    4888 scope.go:122] "RemoveContainer" containerID="77d084411a7aed534925ca3711c62beb55728f847f45ec1544e4a454432b7e1a"
Oct 30 19:43:37.924840 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:37.924819    4888 scope.go:122] "RemoveContainer" containerID="f7e4b2330162644e18b81c185c852390d98e3ec3f146b0a5e16ea2db1a8e781c"
Oct 30 19:43:37.934513 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:37.934496    4888 scope.go:122] "RemoveContainer" containerID="77d084411a7aed534925ca3711c62beb55728f847f45ec1544e4a454432b7e1a"
Oct 30 19:43:37.934832 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:43:37.934803    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"77d084411a7aed534925ca3711c62beb55728f847f45ec1544e4a454432b7e1a\": container with ID starting with 77d084411a7aed534925ca3711c62beb55728f847f45ec1544e4a454432b7e1a not found: ID does not exist" containerID="77d084411a7aed534925ca3711c62beb55728f847f45ec1544e4a454432b7e1a"
Oct 30 19:43:37.934958 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:37.934844    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"77d084411a7aed534925ca3711c62beb55728f847f45ec1544e4a454432b7e1a"} err="failed to get container status \"77d084411a7aed534925ca3711c62beb55728f847f45ec1544e4a454432b7e1a\": rpc error: code = NotFound desc = could not find container \"77d084411a7aed534925ca3711c62beb55728f847f45ec1544e4a454432b7e1a\": container with ID starting with 77d084411a7aed534925ca3711c62beb55728f847f45ec1544e4a454432b7e1a not found: ID does not exist"
Oct 30 19:43:37.934958 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:37.934867    4888 scope.go:122] "RemoveContainer" containerID="f7e4b2330162644e18b81c185c852390d98e3ec3f146b0a5e16ea2db1a8e781c"
Oct 30 19:43:37.935127 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:43:37.935108    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f7e4b2330162644e18b81c185c852390d98e3ec3f146b0a5e16ea2db1a8e781c\": container with ID starting with f7e4b2330162644e18b81c185c852390d98e3ec3f146b0a5e16ea2db1a8e781c not found: ID does not exist" containerID="f7e4b2330162644e18b81c185c852390d98e3ec3f146b0a5e16ea2db1a8e781c"
Oct 30 19:43:37.935189 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:37.935136    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f7e4b2330162644e18b81c185c852390d98e3ec3f146b0a5e16ea2db1a8e781c"} err="failed to get container status \"f7e4b2330162644e18b81c185c852390d98e3ec3f146b0a5e16ea2db1a8e781c\": rpc error: code = NotFound desc = could not find container \"f7e4b2330162644e18b81c185c852390d98e3ec3f146b0a5e16ea2db1a8e781c\": container with ID starting with f7e4b2330162644e18b81c185c852390d98e3ec3f146b0a5e16ea2db1a8e781c not found: ID does not exist"
Oct 30 19:43:38.089966 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:38.089937    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-msdwn"]
Oct 30 19:43:38.110788 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:38.110757    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-fd61e3-openshift-default-748b77c6b9-msdwn"]
Oct 30 19:43:39.890278 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:39.890248    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f49068b0-4dea-487c-a31c-c61704119dad" path="/var/lib/kubelet/pods/f49068b0-4dea-487c-a31c-c61704119dad/volumes"
Oct 30 19:43:56.902589 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:56.902558    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-7x4rb_382e8b31-a7fa-43f3-9f15-4c61627b0c04/console-operator/2.log"
Oct 30 19:43:56.912474 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:56.912446    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xfcdl_3f3fe764-2652-46c9-8e54-7882c0e6bcf1/ovn-acl-logging/0.log"
Oct 30 19:43:56.916529 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:56.916506    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-7x4rb_382e8b31-a7fa-43f3-9f15-4c61627b0c04/console-operator/2.log"
Oct 30 19:43:56.924339 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:56.924323    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xfcdl_3f3fe764-2652-46c9-8e54-7882c0e6bcf1/ovn-acl-logging/0.log"
Oct 30 19:43:58.610193 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:43:58.610162    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-rjmdz"]
Oct 30 19:44:05.617844 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:44:05.617807    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-jbsw6" containerName="extractor"
Oct 30 19:44:15.335982 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:44:15.335948    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-rjmdz"]
Oct 30 19:44:33.085851 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:44:33.085815    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-rjmdz"]
Oct 30 19:44:35.617587 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:44:35.617555    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-jbsw6" containerName="extractor"
Oct 30 19:44:49.149708 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:44:49.149679    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-98b389-7c5dd76cdf-tbchn"]
Oct 30 19:44:49.150144 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:44:49.149883    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-98b389-7c5dd76cdf-tbchn" podUID="f1743252-096e-4c09-85a4-e8d60e0709c0" containerName="maas-api" containerID="cri-o://ef5833ed11ef556be277bfcbebd71a1cbbe6be942029ebe08d7e1b32de9984bc" gracePeriod=30
Oct 30 19:44:49.251560 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:44:49.251527    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-98b389-c94bf75bb-2xl4p"]
Oct 30 19:44:49.344516 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:44:49.344489    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-98b389-7bc5cb55bc-vp4tb"]
Oct 30 19:44:49.354500 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:44:49.354463    4888 generic.go:356] "Generic (PLEG): container finished" podID="f1743252-096e-4c09-85a4-e8d60e0709c0" containerID="ef5833ed11ef556be277bfcbebd71a1cbbe6be942029ebe08d7e1b32de9984bc" exitCode=0
Oct 30 19:44:49.354686 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:44:49.354517    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-98b389-7c5dd76cdf-tbchn" event={"ID":"f1743252-096e-4c09-85a4-e8d60e0709c0","Type":"ContainerDied","Data":"ef5833ed11ef556be277bfcbebd71a1cbbe6be942029ebe08d7e1b32de9984bc"}
Oct 30 19:44:49.409228 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:44:49.409177    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-98b389-7c5dd76cdf-tbchn"
Oct 30 19:44:49.455892 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:44:49.455866    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f1743252-096e-4c09-85a4-e8d60e0709c0-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f1743252-096e-4c09-85a4-e8d60e0709c0-maas-api-tls\") pod \"f1743252-096e-4c09-85a4-e8d60e0709c0\" (UID: \"f1743252-096e-4c09-85a4-e8d60e0709c0\") "
Oct 30 19:44:49.456005 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:44:49.455950    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f1743252-096e-4c09-85a4-e8d60e0709c0-metrics-certs\" (UniqueName: \"kubernetes.io/secret/f1743252-096e-4c09-85a4-e8d60e0709c0-metrics-certs\") pod \"f1743252-096e-4c09-85a4-e8d60e0709c0\" (UID: \"f1743252-096e-4c09-85a4-e8d60e0709c0\") "
Oct 30 19:44:49.456005 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:44:49.455980    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f1743252-096e-4c09-85a4-e8d60e0709c0-kube-api-access-8w6hg\" (UniqueName: \"kubernetes.io/projected/f1743252-096e-4c09-85a4-e8d60e0709c0-kube-api-access-8w6hg\") pod \"f1743252-096e-4c09-85a4-e8d60e0709c0\" (UID: \"f1743252-096e-4c09-85a4-e8d60e0709c0\") "
Oct 30 19:44:49.456096 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:44:49.456025    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f1743252-096e-4c09-85a4-e8d60e0709c0-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f1743252-096e-4c09-85a4-e8d60e0709c0-openshift-service-ca\") pod \"f1743252-096e-4c09-85a4-e8d60e0709c0\" (UID: \"f1743252-096e-4c09-85a4-e8d60e0709c0\") "
Oct 30 19:44:49.456473 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:44:49.456452    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f1743252-096e-4c09-85a4-e8d60e0709c0-openshift-service-ca" pod "f1743252-096e-4c09-85a4-e8d60e0709c0" (UID: "f1743252-096e-4c09-85a4-e8d60e0709c0"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:44:49.458172 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:44:49.458132    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f1743252-096e-4c09-85a4-e8d60e0709c0-metrics-certs" pod "f1743252-096e-4c09-85a4-e8d60e0709c0" (UID: "f1743252-096e-4c09-85a4-e8d60e0709c0"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:44:49.458272 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:44:49.458200    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f1743252-096e-4c09-85a4-e8d60e0709c0-kube-api-access-8w6hg" pod "f1743252-096e-4c09-85a4-e8d60e0709c0" (UID: "f1743252-096e-4c09-85a4-e8d60e0709c0"). InnerVolumeSpecName "kube-api-access-8w6hg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:44:49.458272 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:44:49.458231    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f1743252-096e-4c09-85a4-e8d60e0709c0-maas-api-tls" pod "f1743252-096e-4c09-85a4-e8d60e0709c0" (UID: "f1743252-096e-4c09-85a4-e8d60e0709c0"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:44:49.562936 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:44:49.562903    4888 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f1743252-096e-4c09-85a4-e8d60e0709c0-openshift-service-ca\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:44:49.562936 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:44:49.562937    4888 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f1743252-096e-4c09-85a4-e8d60e0709c0-maas-api-tls\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:44:49.563157 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:44:49.562950    4888 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f1743252-096e-4c09-85a4-e8d60e0709c0-metrics-certs\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:44:49.563157 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:44:49.562964    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8w6hg\" (UniqueName: \"kubernetes.io/projected/f1743252-096e-4c09-85a4-e8d60e0709c0-kube-api-access-8w6hg\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:44:50.361505 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:44:50.361476    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-98b389-7c5dd76cdf-tbchn"
Oct 30 19:44:50.361958 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:44:50.361513    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-98b389-7c5dd76cdf-tbchn" event={"ID":"f1743252-096e-4c09-85a4-e8d60e0709c0","Type":"ContainerDied","Data":"4820f9a76a6d0a80951c69637d56a5674fe101a6b06c9a36554ef3d770e6e788"}
Oct 30 19:44:50.361958 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:44:50.361578    4888 scope.go:122] "RemoveContainer" containerID="ef5833ed11ef556be277bfcbebd71a1cbbe6be942029ebe08d7e1b32de9984bc"
Oct 30 19:44:50.530770 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:44:50.530734    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-98b389-7c5dd76cdf-tbchn"]
Oct 30 19:44:50.581682 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:44:50.581643    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-98b389-7c5dd76cdf-tbchn"]
Oct 30 19:44:51.890242 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:44:51.890207    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f1743252-096e-4c09-85a4-e8d60e0709c0" path="/var/lib/kubelet/pods/f1743252-096e-4c09-85a4-e8d60e0709c0/volumes"
Oct 30 19:44:54.252119 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:44:54.252060    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-extmod-a-98b389-c94bf75bb-2xl4p" podUID="5d128a48-cd8d-4a3f-ab1b-1f007ae62e5d" containerName="payload-processing" containerID="cri-o://019e2c88716d2a5c3511d9be297a7e979ee631e3791f8afe313a085ba2a64ae2" gracePeriod=25
Oct 30 19:44:54.345682 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:44:54.345626    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-98b389-7bc5cb55bc-vp4tb" podUID="a49754b1-9b01-48d6-8146-a4354554e8b3" containerName="payload-pre-processing" containerID="cri-o://c746d54e45340b6fd510cb4f900004d43d494e8ba1fbea6ca8f63aa51aaa7fbd" gracePeriod=25
Oct 30 19:44:54.399374 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:44:54.399341    4888 generic.go:356] "Generic (PLEG): container finished" podID="5d128a48-cd8d-4a3f-ab1b-1f007ae62e5d" containerID="019e2c88716d2a5c3511d9be297a7e979ee631e3791f8afe313a085ba2a64ae2" exitCode=0
Oct 30 19:44:54.399549 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:44:54.399422    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-a-98b389-c94bf75bb-2xl4p" event={"ID":"5d128a48-cd8d-4a3f-ab1b-1f007ae62e5d","Type":"ContainerDied","Data":"019e2c88716d2a5c3511d9be297a7e979ee631e3791f8afe313a085ba2a64ae2"}
Oct 30 19:44:54.630341 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:44:54.630314    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-a-98b389-c94bf75bb-2xl4p"
Oct 30 19:44:54.652159 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:44:54.652135    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-98b389-7bc5cb55bc-vp4tb"
Oct 30 19:44:54.710235 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:44:54.710195    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a49754b1-9b01-48d6-8146-a4354554e8b3-kube-api-access-p5snp\" (UniqueName: \"kubernetes.io/projected/a49754b1-9b01-48d6-8146-a4354554e8b3-kube-api-access-p5snp\") pod \"a49754b1-9b01-48d6-8146-a4354554e8b3\" (UID: \"a49754b1-9b01-48d6-8146-a4354554e8b3\") "
Oct 30 19:44:54.710422 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:44:54.710305    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a49754b1-9b01-48d6-8146-a4354554e8b3-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a49754b1-9b01-48d6-8146-a4354554e8b3-plugins-config-volume\") pod \"a49754b1-9b01-48d6-8146-a4354554e8b3\" (UID: \"a49754b1-9b01-48d6-8146-a4354554e8b3\") "
Oct 30 19:44:54.710422 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:44:54.710373    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5d128a48-cd8d-4a3f-ab1b-1f007ae62e5d-kube-api-access-t4mzc\" (UniqueName: \"kubernetes.io/projected/5d128a48-cd8d-4a3f-ab1b-1f007ae62e5d-kube-api-access-t4mzc\") pod \"5d128a48-cd8d-4a3f-ab1b-1f007ae62e5d\" (UID: \"5d128a48-cd8d-4a3f-ab1b-1f007ae62e5d\") "
Oct 30 19:44:54.710567 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:44:54.710425    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5d128a48-cd8d-4a3f-ab1b-1f007ae62e5d-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5d128a48-cd8d-4a3f-ab1b-1f007ae62e5d-plugins-config-volume\") pod \"5d128a48-cd8d-4a3f-ab1b-1f007ae62e5d\" (UID: \"5d128a48-cd8d-4a3f-ab1b-1f007ae62e5d\") "
Oct 30 19:44:54.710775 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:44:54.710733    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a49754b1-9b01-48d6-8146-a4354554e8b3-plugins-config-volume" pod "a49754b1-9b01-48d6-8146-a4354554e8b3" (UID: "a49754b1-9b01-48d6-8146-a4354554e8b3"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:44:54.711043 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:44:54.711015    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5d128a48-cd8d-4a3f-ab1b-1f007ae62e5d-plugins-config-volume" pod "5d128a48-cd8d-4a3f-ab1b-1f007ae62e5d" (UID: "5d128a48-cd8d-4a3f-ab1b-1f007ae62e5d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:44:54.712777 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:44:54.712750    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5d128a48-cd8d-4a3f-ab1b-1f007ae62e5d-kube-api-access-t4mzc" pod "5d128a48-cd8d-4a3f-ab1b-1f007ae62e5d" (UID: "5d128a48-cd8d-4a3f-ab1b-1f007ae62e5d"). InnerVolumeSpecName "kube-api-access-t4mzc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:44:54.713063 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:44:54.713043    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a49754b1-9b01-48d6-8146-a4354554e8b3-kube-api-access-p5snp" pod "a49754b1-9b01-48d6-8146-a4354554e8b3" (UID: "a49754b1-9b01-48d6-8146-a4354554e8b3"). InnerVolumeSpecName "kube-api-access-p5snp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:44:54.811260 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:44:54.811178    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p5snp\" (UniqueName: \"kubernetes.io/projected/a49754b1-9b01-48d6-8146-a4354554e8b3-kube-api-access-p5snp\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:44:54.811260 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:44:54.811209    4888 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a49754b1-9b01-48d6-8146-a4354554e8b3-plugins-config-volume\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:44:54.811260 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:44:54.811219    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t4mzc\" (UniqueName: \"kubernetes.io/projected/5d128a48-cd8d-4a3f-ab1b-1f007ae62e5d-kube-api-access-t4mzc\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:44:54.811260 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:44:54.811227    4888 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5d128a48-cd8d-4a3f-ab1b-1f007ae62e5d-plugins-config-volume\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:44:55.199843 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:44:55.199799    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-rjmdz"]
Oct 30 19:44:55.406063 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:44:55.406030    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-a-98b389-c94bf75bb-2xl4p" event={"ID":"5d128a48-cd8d-4a3f-ab1b-1f007ae62e5d","Type":"ContainerDied","Data":"8400fc569db191b09bb4318085f49fe4a1f51f8a1cd46857c5646e2f7fffb7fb"}
Oct 30 19:44:55.406566 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:44:55.406094    4888 scope.go:122] "RemoveContainer" containerID="019e2c88716d2a5c3511d9be297a7e979ee631e3791f8afe313a085ba2a64ae2"
Oct 30 19:44:55.406566 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:44:55.406041    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-a-98b389-c94bf75bb-2xl4p"
Oct 30 19:44:55.407891 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:44:55.407863    4888 generic.go:356] "Generic (PLEG): container finished" podID="a49754b1-9b01-48d6-8146-a4354554e8b3" containerID="c746d54e45340b6fd510cb4f900004d43d494e8ba1fbea6ca8f63aa51aaa7fbd" exitCode=0
Oct 30 19:44:55.408037 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:44:55.407915    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-98b389-7bc5cb55bc-vp4tb"
Oct 30 19:44:55.408037 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:44:55.407954    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-98b389-7bc5cb55bc-vp4tb" event={"ID":"a49754b1-9b01-48d6-8146-a4354554e8b3","Type":"ContainerDied","Data":"c746d54e45340b6fd510cb4f900004d43d494e8ba1fbea6ca8f63aa51aaa7fbd"}
Oct 30 19:44:55.408037 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:44:55.408002    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-98b389-7bc5cb55bc-vp4tb" event={"ID":"a49754b1-9b01-48d6-8146-a4354554e8b3","Type":"ContainerDied","Data":"979484f63dfc5df86452ac64b8eafb685e1369defee771abb85fda08e1c61ff4"}
Oct 30 19:44:55.417470 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:44:55.417450    4888 scope.go:122] "RemoveContainer" containerID="c746d54e45340b6fd510cb4f900004d43d494e8ba1fbea6ca8f63aa51aaa7fbd"
Oct 30 19:44:55.427331 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:44:55.427312    4888 scope.go:122] "RemoveContainer" containerID="c746d54e45340b6fd510cb4f900004d43d494e8ba1fbea6ca8f63aa51aaa7fbd"
Oct 30 19:44:55.427877 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:44:55.427851    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c746d54e45340b6fd510cb4f900004d43d494e8ba1fbea6ca8f63aa51aaa7fbd\": container with ID starting with c746d54e45340b6fd510cb4f900004d43d494e8ba1fbea6ca8f63aa51aaa7fbd not found: ID does not exist" containerID="c746d54e45340b6fd510cb4f900004d43d494e8ba1fbea6ca8f63aa51aaa7fbd"
Oct 30 19:44:55.427987 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:44:55.427891    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c746d54e45340b6fd510cb4f900004d43d494e8ba1fbea6ca8f63aa51aaa7fbd"} err="failed to get container status \"c746d54e45340b6fd510cb4f900004d43d494e8ba1fbea6ca8f63aa51aaa7fbd\": rpc error: code = NotFound desc = could not find container \"c746d54e45340b6fd510cb4f900004d43d494e8ba1fbea6ca8f63aa51aaa7fbd\": container with ID starting with c746d54e45340b6fd510cb4f900004d43d494e8ba1fbea6ca8f63aa51aaa7fbd not found: ID does not exist"
Oct 30 19:44:55.578769 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:44:55.578739    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-98b389-c94bf75bb-2xl4p"]
Oct 30 19:44:55.594477 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:44:55.594437    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-98b389-c94bf75bb-2xl4p"]
Oct 30 19:44:55.713197 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:44:55.713165    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-98b389-7bc5cb55bc-vp4tb"]
Oct 30 19:44:55.718486 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:44:55.718454    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-98b389-7bc5cb55bc-vp4tb"]
Oct 30 19:44:55.890119 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:44:55.890030    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5d128a48-cd8d-4a3f-ab1b-1f007ae62e5d" path="/var/lib/kubelet/pods/5d128a48-cd8d-4a3f-ab1b-1f007ae62e5d/volumes"
Oct 30 19:44:55.890565 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:44:55.890545    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a49754b1-9b01-48d6-8146-a4354554e8b3" path="/var/lib/kubelet/pods/a49754b1-9b01-48d6-8146-a4354554e8b3/volumes"
Oct 30 19:45:00.177749 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:00.177718    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-5da1ae-29889825-9bbwx"]
Oct 30 19:45:00.178161 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:00.178149    4888 cpu_manager.go:414] "containerMap: removing container" podUID="a49754b1-9b01-48d6-8146-a4354554e8b3" containerName="payload-pre-processing"
Oct 30 19:45:00.178235 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:00.178163    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a49754b1-9b01-48d6-8146-a4354554e8b3" containerName="payload-pre-processing"
Oct 30 19:45:00.178235 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:00.178175    4888 cpu_manager.go:414] "containerMap: removing container" podUID="f1743252-096e-4c09-85a4-e8d60e0709c0" containerName="maas-api"
Oct 30 19:45:00.178235 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:00.178180    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f1743252-096e-4c09-85a4-e8d60e0709c0" containerName="maas-api"
Oct 30 19:45:00.178235 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:00.178190    4888 cpu_manager.go:414] "containerMap: removing container" podUID="f49068b0-4dea-487c-a31c-c61704119dad" containerName="istio-proxy"
Oct 30 19:45:00.178235 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:00.178194    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f49068b0-4dea-487c-a31c-c61704119dad" containerName="istio-proxy"
Oct 30 19:45:00.178235 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:00.178200    4888 cpu_manager.go:414] "containerMap: removing container" podUID="36712903-e111-4fb5-a600-9b666b1d1c9f" containerName="istio-proxy"
Oct 30 19:45:00.178235 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:00.178204    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="36712903-e111-4fb5-a600-9b666b1d1c9f" containerName="istio-proxy"
Oct 30 19:45:00.178235 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:00.178213    4888 cpu_manager.go:414] "containerMap: removing container" podUID="771fbee7-b699-4a2b-a2cd-a2f78bc8e198" containerName="payload-processing"
Oct 30 19:45:00.178235 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:00.178217    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="771fbee7-b699-4a2b-a2cd-a2f78bc8e198" containerName="payload-processing"
Oct 30 19:45:00.178235 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:00.178223    4888 cpu_manager.go:414] "containerMap: removing container" podUID="a4fd7ad8-cdda-470a-86bf-c0c0df93fda3" containerName="maas-api"
Oct 30 19:45:00.178235 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:00.178227    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a4fd7ad8-cdda-470a-86bf-c0c0df93fda3" containerName="maas-api"
Oct 30 19:45:00.178235 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:00.178233    4888 cpu_manager.go:414] "containerMap: removing container" podUID="36712903-e111-4fb5-a600-9b666b1d1c9f" containerName="istio-proxy"
Oct 30 19:45:00.178235 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:00.178239    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="36712903-e111-4fb5-a600-9b666b1d1c9f" containerName="istio-proxy"
Oct 30 19:45:00.178686 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:00.178253    4888 cpu_manager.go:414] "containerMap: removing container" podUID="bae588a3-e65e-4f9b-b4c4-615aa878741e" containerName="istio-proxy"
Oct 30 19:45:00.178686 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:00.178258    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bae588a3-e65e-4f9b-b4c4-615aa878741e" containerName="istio-proxy"
Oct 30 19:45:00.178686 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:00.178266    4888 cpu_manager.go:414] "containerMap: removing container" podUID="e7d8237d-903d-4136-8e74-b90346f5f619" containerName="payload-pre-processing"
Oct 30 19:45:00.178686 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:00.178270    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e7d8237d-903d-4136-8e74-b90346f5f619" containerName="payload-pre-processing"
Oct 30 19:45:00.178686 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:00.178277    4888 cpu_manager.go:414] "containerMap: removing container" podUID="bae588a3-e65e-4f9b-b4c4-615aa878741e" containerName="istio-proxy"
Oct 30 19:45:00.178686 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:00.178300    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bae588a3-e65e-4f9b-b4c4-615aa878741e" containerName="istio-proxy"
Oct 30 19:45:00.178686 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:00.178311    4888 cpu_manager.go:414] "containerMap: removing container" podUID="50a86a24-23eb-4bd4-abbd-9327c64cc99a" containerName="payload-processing"
Oct 30 19:45:00.178686 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:00.178315    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="50a86a24-23eb-4bd4-abbd-9327c64cc99a" containerName="payload-processing"
Oct 30 19:45:00.178686 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:00.178326    4888 cpu_manager.go:414] "containerMap: removing container" podUID="ddf8afd2-ed32-44bb-87ce-c26cde475f08" containerName="maas-api"
Oct 30 19:45:00.178686 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:00.178330    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ddf8afd2-ed32-44bb-87ce-c26cde475f08" containerName="maas-api"
Oct 30 19:45:00.178686 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:00.178341    4888 cpu_manager.go:414] "containerMap: removing container" podUID="5d128a48-cd8d-4a3f-ab1b-1f007ae62e5d" containerName="payload-processing"
Oct 30 19:45:00.178686 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:00.178345    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5d128a48-cd8d-4a3f-ab1b-1f007ae62e5d" containerName="payload-processing"
Oct 30 19:45:00.178686 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:00.178354    4888 cpu_manager.go:414] "containerMap: removing container" podUID="3160b54f-1df9-48de-80d9-8d8a91d47f56" containerName="main"
Oct 30 19:45:00.178686 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:00.178358    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3160b54f-1df9-48de-80d9-8d8a91d47f56" containerName="main"
Oct 30 19:45:00.178686 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:00.178365    4888 cpu_manager.go:414] "containerMap: removing container" podUID="05eefdb3-fc35-4d7a-b3be-20819a7b7aaf" containerName="maas-api"
Oct 30 19:45:00.178686 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:00.178369    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="05eefdb3-fc35-4d7a-b3be-20819a7b7aaf" containerName="maas-api"
Oct 30 19:45:00.178686 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:00.178432    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="e7d8237d-903d-4136-8e74-b90346f5f619" containerName="payload-pre-processing"
Oct 30 19:45:00.178686 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:00.178439    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="bae588a3-e65e-4f9b-b4c4-615aa878741e" containerName="istio-proxy"
Oct 30 19:45:00.178686 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:00.178447    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="50a86a24-23eb-4bd4-abbd-9327c64cc99a" containerName="payload-processing"
Oct 30 19:45:00.178686 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:00.178454    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="ddf8afd2-ed32-44bb-87ce-c26cde475f08" containerName="maas-api"
Oct 30 19:45:00.178686 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:00.178470    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="5d128a48-cd8d-4a3f-ab1b-1f007ae62e5d" containerName="payload-processing"
Oct 30 19:45:00.178686 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:00.178477    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="3160b54f-1df9-48de-80d9-8d8a91d47f56" containerName="main"
Oct 30 19:45:00.178686 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:00.178482    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="05eefdb3-fc35-4d7a-b3be-20819a7b7aaf" containerName="maas-api"
Oct 30 19:45:00.178686 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:00.178488    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="a49754b1-9b01-48d6-8146-a4354554e8b3" containerName="payload-pre-processing"
Oct 30 19:45:00.178686 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:00.178493    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="f1743252-096e-4c09-85a4-e8d60e0709c0" containerName="maas-api"
Oct 30 19:45:00.178686 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:00.178498    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="f49068b0-4dea-487c-a31c-c61704119dad" containerName="istio-proxy"
Oct 30 19:45:00.178686 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:00.178502    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="36712903-e111-4fb5-a600-9b666b1d1c9f" containerName="istio-proxy"
Oct 30 19:45:00.178686 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:00.178507    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="771fbee7-b699-4a2b-a2cd-a2f78bc8e198" containerName="payload-processing"
Oct 30 19:45:00.178686 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:00.178513    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="a4fd7ad8-cdda-470a-86bf-c0c0df93fda3" containerName="maas-api"
Oct 30 19:45:00.183239 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:00.183217    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-5da1ae-29889825-9bbwx"
Oct 30 19:45:00.187680 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:00.187656    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889825-52gzz"]
Oct 30 19:45:00.188120 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:00.188102    4888 cpu_manager.go:414] "containerMap: removing container" podUID="f49068b0-4dea-487c-a31c-c61704119dad" containerName="istio-proxy"
Oct 30 19:45:00.188220 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:00.188122    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f49068b0-4dea-487c-a31c-c61704119dad" containerName="istio-proxy"
Oct 30 19:45:00.188316 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:00.188268    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="bae588a3-e65e-4f9b-b4c4-615aa878741e" containerName="istio-proxy"
Oct 30 19:45:00.188391 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:00.188317    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="f49068b0-4dea-487c-a31c-c61704119dad" containerName="istio-proxy"
Oct 30 19:45:00.188391 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:00.188328    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="36712903-e111-4fb5-a600-9b666b1d1c9f" containerName="istio-proxy"
Oct 30 19:45:00.191568 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:00.191545    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889825-52gzz"
Oct 30 19:45:00.237432 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:00.237405    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-2qg7n\""
Oct 30 19:45:00.268715 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:00.268685    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e31a8f9-dd8c-43f2-a8bf-ac72a0443f22-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-extmod-b-5da1ae-29889825-9bbwx\" (UID: \"0e31a8f9-dd8c-43f2-a8bf-ac72a0443f22\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-5da1ae-29889825-9bbwx"
Oct 30 19:45:00.268715 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:00.268717    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2hkdz\" (UniqueName: \"kubernetes.io/projected/2b6554ea-aeb3-476a-bb73-76f566b54e00-kube-api-access-2hkdz\") pod \"maas-api-key-cleanup-29889825-52gzz\" (UID: \"2b6554ea-aeb3-476a-bb73-76f566b54e00\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889825-52gzz"
Oct 30 19:45:00.268943 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:00.268749    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2b6554ea-aeb3-476a-bb73-76f566b54e00-maas-api-service-ca\") pod \"maas-api-key-cleanup-29889825-52gzz\" (UID: \"2b6554ea-aeb3-476a-bb73-76f566b54e00\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889825-52gzz"
Oct 30 19:45:00.268943 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:00.268912    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wsdtc\" (UniqueName: \"kubernetes.io/projected/0e31a8f9-dd8c-43f2-a8bf-ac72a0443f22-kube-api-access-wsdtc\") pod \"maas-api-key-cleanup-e2e-extmod-b-5da1ae-29889825-9bbwx\" (UID: \"0e31a8f9-dd8c-43f2-a8bf-ac72a0443f22\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-5da1ae-29889825-9bbwx"
Oct 30 19:45:00.291737 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:00.288270    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 30 19:45:00.291737 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:00.288367    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 30 19:45:00.291737 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:00.291524    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-5da1ae-29889825-9bbwx"]
Oct 30 19:45:00.294012 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:00.293988    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889825-52gzz"]
Oct 30 19:45:00.369911 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:00.369876    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2b6554ea-aeb3-476a-bb73-76f566b54e00-maas-api-service-ca\") pod \"maas-api-key-cleanup-29889825-52gzz\" (UID: \"2b6554ea-aeb3-476a-bb73-76f566b54e00\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889825-52gzz"
Oct 30 19:45:00.370152 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:00.370015    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wsdtc\" (UniqueName: \"kubernetes.io/projected/0e31a8f9-dd8c-43f2-a8bf-ac72a0443f22-kube-api-access-wsdtc\") pod \"maas-api-key-cleanup-e2e-extmod-b-5da1ae-29889825-9bbwx\" (UID: \"0e31a8f9-dd8c-43f2-a8bf-ac72a0443f22\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-5da1ae-29889825-9bbwx"
Oct 30 19:45:00.370152 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:00.370036    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e31a8f9-dd8c-43f2-a8bf-ac72a0443f22-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-extmod-b-5da1ae-29889825-9bbwx\" (UID: \"0e31a8f9-dd8c-43f2-a8bf-ac72a0443f22\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-5da1ae-29889825-9bbwx"
Oct 30 19:45:00.370152 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:00.370060    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2hkdz\" (UniqueName: \"kubernetes.io/projected/2b6554ea-aeb3-476a-bb73-76f566b54e00-kube-api-access-2hkdz\") pod \"maas-api-key-cleanup-29889825-52gzz\" (UID: \"2b6554ea-aeb3-476a-bb73-76f566b54e00\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889825-52gzz"
Oct 30 19:45:00.370767 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:00.370742    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2b6554ea-aeb3-476a-bb73-76f566b54e00-maas-api-service-ca\") pod \"maas-api-key-cleanup-29889825-52gzz\" (UID: \"2b6554ea-aeb3-476a-bb73-76f566b54e00\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889825-52gzz"
Oct 30 19:45:00.370856 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:00.370798    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e31a8f9-dd8c-43f2-a8bf-ac72a0443f22-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-extmod-b-5da1ae-29889825-9bbwx\" (UID: \"0e31a8f9-dd8c-43f2-a8bf-ac72a0443f22\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-5da1ae-29889825-9bbwx"
Oct 30 19:45:00.433495 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:00.433420    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2hkdz\" (UniqueName: \"kubernetes.io/projected/2b6554ea-aeb3-476a-bb73-76f566b54e00-kube-api-access-2hkdz\") pod \"maas-api-key-cleanup-29889825-52gzz\" (UID: \"2b6554ea-aeb3-476a-bb73-76f566b54e00\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889825-52gzz"
Oct 30 19:45:00.433495 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:00.433454    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wsdtc\" (UniqueName: \"kubernetes.io/projected/0e31a8f9-dd8c-43f2-a8bf-ac72a0443f22-kube-api-access-wsdtc\") pod \"maas-api-key-cleanup-e2e-extmod-b-5da1ae-29889825-9bbwx\" (UID: \"0e31a8f9-dd8c-43f2-a8bf-ac72a0443f22\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-5da1ae-29889825-9bbwx"
Oct 30 19:45:00.498137 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:00.498104    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-5da1ae-29889825-9bbwx"
Oct 30 19:45:00.506595 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:00.506567    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889825-52gzz"
Oct 30 19:45:00.901349 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:00.901320    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-5da1ae-29889825-9bbwx"]
Oct 30 19:45:00.902572 ip-10-0-134-46 kubenswrapper[4888]: W1030 19:45:00.902550    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e31a8f9_dd8c_43f2_a8bf_ac72a0443f22.slice/crio-2c726f9185f655510753ef49a8b4f21106fa1d0680bfce0763c4a6fc631ed97c WatchSource:0}: Error finding container 2c726f9185f655510753ef49a8b4f21106fa1d0680bfce0763c4a6fc631ed97c: Status 404 returned error can't find the container with id 2c726f9185f655510753ef49a8b4f21106fa1d0680bfce0763c4a6fc631ed97c
Oct 30 19:45:00.940570 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:00.940549    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889825-52gzz"]
Oct 30 19:45:00.942061 ip-10-0-134-46 kubenswrapper[4888]: W1030 19:45:00.942034    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b6554ea_aeb3_476a_bb73_76f566b54e00.slice/crio-038a7d3db5e75594275f15247302a3d4f7b0ee2096aa98aaea5cc5050c8d19ea WatchSource:0}: Error finding container 038a7d3db5e75594275f15247302a3d4f7b0ee2096aa98aaea5cc5050c8d19ea: Status 404 returned error can't find the container with id 038a7d3db5e75594275f15247302a3d4f7b0ee2096aa98aaea5cc5050c8d19ea
Oct 30 19:45:01.448672 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:01.448591    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-5da1ae-29889825-9bbwx" event={"ID":"0e31a8f9-dd8c-43f2-a8bf-ac72a0443f22","Type":"ContainerStarted","Data":"f2ba537045a95e70f32a87a9c6932f6034cb37514754dabf9512d533f33c358a"}
Oct 30 19:45:01.448672 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:01.448661    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-5da1ae-29889825-9bbwx" event={"ID":"0e31a8f9-dd8c-43f2-a8bf-ac72a0443f22","Type":"ContainerStarted","Data":"2c726f9185f655510753ef49a8b4f21106fa1d0680bfce0763c4a6fc631ed97c"}
Oct 30 19:45:01.450234 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:01.450212    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889825-52gzz" event={"ID":"2b6554ea-aeb3-476a-bb73-76f566b54e00","Type":"ContainerStarted","Data":"c0733caf599f573fdc4c83c65d9563358eba62b9196345d06d8402620424ec32"}
Oct 30 19:45:01.450234 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:01.450249    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889825-52gzz" event={"ID":"2b6554ea-aeb3-476a-bb73-76f566b54e00","Type":"ContainerStarted","Data":"038a7d3db5e75594275f15247302a3d4f7b0ee2096aa98aaea5cc5050c8d19ea"}
Oct 30 19:45:01.567688 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:01.567627    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-5da1ae-29889825-9bbwx" podStartSLOduration=1.567608318 podStartE2EDuration="1.567608318s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:45:01.565768455 +0000 UTC m=+4265.813174520" watchObservedRunningTime="2026-10-30 19:45:01.567608318 +0000 UTC m=+4265.815014345"
Oct 30 19:45:01.695169 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:01.695118    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889825-52gzz" podStartSLOduration=1.695103456 podStartE2EDuration="1.695103456s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:45:01.693718429 +0000 UTC m=+4265.941124455" watchObservedRunningTime="2026-10-30 19:45:01.695103456 +0000 UTC m=+4265.942509525"
Oct 30 19:45:02.398187 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:02.398154    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-a-98b389-openshift-default-57c5bc4c69-t5g6x"]
Oct 30 19:45:02.398419 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:02.398396    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-extmod-a-98b389-openshift-default-57c5bc4c69-t5g6x" podUID="9f8e71e3-8cd2-4213-952e-e9db1194df27" containerName="istio-proxy" containerID="cri-o://c86359bd4f06848e4ea92cc55c6e73074f2feadbc0f5b617b62e908c4a1ac03e" gracePeriod=30
Oct 30 19:45:02.458737 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:02.458701    4888 generic.go:356] "Generic (PLEG): container finished" podID="2b6554ea-aeb3-476a-bb73-76f566b54e00" containerID="c0733caf599f573fdc4c83c65d9563358eba62b9196345d06d8402620424ec32" exitCode=60
Oct 30 19:45:02.459196 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:02.458787    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889825-52gzz" event={"ID":"2b6554ea-aeb3-476a-bb73-76f566b54e00","Type":"ContainerDied","Data":"c0733caf599f573fdc4c83c65d9563358eba62b9196345d06d8402620424ec32"}
Oct 30 19:45:02.459196 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:02.459070    4888 scope.go:122] "RemoveContainer" containerID="c0733caf599f573fdc4c83c65d9563358eba62b9196345d06d8402620424ec32"
Oct 30 19:45:02.460468 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:02.460439    4888 generic.go:356] "Generic (PLEG): container finished" podID="0e31a8f9-dd8c-43f2-a8bf-ac72a0443f22" containerID="f2ba537045a95e70f32a87a9c6932f6034cb37514754dabf9512d533f33c358a" exitCode=60
Oct 30 19:45:02.460468 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:02.460465    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-5da1ae-29889825-9bbwx" event={"ID":"0e31a8f9-dd8c-43f2-a8bf-ac72a0443f22","Type":"ContainerDied","Data":"f2ba537045a95e70f32a87a9c6932f6034cb37514754dabf9512d533f33c358a"}
Oct 30 19:45:02.460719 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:02.460702    4888 scope.go:122] "RemoveContainer" containerID="f2ba537045a95e70f32a87a9c6932f6034cb37514754dabf9512d533f33c358a"
Oct 30 19:45:03.469116 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:03.469081    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889825-52gzz" event={"ID":"2b6554ea-aeb3-476a-bb73-76f566b54e00","Type":"ContainerStarted","Data":"d61c9a84836f966362e6a9f876c544af2692fb6a4494ff4996bcc8fe1356e4e9"}
Oct 30 19:45:03.471541 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:03.471505    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-5da1ae-29889825-9bbwx" event={"ID":"0e31a8f9-dd8c-43f2-a8bf-ac72a0443f22","Type":"ContainerStarted","Data":"b76da3a2968cdbdb669435c8ea1ebeedd20721894e51c2261019eb0fa2db2f5a"}
Oct 30 19:45:03.586764 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:03.586726    4888 patch_prober.go:29] interesting pod/e2e-extmod-a-98b389-openshift-default-57c5bc4c69-t5g6x container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 19:45:03.586931 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:03.586793    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-extmod-a-98b389-openshift-default-57c5bc4c69-t5g6x" podUID="9f8e71e3-8cd2-4213-952e-e9db1194df27" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 19:45:04.478599 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:04.478560    4888 generic.go:356] "Generic (PLEG): container finished" podID="2b6554ea-aeb3-476a-bb73-76f566b54e00" containerID="d61c9a84836f966362e6a9f876c544af2692fb6a4494ff4996bcc8fe1356e4e9" exitCode=60
Oct 30 19:45:04.479199 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:04.478653    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889825-52gzz" event={"ID":"2b6554ea-aeb3-476a-bb73-76f566b54e00","Type":"ContainerDied","Data":"d61c9a84836f966362e6a9f876c544af2692fb6a4494ff4996bcc8fe1356e4e9"}
Oct 30 19:45:04.479199 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:04.478715    4888 scope.go:122] "RemoveContainer" containerID="c0733caf599f573fdc4c83c65d9563358eba62b9196345d06d8402620424ec32"
Oct 30 19:45:04.479199 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:04.478970    4888 scope.go:122] "RemoveContainer" containerID="d61c9a84836f966362e6a9f876c544af2692fb6a4494ff4996bcc8fe1356e4e9"
Oct 30 19:45:04.479417 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:45:04.479260    4888 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-29889825-52gzz_odh-ai-gateway-infra(2b6554ea-aeb3-476a-bb73-76f566b54e00)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889825-52gzz" podUID="2b6554ea-aeb3-476a-bb73-76f566b54e00"
Oct 30 19:45:04.480627 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:04.480605    4888 generic.go:356] "Generic (PLEG): container finished" podID="0e31a8f9-dd8c-43f2-a8bf-ac72a0443f22" containerID="b76da3a2968cdbdb669435c8ea1ebeedd20721894e51c2261019eb0fa2db2f5a" exitCode=60
Oct 30 19:45:04.480731 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:04.480693    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-5da1ae-29889825-9bbwx" event={"ID":"0e31a8f9-dd8c-43f2-a8bf-ac72a0443f22","Type":"ContainerDied","Data":"b76da3a2968cdbdb669435c8ea1ebeedd20721894e51c2261019eb0fa2db2f5a"}
Oct 30 19:45:04.480895 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:04.480882    4888 scope.go:122] "RemoveContainer" containerID="b76da3a2968cdbdb669435c8ea1ebeedd20721894e51c2261019eb0fa2db2f5a"
Oct 30 19:45:04.481117 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:45:04.481102    4888 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-extmod-b-5da1ae-29889825-9bbwx_odh-ai-gateway-infra(0e31a8f9-dd8c-43f2-a8bf-ac72a0443f22)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-5da1ae-29889825-9bbwx" podUID="0e31a8f9-dd8c-43f2-a8bf-ac72a0443f22"
Oct 30 19:45:04.490576 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:04.490551    4888 scope.go:122] "RemoveContainer" containerID="f2ba537045a95e70f32a87a9c6932f6034cb37514754dabf9512d533f33c358a"
Oct 30 19:45:05.489691 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:05.489612    4888 scope.go:122] "RemoveContainer" containerID="d61c9a84836f966362e6a9f876c544af2692fb6a4494ff4996bcc8fe1356e4e9"
Oct 30 19:45:05.490158 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:45:05.489879    4888 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-29889825-52gzz_odh-ai-gateway-infra(2b6554ea-aeb3-476a-bb73-76f566b54e00)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889825-52gzz" podUID="2b6554ea-aeb3-476a-bb73-76f566b54e00"
Oct 30 19:45:05.491275 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:05.491255    4888 scope.go:122] "RemoveContainer" containerID="b76da3a2968cdbdb669435c8ea1ebeedd20721894e51c2261019eb0fa2db2f5a"
Oct 30 19:45:05.491546 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:45:05.491524    4888 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-extmod-b-5da1ae-29889825-9bbwx_odh-ai-gateway-infra(0e31a8f9-dd8c-43f2-a8bf-ac72a0443f22)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-5da1ae-29889825-9bbwx" podUID="0e31a8f9-dd8c-43f2-a8bf-ac72a0443f22"
Oct 30 19:45:05.620828 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:45:05.620787    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-jbsw6" containerName="extractor"
Oct 30 19:45:07.661096 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:07.661036    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-98b389-openshift-default-57c5bc4c69-t5g6x"
Oct 30 19:45:07.752224 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:07.752186    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/9f8e71e3-8cd2-4213-952e-e9db1194df27-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9f8e71e3-8cd2-4213-952e-e9db1194df27-istio-podinfo\") pod \"9f8e71e3-8cd2-4213-952e-e9db1194df27\" (UID: \"9f8e71e3-8cd2-4213-952e-e9db1194df27\") "
Oct 30 19:45:07.752434 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:07.752244    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9f8e71e3-8cd2-4213-952e-e9db1194df27-istio-token\" (UniqueName: \"kubernetes.io/projected/9f8e71e3-8cd2-4213-952e-e9db1194df27-istio-token\") pod \"9f8e71e3-8cd2-4213-952e-e9db1194df27\" (UID: \"9f8e71e3-8cd2-4213-952e-e9db1194df27\") "
Oct 30 19:45:07.752434 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:07.752272    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9f8e71e3-8cd2-4213-952e-e9db1194df27-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f8e71e3-8cd2-4213-952e-e9db1194df27-credential-socket\") pod \"9f8e71e3-8cd2-4213-952e-e9db1194df27\" (UID: \"9f8e71e3-8cd2-4213-952e-e9db1194df27\") "
Oct 30 19:45:07.752434 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:07.752321    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9f8e71e3-8cd2-4213-952e-e9db1194df27-kube-api-access-4blg2\" (UniqueName: \"kubernetes.io/projected/9f8e71e3-8cd2-4213-952e-e9db1194df27-kube-api-access-4blg2\") pod \"9f8e71e3-8cd2-4213-952e-e9db1194df27\" (UID: \"9f8e71e3-8cd2-4213-952e-e9db1194df27\") "
Oct 30 19:45:07.752434 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:07.752350    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9f8e71e3-8cd2-4213-952e-e9db1194df27-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f8e71e3-8cd2-4213-952e-e9db1194df27-workload-socket\") pod \"9f8e71e3-8cd2-4213-952e-e9db1194df27\" (UID: \"9f8e71e3-8cd2-4213-952e-e9db1194df27\") "
Oct 30 19:45:07.752434 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:07.752411    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9f8e71e3-8cd2-4213-952e-e9db1194df27-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9f8e71e3-8cd2-4213-952e-e9db1194df27-istio-envoy\") pod \"9f8e71e3-8cd2-4213-952e-e9db1194df27\" (UID: \"9f8e71e3-8cd2-4213-952e-e9db1194df27\") "
Oct 30 19:45:07.752704 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:07.752471    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9f8e71e3-8cd2-4213-952e-e9db1194df27-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9f8e71e3-8cd2-4213-952e-e9db1194df27-istiod-ca-cert\") pod \"9f8e71e3-8cd2-4213-952e-e9db1194df27\" (UID: \"9f8e71e3-8cd2-4213-952e-e9db1194df27\") "
Oct 30 19:45:07.752704 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:07.752522    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9f8e71e3-8cd2-4213-952e-e9db1194df27-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9f8e71e3-8cd2-4213-952e-e9db1194df27-istio-data\") pod \"9f8e71e3-8cd2-4213-952e-e9db1194df27\" (UID: \"9f8e71e3-8cd2-4213-952e-e9db1194df27\") "
Oct 30 19:45:07.752704 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:07.752576    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9f8e71e3-8cd2-4213-952e-e9db1194df27-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9f8e71e3-8cd2-4213-952e-e9db1194df27-workload-certs\") pod \"9f8e71e3-8cd2-4213-952e-e9db1194df27\" (UID: \"9f8e71e3-8cd2-4213-952e-e9db1194df27\") "
Oct 30 19:45:07.752704 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:07.752605    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f8e71e3-8cd2-4213-952e-e9db1194df27-credential-socket" pod "9f8e71e3-8cd2-4213-952e-e9db1194df27" (UID: "9f8e71e3-8cd2-4213-952e-e9db1194df27"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:45:07.752704 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:07.752680    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f8e71e3-8cd2-4213-952e-e9db1194df27-workload-socket" pod "9f8e71e3-8cd2-4213-952e-e9db1194df27" (UID: "9f8e71e3-8cd2-4213-952e-e9db1194df27"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:45:07.752981 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:07.752865    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f8e71e3-8cd2-4213-952e-e9db1194df27-workload-certs" pod "9f8e71e3-8cd2-4213-952e-e9db1194df27" (UID: "9f8e71e3-8cd2-4213-952e-e9db1194df27"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:45:07.752981 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:07.752929    4888 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f8e71e3-8cd2-4213-952e-e9db1194df27-credential-socket\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:45:07.752981 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:07.752943    4888 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f8e71e3-8cd2-4213-952e-e9db1194df27-workload-socket\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:45:07.753145 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:07.753048    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f8e71e3-8cd2-4213-952e-e9db1194df27-istio-data" pod "9f8e71e3-8cd2-4213-952e-e9db1194df27" (UID: "9f8e71e3-8cd2-4213-952e-e9db1194df27"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:45:07.753233 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:07.752964    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9f8e71e3-8cd2-4213-952e-e9db1194df27-istiod-ca-cert" pod "9f8e71e3-8cd2-4213-952e-e9db1194df27" (UID: "9f8e71e3-8cd2-4213-952e-e9db1194df27"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:45:07.754743 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:07.754714    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9f8e71e3-8cd2-4213-952e-e9db1194df27-istio-podinfo" pod "9f8e71e3-8cd2-4213-952e-e9db1194df27" (UID: "9f8e71e3-8cd2-4213-952e-e9db1194df27"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:45:07.754904 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:07.754885    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f8e71e3-8cd2-4213-952e-e9db1194df27-istio-envoy" pod "9f8e71e3-8cd2-4213-952e-e9db1194df27" (UID: "9f8e71e3-8cd2-4213-952e-e9db1194df27"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:45:07.755197 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:07.755175    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9f8e71e3-8cd2-4213-952e-e9db1194df27-kube-api-access-4blg2" pod "9f8e71e3-8cd2-4213-952e-e9db1194df27" (UID: "9f8e71e3-8cd2-4213-952e-e9db1194df27"). InnerVolumeSpecName "kube-api-access-4blg2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:45:07.755319 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:07.755198    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9f8e71e3-8cd2-4213-952e-e9db1194df27-istio-token" pod "9f8e71e3-8cd2-4213-952e-e9db1194df27" (UID: "9f8e71e3-8cd2-4213-952e-e9db1194df27"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:45:07.853625 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:07.853594    4888 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9f8e71e3-8cd2-4213-952e-e9db1194df27-istio-token\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:45:07.853799 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:07.853630    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4blg2\" (UniqueName: \"kubernetes.io/projected/9f8e71e3-8cd2-4213-952e-e9db1194df27-kube-api-access-4blg2\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:45:07.853799 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:07.853650    4888 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9f8e71e3-8cd2-4213-952e-e9db1194df27-istio-envoy\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:45:07.853799 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:07.853665    4888 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9f8e71e3-8cd2-4213-952e-e9db1194df27-istiod-ca-cert\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:45:07.853799 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:07.853680    4888 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9f8e71e3-8cd2-4213-952e-e9db1194df27-istio-data\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:45:07.853799 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:07.853694    4888 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9f8e71e3-8cd2-4213-952e-e9db1194df27-workload-certs\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:45:07.853799 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:07.853709    4888 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9f8e71e3-8cd2-4213-952e-e9db1194df27-istio-podinfo\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:45:08.510655 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:08.510621    4888 generic.go:356] "Generic (PLEG): container finished" podID="9f8e71e3-8cd2-4213-952e-e9db1194df27" containerID="c86359bd4f06848e4ea92cc55c6e73074f2feadbc0f5b617b62e908c4a1ac03e" exitCode=0
Oct 30 19:45:08.510835 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:08.510675    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-98b389-openshift-default-57c5bc4c69-t5g6x"
Oct 30 19:45:08.510835 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:08.510715    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-98b389-openshift-default-57c5bc4c69-t5g6x" event={"ID":"9f8e71e3-8cd2-4213-952e-e9db1194df27","Type":"ContainerDied","Data":"c86359bd4f06848e4ea92cc55c6e73074f2feadbc0f5b617b62e908c4a1ac03e"}
Oct 30 19:45:08.510835 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:08.510777    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-98b389-openshift-default-57c5bc4c69-t5g6x" event={"ID":"9f8e71e3-8cd2-4213-952e-e9db1194df27","Type":"ContainerDied","Data":"24b18465771fe148707f05ae67bd794970309b1ba8e4e0b843e99383b611f0b4"}
Oct 30 19:45:08.510835 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:08.510809    4888 scope.go:122] "RemoveContainer" containerID="c86359bd4f06848e4ea92cc55c6e73074f2feadbc0f5b617b62e908c4a1ac03e"
Oct 30 19:45:08.520782 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:08.520631    4888 scope.go:122] "RemoveContainer" containerID="dfe03ddaf9ba723ebdb7d76350970820fb9dd9e8f07360d24e20420d2a1936b9"
Oct 30 19:45:08.529972 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:08.529955    4888 scope.go:122] "RemoveContainer" containerID="c86359bd4f06848e4ea92cc55c6e73074f2feadbc0f5b617b62e908c4a1ac03e"
Oct 30 19:45:08.530225 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:45:08.530197    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c86359bd4f06848e4ea92cc55c6e73074f2feadbc0f5b617b62e908c4a1ac03e\": container with ID starting with c86359bd4f06848e4ea92cc55c6e73074f2feadbc0f5b617b62e908c4a1ac03e not found: ID does not exist" containerID="c86359bd4f06848e4ea92cc55c6e73074f2feadbc0f5b617b62e908c4a1ac03e"
Oct 30 19:45:08.530274 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:08.530229    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c86359bd4f06848e4ea92cc55c6e73074f2feadbc0f5b617b62e908c4a1ac03e"} err="failed to get container status \"c86359bd4f06848e4ea92cc55c6e73074f2feadbc0f5b617b62e908c4a1ac03e\": rpc error: code = NotFound desc = could not find container \"c86359bd4f06848e4ea92cc55c6e73074f2feadbc0f5b617b62e908c4a1ac03e\": container with ID starting with c86359bd4f06848e4ea92cc55c6e73074f2feadbc0f5b617b62e908c4a1ac03e not found: ID does not exist"
Oct 30 19:45:08.530274 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:08.530247    4888 scope.go:122] "RemoveContainer" containerID="dfe03ddaf9ba723ebdb7d76350970820fb9dd9e8f07360d24e20420d2a1936b9"
Oct 30 19:45:08.530511 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:45:08.530494    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dfe03ddaf9ba723ebdb7d76350970820fb9dd9e8f07360d24e20420d2a1936b9\": container with ID starting with dfe03ddaf9ba723ebdb7d76350970820fb9dd9e8f07360d24e20420d2a1936b9 not found: ID does not exist" containerID="dfe03ddaf9ba723ebdb7d76350970820fb9dd9e8f07360d24e20420d2a1936b9"
Oct 30 19:45:08.530563 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:08.530519    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dfe03ddaf9ba723ebdb7d76350970820fb9dd9e8f07360d24e20420d2a1936b9"} err="failed to get container status \"dfe03ddaf9ba723ebdb7d76350970820fb9dd9e8f07360d24e20420d2a1936b9\": rpc error: code = NotFound desc = could not find container \"dfe03ddaf9ba723ebdb7d76350970820fb9dd9e8f07360d24e20420d2a1936b9\": container with ID starting with dfe03ddaf9ba723ebdb7d76350970820fb9dd9e8f07360d24e20420d2a1936b9 not found: ID does not exist"
Oct 30 19:45:08.731222 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:08.731187    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-a-98b389-openshift-default-57c5bc4c69-t5g6x"]
Oct 30 19:45:08.782012 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:08.781937    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-extmod-a-98b389-openshift-default-57c5bc4c69-t5g6x"]
Oct 30 19:45:09.889036 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:09.889005    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9f8e71e3-8cd2-4213-952e-e9db1194df27" path="/var/lib/kubelet/pods/9f8e71e3-8cd2-4213-952e-e9db1194df27/volumes"
Oct 30 19:45:14.883605 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:14.883579    4888 scope.go:122] "RemoveContainer" containerID="b76da3a2968cdbdb669435c8ea1ebeedd20721894e51c2261019eb0fa2db2f5a"
Oct 30 19:45:15.552422 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:15.552328    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-5da1ae-29889825-9bbwx" event={"ID":"0e31a8f9-dd8c-43f2-a8bf-ac72a0443f22","Type":"ContainerStarted","Data":"dd191814cf4c53b80df56413269d4277686d08e732cc71f202e5e8ed1aa3c169"}
Oct 30 19:45:16.560364 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:16.560335    4888 generic.go:356] "Generic (PLEG): container finished" podID="0e31a8f9-dd8c-43f2-a8bf-ac72a0443f22" containerID="dd191814cf4c53b80df56413269d4277686d08e732cc71f202e5e8ed1aa3c169" exitCode=60
Oct 30 19:45:16.560801 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:16.560419    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-5da1ae-29889825-9bbwx" event={"ID":"0e31a8f9-dd8c-43f2-a8bf-ac72a0443f22","Type":"ContainerDied","Data":"dd191814cf4c53b80df56413269d4277686d08e732cc71f202e5e8ed1aa3c169"}
Oct 30 19:45:16.560801 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:16.560485    4888 scope.go:122] "RemoveContainer" containerID="b76da3a2968cdbdb669435c8ea1ebeedd20721894e51c2261019eb0fa2db2f5a"
Oct 30 19:45:16.560801 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:16.560778    4888 scope.go:122] "RemoveContainer" containerID="dd191814cf4c53b80df56413269d4277686d08e732cc71f202e5e8ed1aa3c169"
Oct 30 19:45:16.561041 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:45:16.561019    4888 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-extmod-b-5da1ae-29889825-9bbwx_odh-ai-gateway-infra(0e31a8f9-dd8c-43f2-a8bf-ac72a0443f22)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-5da1ae-29889825-9bbwx" podUID="0e31a8f9-dd8c-43f2-a8bf-ac72a0443f22"
Oct 30 19:45:16.684902 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:16.684862    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-5da1ae-29889825-9bbwx"]
Oct 30 19:45:17.707404 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:17.707371    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-5da1ae-29889825-9bbwx"
Oct 30 19:45:17.752121 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:17.752089    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0e31a8f9-dd8c-43f2-a8bf-ac72a0443f22-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e31a8f9-dd8c-43f2-a8bf-ac72a0443f22-maas-api-service-ca\") pod \"0e31a8f9-dd8c-43f2-a8bf-ac72a0443f22\" (UID: \"0e31a8f9-dd8c-43f2-a8bf-ac72a0443f22\") "
Oct 30 19:45:17.752334 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:17.752189    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0e31a8f9-dd8c-43f2-a8bf-ac72a0443f22-kube-api-access-wsdtc\" (UniqueName: \"kubernetes.io/projected/0e31a8f9-dd8c-43f2-a8bf-ac72a0443f22-kube-api-access-wsdtc\") pod \"0e31a8f9-dd8c-43f2-a8bf-ac72a0443f22\" (UID: \"0e31a8f9-dd8c-43f2-a8bf-ac72a0443f22\") "
Oct 30 19:45:17.752476 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:17.752452    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0e31a8f9-dd8c-43f2-a8bf-ac72a0443f22-maas-api-service-ca" pod "0e31a8f9-dd8c-43f2-a8bf-ac72a0443f22" (UID: "0e31a8f9-dd8c-43f2-a8bf-ac72a0443f22"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:45:17.754426 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:17.754397    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0e31a8f9-dd8c-43f2-a8bf-ac72a0443f22-kube-api-access-wsdtc" pod "0e31a8f9-dd8c-43f2-a8bf-ac72a0443f22" (UID: "0e31a8f9-dd8c-43f2-a8bf-ac72a0443f22"). InnerVolumeSpecName "kube-api-access-wsdtc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:45:17.853813 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:17.853783    4888 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e31a8f9-dd8c-43f2-a8bf-ac72a0443f22-maas-api-service-ca\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:45:17.853813 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:17.853812    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wsdtc\" (UniqueName: \"kubernetes.io/projected/0e31a8f9-dd8c-43f2-a8bf-ac72a0443f22-kube-api-access-wsdtc\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:45:17.883220 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:17.883191    4888 scope.go:122] "RemoveContainer" containerID="d61c9a84836f966362e6a9f876c544af2692fb6a4494ff4996bcc8fe1356e4e9"
Oct 30 19:45:18.575143 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:18.575118    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-5da1ae-29889825-9bbwx"
Oct 30 19:45:18.575342 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:18.575152    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-5da1ae-29889825-9bbwx" event={"ID":"0e31a8f9-dd8c-43f2-a8bf-ac72a0443f22","Type":"ContainerDied","Data":"2c726f9185f655510753ef49a8b4f21106fa1d0680bfce0763c4a6fc631ed97c"}
Oct 30 19:45:18.575342 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:18.575221    4888 scope.go:122] "RemoveContainer" containerID="dd191814cf4c53b80df56413269d4277686d08e732cc71f202e5e8ed1aa3c169"
Oct 30 19:45:18.577543 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:18.577512    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889825-52gzz" event={"ID":"2b6554ea-aeb3-476a-bb73-76f566b54e00","Type":"ContainerStarted","Data":"badbfaa51b12d0aba8de0f384671fce458e63468dbdec0fa5b6bea5a0f6d55b7"}
Oct 30 19:45:18.742785 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:18.742744    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-5da1ae-29889825-9bbwx"]
Oct 30 19:45:18.793765 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:18.793728    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-5da1ae-29889825-9bbwx"]
Oct 30 19:45:19.585450 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:19.585420    4888 generic.go:356] "Generic (PLEG): container finished" podID="2b6554ea-aeb3-476a-bb73-76f566b54e00" containerID="badbfaa51b12d0aba8de0f384671fce458e63468dbdec0fa5b6bea5a0f6d55b7" exitCode=60
Oct 30 19:45:19.585656 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:19.585493    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889825-52gzz" event={"ID":"2b6554ea-aeb3-476a-bb73-76f566b54e00","Type":"ContainerDied","Data":"badbfaa51b12d0aba8de0f384671fce458e63468dbdec0fa5b6bea5a0f6d55b7"}
Oct 30 19:45:19.585656 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:19.585549    4888 scope.go:122] "RemoveContainer" containerID="d61c9a84836f966362e6a9f876c544af2692fb6a4494ff4996bcc8fe1356e4e9"
Oct 30 19:45:19.585905 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:19.585888    4888 scope.go:122] "RemoveContainer" containerID="badbfaa51b12d0aba8de0f384671fce458e63468dbdec0fa5b6bea5a0f6d55b7"
Oct 30 19:45:19.586163 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:45:19.586142    4888 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-29889825-52gzz_odh-ai-gateway-infra(2b6554ea-aeb3-476a-bb73-76f566b54e00)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889825-52gzz" podUID="2b6554ea-aeb3-476a-bb73-76f566b54e00"
Oct 30 19:45:19.875543 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:19.875455    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889825-52gzz"]
Oct 30 19:45:19.889655 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:19.889626    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0e31a8f9-dd8c-43f2-a8bf-ac72a0443f22" path="/var/lib/kubelet/pods/0e31a8f9-dd8c-43f2-a8bf-ac72a0443f22/volumes"
Oct 30 19:45:20.736525 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:20.736498    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889825-52gzz"
Oct 30 19:45:20.778598 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:20.778569    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2b6554ea-aeb3-476a-bb73-76f566b54e00-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2b6554ea-aeb3-476a-bb73-76f566b54e00-maas-api-service-ca\") pod \"2b6554ea-aeb3-476a-bb73-76f566b54e00\" (UID: \"2b6554ea-aeb3-476a-bb73-76f566b54e00\") "
Oct 30 19:45:20.778764 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:20.778679    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2b6554ea-aeb3-476a-bb73-76f566b54e00-kube-api-access-2hkdz\" (UniqueName: \"kubernetes.io/projected/2b6554ea-aeb3-476a-bb73-76f566b54e00-kube-api-access-2hkdz\") pod \"2b6554ea-aeb3-476a-bb73-76f566b54e00\" (UID: \"2b6554ea-aeb3-476a-bb73-76f566b54e00\") "
Oct 30 19:45:20.778922 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:20.778902    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2b6554ea-aeb3-476a-bb73-76f566b54e00-maas-api-service-ca" pod "2b6554ea-aeb3-476a-bb73-76f566b54e00" (UID: "2b6554ea-aeb3-476a-bb73-76f566b54e00"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:45:20.780820 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:20.780796    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2b6554ea-aeb3-476a-bb73-76f566b54e00-kube-api-access-2hkdz" pod "2b6554ea-aeb3-476a-bb73-76f566b54e00" (UID: "2b6554ea-aeb3-476a-bb73-76f566b54e00"). InnerVolumeSpecName "kube-api-access-2hkdz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:45:20.880115 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:20.880027    4888 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2b6554ea-aeb3-476a-bb73-76f566b54e00-maas-api-service-ca\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:45:20.880115 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:20.880052    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2hkdz\" (UniqueName: \"kubernetes.io/projected/2b6554ea-aeb3-476a-bb73-76f566b54e00-kube-api-access-2hkdz\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:45:21.599616 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:21.599578    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889825-52gzz"
Oct 30 19:45:21.599861 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:21.599582    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889825-52gzz" event={"ID":"2b6554ea-aeb3-476a-bb73-76f566b54e00","Type":"ContainerDied","Data":"038a7d3db5e75594275f15247302a3d4f7b0ee2096aa98aaea5cc5050c8d19ea"}
Oct 30 19:45:21.599861 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:21.599712    4888 scope.go:122] "RemoveContainer" containerID="badbfaa51b12d0aba8de0f384671fce458e63468dbdec0fa5b6bea5a0f6d55b7"
Oct 30 19:45:21.769960 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:21.769931    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889825-52gzz"]
Oct 30 19:45:21.775415 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:21.775387    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889825-52gzz"]
Oct 30 19:45:21.889169 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:21.889084    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2b6554ea-aeb3-476a-bb73-76f566b54e00" path="/var/lib/kubelet/pods/2b6554ea-aeb3-476a-bb73-76f566b54e00/volumes"
Oct 30 19:45:31.076645 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:31.076606    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-rjmdz"]
Oct 30 19:45:34.920205 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:34.920172    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-b-5da1ae-openshift-default-77c657bb8f-nng6m"]
Oct 30 19:45:34.920653 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:34.920402    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-extmod-b-5da1ae-openshift-default-77c657bb8f-nng6m" podUID="242ace3b-f2c6-4fd5-abe5-31ce3ae9a12c" containerName="istio-proxy" containerID="cri-o://83b2c2def8c2419f26db112578fc3501a1f8ab0e41fecc3728aceff3cf05e7ae" gracePeriod=30
Oct 30 19:45:35.618977 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:45:35.618938    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-jbsw6" containerName="extractor"
Oct 30 19:45:35.953095 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:35.953000    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-e7929d-t75nt"]
Oct 30 19:45:35.953607 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:35.953440    4888 cpu_manager.go:414] "containerMap: removing container" podUID="0e31a8f9-dd8c-43f2-a8bf-ac72a0443f22" containerName="cleanup"
Oct 30 19:45:35.953607 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:35.953452    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0e31a8f9-dd8c-43f2-a8bf-ac72a0443f22" containerName="cleanup"
Oct 30 19:45:35.953607 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:35.953473    4888 cpu_manager.go:414] "containerMap: removing container" podUID="0e31a8f9-dd8c-43f2-a8bf-ac72a0443f22" containerName="cleanup"
Oct 30 19:45:35.953607 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:35.953478    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0e31a8f9-dd8c-43f2-a8bf-ac72a0443f22" containerName="cleanup"
Oct 30 19:45:35.953607 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:35.953485    4888 cpu_manager.go:414] "containerMap: removing container" podUID="0e31a8f9-dd8c-43f2-a8bf-ac72a0443f22" containerName="cleanup"
Oct 30 19:45:35.953607 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:35.953490    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0e31a8f9-dd8c-43f2-a8bf-ac72a0443f22" containerName="cleanup"
Oct 30 19:45:35.953607 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:35.953496    4888 cpu_manager.go:414] "containerMap: removing container" podUID="2b6554ea-aeb3-476a-bb73-76f566b54e00" containerName="cleanup"
Oct 30 19:45:35.953607 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:35.953500    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2b6554ea-aeb3-476a-bb73-76f566b54e00" containerName="cleanup"
Oct 30 19:45:35.953607 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:35.953509    4888 cpu_manager.go:414] "containerMap: removing container" podUID="2b6554ea-aeb3-476a-bb73-76f566b54e00" containerName="cleanup"
Oct 30 19:45:35.953607 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:35.953513    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2b6554ea-aeb3-476a-bb73-76f566b54e00" containerName="cleanup"
Oct 30 19:45:35.953607 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:35.953533    4888 cpu_manager.go:414] "containerMap: removing container" podUID="9f8e71e3-8cd2-4213-952e-e9db1194df27" containerName="istio-proxy"
Oct 30 19:45:35.953607 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:35.953538    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9f8e71e3-8cd2-4213-952e-e9db1194df27" containerName="istio-proxy"
Oct 30 19:45:35.953607 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:35.953547    4888 cpu_manager.go:414] "containerMap: removing container" podUID="9f8e71e3-8cd2-4213-952e-e9db1194df27" containerName="istio-proxy"
Oct 30 19:45:35.953607 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:35.953552    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9f8e71e3-8cd2-4213-952e-e9db1194df27" containerName="istio-proxy"
Oct 30 19:45:35.954275 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:35.953618    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="0e31a8f9-dd8c-43f2-a8bf-ac72a0443f22" containerName="cleanup"
Oct 30 19:45:35.954275 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:35.953625    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="0e31a8f9-dd8c-43f2-a8bf-ac72a0443f22" containerName="cleanup"
Oct 30 19:45:35.954275 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:35.953630    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="2b6554ea-aeb3-476a-bb73-76f566b54e00" containerName="cleanup"
Oct 30 19:45:35.954275 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:35.953636    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="2b6554ea-aeb3-476a-bb73-76f566b54e00" containerName="cleanup"
Oct 30 19:45:35.954275 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:35.953645    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="9f8e71e3-8cd2-4213-952e-e9db1194df27" containerName="istio-proxy"
Oct 30 19:45:35.954275 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:35.953651    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="9f8e71e3-8cd2-4213-952e-e9db1194df27" containerName="istio-proxy"
Oct 30 19:45:35.956865 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:35.956845    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-e7929d-t75nt"
Oct 30 19:45:36.015513 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:36.015471    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-e7929d-t75nt"]
Oct 30 19:45:36.021736 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:36.021709    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/3091e784-c5c0-401c-8b39-d5fd35db06ad-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w4-e7929d-t75nt\" (UID: \"3091e784-c5c0-401c-8b39-d5fd35db06ad\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-e7929d-t75nt"
Oct 30 19:45:36.021917 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:36.021767    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/3091e784-c5c0-401c-8b39-d5fd35db06ad-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w4-e7929d-t75nt\" (UID: \"3091e784-c5c0-401c-8b39-d5fd35db06ad\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-e7929d-t75nt"
Oct 30 19:45:36.021917 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:36.021856    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g6x7w\" (UniqueName: \"kubernetes.io/projected/3091e784-c5c0-401c-8b39-d5fd35db06ad-kube-api-access-g6x7w\") pod \"maas-api-revoke-keys-e2e-worker-w4-e7929d-t75nt\" (UID: \"3091e784-c5c0-401c-8b39-d5fd35db06ad\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-e7929d-t75nt"
Oct 30 19:45:36.063485 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:36.063443    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 30 19:45:36.063485 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:36.063455    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 19:45:36.063714 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:36.063493    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 30 19:45:36.063714 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:36.063529    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-2qg7n\""
Oct 30 19:45:36.123230 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:36.123194    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/3091e784-c5c0-401c-8b39-d5fd35db06ad-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w4-e7929d-t75nt\" (UID: \"3091e784-c5c0-401c-8b39-d5fd35db06ad\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-e7929d-t75nt"
Oct 30 19:45:36.123460 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:36.123321    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g6x7w\" (UniqueName: \"kubernetes.io/projected/3091e784-c5c0-401c-8b39-d5fd35db06ad-kube-api-access-g6x7w\") pod \"maas-api-revoke-keys-e2e-worker-w4-e7929d-t75nt\" (UID: \"3091e784-c5c0-401c-8b39-d5fd35db06ad\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-e7929d-t75nt"
Oct 30 19:45:36.123460 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:36.123383    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/3091e784-c5c0-401c-8b39-d5fd35db06ad-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w4-e7929d-t75nt\" (UID: \"3091e784-c5c0-401c-8b39-d5fd35db06ad\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-e7929d-t75nt"
Oct 30 19:45:36.123997 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:36.123971    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/3091e784-c5c0-401c-8b39-d5fd35db06ad-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w4-e7929d-t75nt\" (UID: \"3091e784-c5c0-401c-8b39-d5fd35db06ad\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-e7929d-t75nt"
Oct 30 19:45:36.127003 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:36.126049    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/3091e784-c5c0-401c-8b39-d5fd35db06ad-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w4-e7929d-t75nt\" (UID: \"3091e784-c5c0-401c-8b39-d5fd35db06ad\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-e7929d-t75nt"
Oct 30 19:45:36.185766 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:36.185739    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g6x7w\" (UniqueName: \"kubernetes.io/projected/3091e784-c5c0-401c-8b39-d5fd35db06ad-kube-api-access-g6x7w\") pod \"maas-api-revoke-keys-e2e-worker-w4-e7929d-t75nt\" (UID: \"3091e784-c5c0-401c-8b39-d5fd35db06ad\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-e7929d-t75nt"
Oct 30 19:45:36.270053 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:36.269981    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-e7929d-t75nt"
Oct 30 19:45:36.646141 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:36.646119    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-e7929d-t75nt"]
Oct 30 19:45:36.647417 ip-10-0-134-46 kubenswrapper[4888]: W1030 19:45:36.647394    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3091e784_c5c0_401c_8b39_d5fd35db06ad.slice/crio-bbecddf4f74241af74829a066e46b28ceb8a262f2bb5884fdc8e6309b1029dd3 WatchSource:0}: Error finding container bbecddf4f74241af74829a066e46b28ceb8a262f2bb5884fdc8e6309b1029dd3: Status 404 returned error can't find the container with id bbecddf4f74241af74829a066e46b28ceb8a262f2bb5884fdc8e6309b1029dd3
Oct 30 19:45:36.686867 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:36.686825    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-e7929d-t75nt" event={"ID":"3091e784-c5c0-401c-8b39-d5fd35db06ad","Type":"ContainerStarted","Data":"bbecddf4f74241af74829a066e46b28ceb8a262f2bb5884fdc8e6309b1029dd3"}
Oct 30 19:45:37.694911 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:37.694872    4888 generic.go:356] "Generic (PLEG): container finished" podID="3091e784-c5c0-401c-8b39-d5fd35db06ad" containerID="8cc44b01b96528a807063a31729f500ff1c80881f3c9bb216c65f403e5643cef" exitCode=0
Oct 30 19:45:37.695370 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:37.694960    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-e7929d-t75nt" event={"ID":"3091e784-c5c0-401c-8b39-d5fd35db06ad","Type":"ContainerDied","Data":"8cc44b01b96528a807063a31729f500ff1c80881f3c9bb216c65f403e5643cef"}
Oct 30 19:45:38.850585 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:38.850561    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-e7929d-t75nt"
Oct 30 19:45:38.951007 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:38.950936    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3091e784-c5c0-401c-8b39-d5fd35db06ad-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/3091e784-c5c0-401c-8b39-d5fd35db06ad-maas-api-openshift-ca\") pod \"3091e784-c5c0-401c-8b39-d5fd35db06ad\" (UID: \"3091e784-c5c0-401c-8b39-d5fd35db06ad\") "
Oct 30 19:45:38.951007 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:38.950986    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3091e784-c5c0-401c-8b39-d5fd35db06ad-kube-api-access-g6x7w\" (UniqueName: \"kubernetes.io/projected/3091e784-c5c0-401c-8b39-d5fd35db06ad-kube-api-access-g6x7w\") pod \"3091e784-c5c0-401c-8b39-d5fd35db06ad\" (UID: \"3091e784-c5c0-401c-8b39-d5fd35db06ad\") "
Oct 30 19:45:38.951231 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:38.951026    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3091e784-c5c0-401c-8b39-d5fd35db06ad-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/3091e784-c5c0-401c-8b39-d5fd35db06ad-maas-api-xks-ca\") pod \"3091e784-c5c0-401c-8b39-d5fd35db06ad\" (UID: \"3091e784-c5c0-401c-8b39-d5fd35db06ad\") "
Oct 30 19:45:38.951648 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:38.951377    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3091e784-c5c0-401c-8b39-d5fd35db06ad-maas-api-openshift-ca" pod "3091e784-c5c0-401c-8b39-d5fd35db06ad" (UID: "3091e784-c5c0-401c-8b39-d5fd35db06ad"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:45:38.951648 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:38.951457    4888 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/3091e784-c5c0-401c-8b39-d5fd35db06ad-maas-api-openshift-ca\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:45:38.953444 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:38.953423    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3091e784-c5c0-401c-8b39-d5fd35db06ad-kube-api-access-g6x7w" pod "3091e784-c5c0-401c-8b39-d5fd35db06ad" (UID: "3091e784-c5c0-401c-8b39-d5fd35db06ad"). InnerVolumeSpecName "kube-api-access-g6x7w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:45:38.953515 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:38.953473    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3091e784-c5c0-401c-8b39-d5fd35db06ad-maas-api-xks-ca" pod "3091e784-c5c0-401c-8b39-d5fd35db06ad" (UID: "3091e784-c5c0-401c-8b39-d5fd35db06ad"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:45:39.051966 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:39.051925    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g6x7w\" (UniqueName: \"kubernetes.io/projected/3091e784-c5c0-401c-8b39-d5fd35db06ad-kube-api-access-g6x7w\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:45:39.051966 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:39.051959    4888 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/3091e784-c5c0-401c-8b39-d5fd35db06ad-maas-api-xks-ca\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:45:39.707797 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:39.707761    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-e7929d-t75nt" event={"ID":"3091e784-c5c0-401c-8b39-d5fd35db06ad","Type":"ContainerDied","Data":"bbecddf4f74241af74829a066e46b28ceb8a262f2bb5884fdc8e6309b1029dd3"}
Oct 30 19:45:39.707960 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:39.707838    4888 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bbecddf4f74241af74829a066e46b28ceb8a262f2bb5884fdc8e6309b1029dd3"
Oct 30 19:45:39.707960 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:39.707776    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-e7929d-t75nt"
Oct 30 19:45:40.188430 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:40.188405    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-5da1ae-openshift-default-77c657bb8f-nng6m"
Oct 30 19:45:40.263551 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:40.263516    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/242ace3b-f2c6-4fd5-abe5-31ce3ae9a12c-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/242ace3b-f2c6-4fd5-abe5-31ce3ae9a12c-credential-socket\") pod \"242ace3b-f2c6-4fd5-abe5-31ce3ae9a12c\" (UID: \"242ace3b-f2c6-4fd5-abe5-31ce3ae9a12c\") "
Oct 30 19:45:40.263749 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:40.263613    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/242ace3b-f2c6-4fd5-abe5-31ce3ae9a12c-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/242ace3b-f2c6-4fd5-abe5-31ce3ae9a12c-workload-certs\") pod \"242ace3b-f2c6-4fd5-abe5-31ce3ae9a12c\" (UID: \"242ace3b-f2c6-4fd5-abe5-31ce3ae9a12c\") "
Oct 30 19:45:40.263749 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:40.263685    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/242ace3b-f2c6-4fd5-abe5-31ce3ae9a12c-kube-api-access-b5shf\" (UniqueName: \"kubernetes.io/projected/242ace3b-f2c6-4fd5-abe5-31ce3ae9a12c-kube-api-access-b5shf\") pod \"242ace3b-f2c6-4fd5-abe5-31ce3ae9a12c\" (UID: \"242ace3b-f2c6-4fd5-abe5-31ce3ae9a12c\") "
Oct 30 19:45:40.263749 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:40.263733    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/242ace3b-f2c6-4fd5-abe5-31ce3ae9a12c-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/242ace3b-f2c6-4fd5-abe5-31ce3ae9a12c-istio-envoy\") pod \"242ace3b-f2c6-4fd5-abe5-31ce3ae9a12c\" (UID: \"242ace3b-f2c6-4fd5-abe5-31ce3ae9a12c\") "
Oct 30 19:45:40.263935 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:40.263763    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/242ace3b-f2c6-4fd5-abe5-31ce3ae9a12c-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/242ace3b-f2c6-4fd5-abe5-31ce3ae9a12c-workload-socket\") pod \"242ace3b-f2c6-4fd5-abe5-31ce3ae9a12c\" (UID: \"242ace3b-f2c6-4fd5-abe5-31ce3ae9a12c\") "
Oct 30 19:45:40.263935 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:40.263791    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/242ace3b-f2c6-4fd5-abe5-31ce3ae9a12c-credential-socket" pod "242ace3b-f2c6-4fd5-abe5-31ce3ae9a12c" (UID: "242ace3b-f2c6-4fd5-abe5-31ce3ae9a12c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:45:40.263935 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:40.263803    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/242ace3b-f2c6-4fd5-abe5-31ce3ae9a12c-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/242ace3b-f2c6-4fd5-abe5-31ce3ae9a12c-istio-podinfo\") pod \"242ace3b-f2c6-4fd5-abe5-31ce3ae9a12c\" (UID: \"242ace3b-f2c6-4fd5-abe5-31ce3ae9a12c\") "
Oct 30 19:45:40.263935 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:40.263854    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/242ace3b-f2c6-4fd5-abe5-31ce3ae9a12c-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/242ace3b-f2c6-4fd5-abe5-31ce3ae9a12c-istiod-ca-cert\") pod \"242ace3b-f2c6-4fd5-abe5-31ce3ae9a12c\" (UID: \"242ace3b-f2c6-4fd5-abe5-31ce3ae9a12c\") "
Oct 30 19:45:40.263935 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:40.263913    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/242ace3b-f2c6-4fd5-abe5-31ce3ae9a12c-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/242ace3b-f2c6-4fd5-abe5-31ce3ae9a12c-istio-data\") pod \"242ace3b-f2c6-4fd5-abe5-31ce3ae9a12c\" (UID: \"242ace3b-f2c6-4fd5-abe5-31ce3ae9a12c\") "
Oct 30 19:45:40.264197 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:40.263955    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/242ace3b-f2c6-4fd5-abe5-31ce3ae9a12c-workload-certs" pod "242ace3b-f2c6-4fd5-abe5-31ce3ae9a12c" (UID: "242ace3b-f2c6-4fd5-abe5-31ce3ae9a12c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:45:40.264197 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:40.264048    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/242ace3b-f2c6-4fd5-abe5-31ce3ae9a12c-istio-token\" (UniqueName: \"kubernetes.io/projected/242ace3b-f2c6-4fd5-abe5-31ce3ae9a12c-istio-token\") pod \"242ace3b-f2c6-4fd5-abe5-31ce3ae9a12c\" (UID: \"242ace3b-f2c6-4fd5-abe5-31ce3ae9a12c\") "
Oct 30 19:45:40.264197 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:40.264083    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/242ace3b-f2c6-4fd5-abe5-31ce3ae9a12c-workload-socket" pod "242ace3b-f2c6-4fd5-abe5-31ce3ae9a12c" (UID: "242ace3b-f2c6-4fd5-abe5-31ce3ae9a12c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:45:40.264197 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:40.264159    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/242ace3b-f2c6-4fd5-abe5-31ce3ae9a12c-istio-data" pod "242ace3b-f2c6-4fd5-abe5-31ce3ae9a12c" (UID: "242ace3b-f2c6-4fd5-abe5-31ce3ae9a12c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:45:40.264389 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:40.264236    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/242ace3b-f2c6-4fd5-abe5-31ce3ae9a12c-istiod-ca-cert" pod "242ace3b-f2c6-4fd5-abe5-31ce3ae9a12c" (UID: "242ace3b-f2c6-4fd5-abe5-31ce3ae9a12c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:45:40.264639 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:40.264619    4888 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/242ace3b-f2c6-4fd5-abe5-31ce3ae9a12c-workload-socket\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:45:40.264694 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:40.264645    4888 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/242ace3b-f2c6-4fd5-abe5-31ce3ae9a12c-istiod-ca-cert\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:45:40.264694 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:40.264659    4888 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/242ace3b-f2c6-4fd5-abe5-31ce3ae9a12c-istio-data\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:45:40.264694 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:40.264674    4888 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/242ace3b-f2c6-4fd5-abe5-31ce3ae9a12c-credential-socket\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:45:40.264694 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:40.264686    4888 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/242ace3b-f2c6-4fd5-abe5-31ce3ae9a12c-workload-certs\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:45:40.266250 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:40.266220    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/242ace3b-f2c6-4fd5-abe5-31ce3ae9a12c-kube-api-access-b5shf" pod "242ace3b-f2c6-4fd5-abe5-31ce3ae9a12c" (UID: "242ace3b-f2c6-4fd5-abe5-31ce3ae9a12c"). InnerVolumeSpecName "kube-api-access-b5shf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:45:40.266697 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:40.266676    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/242ace3b-f2c6-4fd5-abe5-31ce3ae9a12c-istio-token" pod "242ace3b-f2c6-4fd5-abe5-31ce3ae9a12c" (UID: "242ace3b-f2c6-4fd5-abe5-31ce3ae9a12c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:45:40.266697 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:40.266683    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/242ace3b-f2c6-4fd5-abe5-31ce3ae9a12c-istio-envoy" pod "242ace3b-f2c6-4fd5-abe5-31ce3ae9a12c" (UID: "242ace3b-f2c6-4fd5-abe5-31ce3ae9a12c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:45:40.266831 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:40.266787    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/242ace3b-f2c6-4fd5-abe5-31ce3ae9a12c-istio-podinfo" pod "242ace3b-f2c6-4fd5-abe5-31ce3ae9a12c" (UID: "242ace3b-f2c6-4fd5-abe5-31ce3ae9a12c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:45:40.365385 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:40.365349    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b5shf\" (UniqueName: \"kubernetes.io/projected/242ace3b-f2c6-4fd5-abe5-31ce3ae9a12c-kube-api-access-b5shf\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:45:40.365385 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:40.365378    4888 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/242ace3b-f2c6-4fd5-abe5-31ce3ae9a12c-istio-envoy\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:45:40.365385 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:40.365388    4888 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/242ace3b-f2c6-4fd5-abe5-31ce3ae9a12c-istio-podinfo\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:45:40.365641 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:40.365396    4888 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/242ace3b-f2c6-4fd5-abe5-31ce3ae9a12c-istio-token\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:45:40.443881 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:40.443847    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-e7929d/distinct-2-w4-e7929d-kserve-5b479cc87b-htv9p"]
Oct 30 19:45:40.449984 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:40.449952    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-e7929d/distinct-w4-e7929d-kserve-5bc9ff5bf5-kmrwv"]
Oct 30 19:45:40.715061 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:40.714981    4888 generic.go:356] "Generic (PLEG): container finished" podID="242ace3b-f2c6-4fd5-abe5-31ce3ae9a12c" containerID="83b2c2def8c2419f26db112578fc3501a1f8ab0e41fecc3728aceff3cf05e7ae" exitCode=0
Oct 30 19:45:40.715061 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:40.715026    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-5da1ae-openshift-default-77c657bb8f-nng6m"
Oct 30 19:45:40.715061 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:40.715047    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-5da1ae-openshift-default-77c657bb8f-nng6m" event={"ID":"242ace3b-f2c6-4fd5-abe5-31ce3ae9a12c","Type":"ContainerDied","Data":"83b2c2def8c2419f26db112578fc3501a1f8ab0e41fecc3728aceff3cf05e7ae"}
Oct 30 19:45:40.715359 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:40.715088    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-5da1ae-openshift-default-77c657bb8f-nng6m" event={"ID":"242ace3b-f2c6-4fd5-abe5-31ce3ae9a12c","Type":"ContainerDied","Data":"d535ec0f9ea74d8f02efb67dd2a3a94918453d74492a1045fb1184af68d4b11c"}
Oct 30 19:45:40.715359 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:40.715108    4888 scope.go:122] "RemoveContainer" containerID="83b2c2def8c2419f26db112578fc3501a1f8ab0e41fecc3728aceff3cf05e7ae"
Oct 30 19:45:40.726072 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:40.726052    4888 scope.go:122] "RemoveContainer" containerID="296603c2add4596d5de87bdb74d1cb67d542d22cc7ec6581922a41b57a855676"
Oct 30 19:45:40.734818 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:40.734800    4888 scope.go:122] "RemoveContainer" containerID="83b2c2def8c2419f26db112578fc3501a1f8ab0e41fecc3728aceff3cf05e7ae"
Oct 30 19:45:40.735084 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:45:40.735064    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"83b2c2def8c2419f26db112578fc3501a1f8ab0e41fecc3728aceff3cf05e7ae\": container with ID starting with 83b2c2def8c2419f26db112578fc3501a1f8ab0e41fecc3728aceff3cf05e7ae not found: ID does not exist" containerID="83b2c2def8c2419f26db112578fc3501a1f8ab0e41fecc3728aceff3cf05e7ae"
Oct 30 19:45:40.735137 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:40.735092    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"83b2c2def8c2419f26db112578fc3501a1f8ab0e41fecc3728aceff3cf05e7ae"} err="failed to get container status \"83b2c2def8c2419f26db112578fc3501a1f8ab0e41fecc3728aceff3cf05e7ae\": rpc error: code = NotFound desc = could not find container \"83b2c2def8c2419f26db112578fc3501a1f8ab0e41fecc3728aceff3cf05e7ae\": container with ID starting with 83b2c2def8c2419f26db112578fc3501a1f8ab0e41fecc3728aceff3cf05e7ae not found: ID does not exist"
Oct 30 19:45:40.735137 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:40.735109    4888 scope.go:122] "RemoveContainer" containerID="296603c2add4596d5de87bdb74d1cb67d542d22cc7ec6581922a41b57a855676"
Oct 30 19:45:40.735422 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:45:40.735406    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"296603c2add4596d5de87bdb74d1cb67d542d22cc7ec6581922a41b57a855676\": container with ID starting with 296603c2add4596d5de87bdb74d1cb67d542d22cc7ec6581922a41b57a855676 not found: ID does not exist" containerID="296603c2add4596d5de87bdb74d1cb67d542d22cc7ec6581922a41b57a855676"
Oct 30 19:45:40.735499 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:40.735428    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"296603c2add4596d5de87bdb74d1cb67d542d22cc7ec6581922a41b57a855676"} err="failed to get container status \"296603c2add4596d5de87bdb74d1cb67d542d22cc7ec6581922a41b57a855676\": rpc error: code = NotFound desc = could not find container \"296603c2add4596d5de87bdb74d1cb67d542d22cc7ec6581922a41b57a855676\": container with ID starting with 296603c2add4596d5de87bdb74d1cb67d542d22cc7ec6581922a41b57a855676 not found: ID does not exist"
Oct 30 19:45:40.884997 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:40.884966    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-b-5da1ae-openshift-default-77c657bb8f-nng6m"]
Oct 30 19:45:40.889631 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:40.889595    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-extmod-b-5da1ae-openshift-default-77c657bb8f-nng6m"]
Oct 30 19:45:41.888942 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:41.888902    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="242ace3b-f2c6-4fd5-abe5-31ce3ae9a12c" path="/var/lib/kubelet/pods/242ace3b-f2c6-4fd5-abe5-31ce3ae9a12c/volumes"
Oct 30 19:45:46.551864 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:46.551819    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-a88dfc30-lmzpr"]
Oct 30 19:45:46.552499 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:46.552477    4888 cpu_manager.go:414] "containerMap: removing container" podUID="2b6554ea-aeb3-476a-bb73-76f566b54e00" containerName="cleanup"
Oct 30 19:45:46.552625 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:46.552501    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2b6554ea-aeb3-476a-bb73-76f566b54e00" containerName="cleanup"
Oct 30 19:45:46.552625 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:46.552514    4888 cpu_manager.go:414] "containerMap: removing container" podUID="242ace3b-f2c6-4fd5-abe5-31ce3ae9a12c" containerName="istio-proxy"
Oct 30 19:45:46.552625 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:46.552522    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="242ace3b-f2c6-4fd5-abe5-31ce3ae9a12c" containerName="istio-proxy"
Oct 30 19:45:46.552625 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:46.552539    4888 cpu_manager.go:414] "containerMap: removing container" podUID="3091e784-c5c0-401c-8b39-d5fd35db06ad" containerName="revoke-keys"
Oct 30 19:45:46.552625 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:46.552546    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3091e784-c5c0-401c-8b39-d5fd35db06ad" containerName="revoke-keys"
Oct 30 19:45:46.552889 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:46.552672    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="0e31a8f9-dd8c-43f2-a8bf-ac72a0443f22" containerName="cleanup"
Oct 30 19:45:46.552889 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:46.552684    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="2b6554ea-aeb3-476a-bb73-76f566b54e00" containerName="cleanup"
Oct 30 19:45:46.552889 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:46.552694    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="242ace3b-f2c6-4fd5-abe5-31ce3ae9a12c" containerName="istio-proxy"
Oct 30 19:45:46.552889 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:46.552706    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="3091e784-c5c0-401c-8b39-d5fd35db06ad" containerName="revoke-keys"
Oct 30 19:45:46.552889 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:46.552720    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="242ace3b-f2c6-4fd5-abe5-31ce3ae9a12c" containerName="istio-proxy"
Oct 30 19:45:46.557578 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:46.557557    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a88dfc30-lmzpr"
Oct 30 19:45:46.611638 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:46.611591    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 19:45:46.621846 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:46.621816    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5qhgp\" (UniqueName: \"kubernetes.io/projected/730325c3-8bae-453d-92b2-6a9a56027ce0-kube-api-access-5qhgp\") pod \"maas-api-revoke-sub-a88dfc30-lmzpr\" (UID: \"730325c3-8bae-453d-92b2-6a9a56027ce0\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a88dfc30-lmzpr"
Oct 30 19:45:46.621982 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:46.621935    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/730325c3-8bae-453d-92b2-6a9a56027ce0-maas-api-xks-ca\") pod \"maas-api-revoke-sub-a88dfc30-lmzpr\" (UID: \"730325c3-8bae-453d-92b2-6a9a56027ce0\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a88dfc30-lmzpr"
Oct 30 19:45:46.621982 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:46.621961    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/730325c3-8bae-453d-92b2-6a9a56027ce0-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-a88dfc30-lmzpr\" (UID: \"730325c3-8bae-453d-92b2-6a9a56027ce0\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a88dfc30-lmzpr"
Oct 30 19:45:46.662444 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:46.662415    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 30 19:45:46.662444 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:46.662415    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 30 19:45:46.662664 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:46.662430    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-2qg7n\""
Oct 30 19:45:46.664940 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:46.664918    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-a88dfc30-lmzpr"]
Oct 30 19:45:46.722911 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:46.722864    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/730325c3-8bae-453d-92b2-6a9a56027ce0-maas-api-xks-ca\") pod \"maas-api-revoke-sub-a88dfc30-lmzpr\" (UID: \"730325c3-8bae-453d-92b2-6a9a56027ce0\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a88dfc30-lmzpr"
Oct 30 19:45:46.723077 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:46.722925    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/730325c3-8bae-453d-92b2-6a9a56027ce0-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-a88dfc30-lmzpr\" (UID: \"730325c3-8bae-453d-92b2-6a9a56027ce0\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a88dfc30-lmzpr"
Oct 30 19:45:46.723077 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:46.722976    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5qhgp\" (UniqueName: \"kubernetes.io/projected/730325c3-8bae-453d-92b2-6a9a56027ce0-kube-api-access-5qhgp\") pod \"maas-api-revoke-sub-a88dfc30-lmzpr\" (UID: \"730325c3-8bae-453d-92b2-6a9a56027ce0\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a88dfc30-lmzpr"
Oct 30 19:45:46.723719 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:46.723692    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/730325c3-8bae-453d-92b2-6a9a56027ce0-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-a88dfc30-lmzpr\" (UID: \"730325c3-8bae-453d-92b2-6a9a56027ce0\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a88dfc30-lmzpr"
Oct 30 19:45:46.725442 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:46.725426    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/730325c3-8bae-453d-92b2-6a9a56027ce0-maas-api-xks-ca\") pod \"maas-api-revoke-sub-a88dfc30-lmzpr\" (UID: \"730325c3-8bae-453d-92b2-6a9a56027ce0\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a88dfc30-lmzpr"
Oct 30 19:45:46.785825 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:46.785794    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5qhgp\" (UniqueName: \"kubernetes.io/projected/730325c3-8bae-453d-92b2-6a9a56027ce0-kube-api-access-5qhgp\") pod \"maas-api-revoke-sub-a88dfc30-lmzpr\" (UID: \"730325c3-8bae-453d-92b2-6a9a56027ce0\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a88dfc30-lmzpr"
Oct 30 19:45:46.870764 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:46.870731    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a88dfc30-lmzpr"
Oct 30 19:45:47.252976 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:47.252944    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-a88dfc30-lmzpr"]
Oct 30 19:45:47.254192 ip-10-0-134-46 kubenswrapper[4888]: W1030 19:45:47.254169    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod730325c3_8bae_453d_92b2_6a9a56027ce0.slice/crio-3d02dcd8d554db41fc9d41fad133822821d8d7dafa06340a63f540855748b1a0 WatchSource:0}: Error finding container 3d02dcd8d554db41fc9d41fad133822821d8d7dafa06340a63f540855748b1a0: Status 404 returned error can't find the container with id 3d02dcd8d554db41fc9d41fad133822821d8d7dafa06340a63f540855748b1a0
Oct 30 19:45:47.570937 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:47.570909    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-e7929d-7cbc876b55-5mnbp"]
Oct 30 19:45:47.765673 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:47.765641    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a88dfc30-lmzpr" event={"ID":"730325c3-8bae-453d-92b2-6a9a56027ce0","Type":"ContainerStarted","Data":"9bf2f6ff04f963aab90069d63c3748bb3741b77437df04988c3af7b5b9dcf2d5"}
Oct 30 19:45:47.765875 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:47.765701    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a88dfc30-lmzpr" event={"ID":"730325c3-8bae-453d-92b2-6a9a56027ce0","Type":"ContainerStarted","Data":"3d02dcd8d554db41fc9d41fad133822821d8d7dafa06340a63f540855748b1a0"}
Oct 30 19:45:47.887278 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:47.887207    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a88dfc30-lmzpr" podStartSLOduration=1.887189386 podStartE2EDuration="1.887189386s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:45:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:45:47.884820863 +0000 UTC m=+4312.132226902" watchObservedRunningTime="2026-10-30 19:45:47.887189386 +0000 UTC m=+4312.134595412"
Oct 30 19:45:48.772601 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:48.772567    4888 generic.go:356] "Generic (PLEG): container finished" podID="730325c3-8bae-453d-92b2-6a9a56027ce0" containerID="9bf2f6ff04f963aab90069d63c3748bb3741b77437df04988c3af7b5b9dcf2d5" exitCode=0
Oct 30 19:45:48.773044 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:48.772660    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a88dfc30-lmzpr" event={"ID":"730325c3-8bae-453d-92b2-6a9a56027ce0","Type":"ContainerDied","Data":"9bf2f6ff04f963aab90069d63c3748bb3741b77437df04988c3af7b5b9dcf2d5"}
Oct 30 19:45:49.919192 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:49.919167    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a88dfc30-lmzpr"
Oct 30 19:45:50.054798 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:50.054717    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/730325c3-8bae-453d-92b2-6a9a56027ce0-kube-api-access-5qhgp\" (UniqueName: \"kubernetes.io/projected/730325c3-8bae-453d-92b2-6a9a56027ce0-kube-api-access-5qhgp\") pod \"730325c3-8bae-453d-92b2-6a9a56027ce0\" (UID: \"730325c3-8bae-453d-92b2-6a9a56027ce0\") "
Oct 30 19:45:50.054979 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:50.054815    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/730325c3-8bae-453d-92b2-6a9a56027ce0-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/730325c3-8bae-453d-92b2-6a9a56027ce0-maas-api-xks-ca\") pod \"730325c3-8bae-453d-92b2-6a9a56027ce0\" (UID: \"730325c3-8bae-453d-92b2-6a9a56027ce0\") "
Oct 30 19:45:50.054979 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:50.054905    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/730325c3-8bae-453d-92b2-6a9a56027ce0-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/730325c3-8bae-453d-92b2-6a9a56027ce0-maas-api-openshift-ca\") pod \"730325c3-8bae-453d-92b2-6a9a56027ce0\" (UID: \"730325c3-8bae-453d-92b2-6a9a56027ce0\") "
Oct 30 19:45:50.055487 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:50.055453    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/730325c3-8bae-453d-92b2-6a9a56027ce0-maas-api-openshift-ca" pod "730325c3-8bae-453d-92b2-6a9a56027ce0" (UID: "730325c3-8bae-453d-92b2-6a9a56027ce0"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:45:50.057200 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:50.057164    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/730325c3-8bae-453d-92b2-6a9a56027ce0-kube-api-access-5qhgp" pod "730325c3-8bae-453d-92b2-6a9a56027ce0" (UID: "730325c3-8bae-453d-92b2-6a9a56027ce0"). InnerVolumeSpecName "kube-api-access-5qhgp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:45:50.057200 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:50.057191    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/730325c3-8bae-453d-92b2-6a9a56027ce0-maas-api-xks-ca" pod "730325c3-8bae-453d-92b2-6a9a56027ce0" (UID: "730325c3-8bae-453d-92b2-6a9a56027ce0"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:45:50.156460 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:50.156427    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5qhgp\" (UniqueName: \"kubernetes.io/projected/730325c3-8bae-453d-92b2-6a9a56027ce0-kube-api-access-5qhgp\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:45:50.156460 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:50.156457    4888 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/730325c3-8bae-453d-92b2-6a9a56027ce0-maas-api-xks-ca\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:45:50.156460 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:50.156467    4888 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/730325c3-8bae-453d-92b2-6a9a56027ce0-maas-api-openshift-ca\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:45:50.785246 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:50.785219    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a88dfc30-lmzpr"
Oct 30 19:45:50.785445 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:50.785219    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a88dfc30-lmzpr" event={"ID":"730325c3-8bae-453d-92b2-6a9a56027ce0","Type":"ContainerDied","Data":"3d02dcd8d554db41fc9d41fad133822821d8d7dafa06340a63f540855748b1a0"}
Oct 30 19:45:50.785445 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:50.785368    4888 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3d02dcd8d554db41fc9d41fad133822821d8d7dafa06340a63f540855748b1a0"
Oct 30 19:45:52.571915 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:52.571848    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-e7929d-7cbc876b55-5mnbp" podUID="b13a4e48-bc35-4bff-9840-640c9a1d6c92" containerName="payload-pre-processing" containerID="cri-o://90d3b1437af10ab33425c776de11ff80083fde4297aa09d707d7a668ad21d8c6" gracePeriod=25
Oct 30 19:45:54.386492 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:54.386446    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-e7929d-7cbc876b55-5mnbp" podUID="b13a4e48-bc35-4bff-9840-640c9a1d6c92" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:45:55.464988 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:55.464940    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-e7929d/distinct-2-w4-e7929d-kserve-5b479cc87b-htv9p" podUID="f6df655d-86e8-44ff-ace8-27440f56294f" containerName="main" containerID="cri-o://2b97fd93ad52f3a83c6c5e51a3ea4b48f8890b63980c41432818aebe19fb8bb4" gracePeriod=45
Oct 30 19:45:55.470204 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:55.469708    4888 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-e7929d/distinct-w4-e7929d-kserve-5bc9ff5bf5-kmrwv" podUID="abfd315d-a01a-42dd-9c06-2a80d3f8579a" containerName="main" containerID="cri-o://ee4d9f5543c3dc362899c92e8ef587b30633ccffdad1ab23ee1c2dc2ee84eee4" gracePeriod=45
Oct 30 19:45:55.726673 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:55.726609    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-e7929d/distinct-2-w4-e7929d-kserve-5b479cc87b-htv9p"
Oct 30 19:45:55.746817 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:55.746790    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f6df655d-86e8-44ff-ace8-27440f56294f-kube-api-access-fhfmt\" (UniqueName: \"kubernetes.io/projected/f6df655d-86e8-44ff-ace8-27440f56294f-kube-api-access-fhfmt\") pod \"f6df655d-86e8-44ff-ace8-27440f56294f\" (UID: \"f6df655d-86e8-44ff-ace8-27440f56294f\") "
Oct 30 19:45:55.746955 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:55.746839    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f6df655d-86e8-44ff-ace8-27440f56294f-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f6df655d-86e8-44ff-ace8-27440f56294f-model-cache\") pod \"f6df655d-86e8-44ff-ace8-27440f56294f\" (UID: \"f6df655d-86e8-44ff-ace8-27440f56294f\") "
Oct 30 19:45:55.746955 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:55.746941    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f6df655d-86e8-44ff-ace8-27440f56294f-dshm\" (UniqueName: \"kubernetes.io/empty-dir/f6df655d-86e8-44ff-ace8-27440f56294f-dshm\") pod \"f6df655d-86e8-44ff-ace8-27440f56294f\" (UID: \"f6df655d-86e8-44ff-ace8-27440f56294f\") "
Oct 30 19:45:55.747089 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:55.747026    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f6df655d-86e8-44ff-ace8-27440f56294f-tls-certs\" (UniqueName: \"kubernetes.io/secret/f6df655d-86e8-44ff-ace8-27440f56294f-tls-certs\") pod \"f6df655d-86e8-44ff-ace8-27440f56294f\" (UID: \"f6df655d-86e8-44ff-ace8-27440f56294f\") "
Oct 30 19:45:55.747145 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:55.747086    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f6df655d-86e8-44ff-ace8-27440f56294f-home\" (UniqueName: \"kubernetes.io/empty-dir/f6df655d-86e8-44ff-ace8-27440f56294f-home\") pod \"f6df655d-86e8-44ff-ace8-27440f56294f\" (UID: \"f6df655d-86e8-44ff-ace8-27440f56294f\") "
Oct 30 19:45:55.747145 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:55.747089    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f6df655d-86e8-44ff-ace8-27440f56294f-model-cache" pod "f6df655d-86e8-44ff-ace8-27440f56294f" (UID: "f6df655d-86e8-44ff-ace8-27440f56294f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:45:55.747256 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:55.747157    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f6df655d-86e8-44ff-ace8-27440f56294f-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f6df655d-86e8-44ff-ace8-27440f56294f-tmp-dir\") pod \"f6df655d-86e8-44ff-ace8-27440f56294f\" (UID: \"f6df655d-86e8-44ff-ace8-27440f56294f\") "
Oct 30 19:45:55.747458 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:55.747434    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f6df655d-86e8-44ff-ace8-27440f56294f-tmp-dir" pod "f6df655d-86e8-44ff-ace8-27440f56294f" (UID: "f6df655d-86e8-44ff-ace8-27440f56294f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:45:55.747596 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:55.747460    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f6df655d-86e8-44ff-ace8-27440f56294f-home" pod "f6df655d-86e8-44ff-ace8-27440f56294f" (UID: "f6df655d-86e8-44ff-ace8-27440f56294f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:45:55.747596 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:55.747567    4888 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f6df655d-86e8-44ff-ace8-27440f56294f-tmp-dir\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:45:55.747596 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:55.747586    4888 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f6df655d-86e8-44ff-ace8-27440f56294f-model-cache\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:45:55.749566 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:55.749545    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f6df655d-86e8-44ff-ace8-27440f56294f-dshm" pod "f6df655d-86e8-44ff-ace8-27440f56294f" (UID: "f6df655d-86e8-44ff-ace8-27440f56294f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:45:55.749754 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:55.749730    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f6df655d-86e8-44ff-ace8-27440f56294f-tls-certs" pod "f6df655d-86e8-44ff-ace8-27440f56294f" (UID: "f6df655d-86e8-44ff-ace8-27440f56294f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:45:55.749847 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:55.749778    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f6df655d-86e8-44ff-ace8-27440f56294f-kube-api-access-fhfmt" pod "f6df655d-86e8-44ff-ace8-27440f56294f" (UID: "f6df655d-86e8-44ff-ace8-27440f56294f"). InnerVolumeSpecName "kube-api-access-fhfmt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:45:55.752360 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:55.752343    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-e7929d/distinct-w4-e7929d-kserve-5bc9ff5bf5-kmrwv"
Oct 30 19:45:55.817170 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:55.817143    4888 generic.go:356] "Generic (PLEG): container finished" podID="abfd315d-a01a-42dd-9c06-2a80d3f8579a" containerID="ee4d9f5543c3dc362899c92e8ef587b30633ccffdad1ab23ee1c2dc2ee84eee4" exitCode=0
Oct 30 19:45:55.817339 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:55.817188    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-e7929d/distinct-w4-e7929d-kserve-5bc9ff5bf5-kmrwv"
Oct 30 19:45:55.817339 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:55.817222    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-e7929d/distinct-w4-e7929d-kserve-5bc9ff5bf5-kmrwv" event={"ID":"abfd315d-a01a-42dd-9c06-2a80d3f8579a","Type":"ContainerDied","Data":"ee4d9f5543c3dc362899c92e8ef587b30633ccffdad1ab23ee1c2dc2ee84eee4"}
Oct 30 19:45:55.817339 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:55.817262    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-e7929d/distinct-w4-e7929d-kserve-5bc9ff5bf5-kmrwv" event={"ID":"abfd315d-a01a-42dd-9c06-2a80d3f8579a","Type":"ContainerDied","Data":"b7da3838a1af55f6dd6a7746ba83162bab5472ed3c2c785826529fd9261d72d1"}
Oct 30 19:45:55.817339 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:55.817278    4888 scope.go:122] "RemoveContainer" containerID="ee4d9f5543c3dc362899c92e8ef587b30633ccffdad1ab23ee1c2dc2ee84eee4"
Oct 30 19:45:55.818637 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:55.818613    4888 generic.go:356] "Generic (PLEG): container finished" podID="f6df655d-86e8-44ff-ace8-27440f56294f" containerID="2b97fd93ad52f3a83c6c5e51a3ea4b48f8890b63980c41432818aebe19fb8bb4" exitCode=0
Oct 30 19:45:55.818757 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:55.818656    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-e7929d/distinct-2-w4-e7929d-kserve-5b479cc87b-htv9p"
Oct 30 19:45:55.818757 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:55.818656    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-e7929d/distinct-2-w4-e7929d-kserve-5b479cc87b-htv9p" event={"ID":"f6df655d-86e8-44ff-ace8-27440f56294f","Type":"ContainerDied","Data":"2b97fd93ad52f3a83c6c5e51a3ea4b48f8890b63980c41432818aebe19fb8bb4"}
Oct 30 19:45:55.818757 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:55.818746    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-e7929d/distinct-2-w4-e7929d-kserve-5b479cc87b-htv9p" event={"ID":"f6df655d-86e8-44ff-ace8-27440f56294f","Type":"ContainerDied","Data":"392f86fa58d68f94c737ef20cb468cdaf5ad3291120b92c80b32924e6dd9c2a1"}
Oct 30 19:45:55.826848 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:55.826831    4888 scope.go:122] "RemoveContainer" containerID="ee4d9f5543c3dc362899c92e8ef587b30633ccffdad1ab23ee1c2dc2ee84eee4"
Oct 30 19:45:55.827183 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:45:55.827159    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ee4d9f5543c3dc362899c92e8ef587b30633ccffdad1ab23ee1c2dc2ee84eee4\": container with ID starting with ee4d9f5543c3dc362899c92e8ef587b30633ccffdad1ab23ee1c2dc2ee84eee4 not found: ID does not exist" containerID="ee4d9f5543c3dc362899c92e8ef587b30633ccffdad1ab23ee1c2dc2ee84eee4"
Oct 30 19:45:55.827233 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:55.827192    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ee4d9f5543c3dc362899c92e8ef587b30633ccffdad1ab23ee1c2dc2ee84eee4"} err="failed to get container status \"ee4d9f5543c3dc362899c92e8ef587b30633ccffdad1ab23ee1c2dc2ee84eee4\": rpc error: code = NotFound desc = could not find container \"ee4d9f5543c3dc362899c92e8ef587b30633ccffdad1ab23ee1c2dc2ee84eee4\": container with ID starting with ee4d9f5543c3dc362899c92e8ef587b30633ccffdad1ab23ee1c2dc2ee84eee4 not found: ID does not exist"
Oct 30 19:45:55.827233 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:55.827209    4888 scope.go:122] "RemoveContainer" containerID="2b97fd93ad52f3a83c6c5e51a3ea4b48f8890b63980c41432818aebe19fb8bb4"
Oct 30 19:45:55.836224 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:55.836209    4888 scope.go:122] "RemoveContainer" containerID="2b97fd93ad52f3a83c6c5e51a3ea4b48f8890b63980c41432818aebe19fb8bb4"
Oct 30 19:45:55.836514 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:45:55.836492    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2b97fd93ad52f3a83c6c5e51a3ea4b48f8890b63980c41432818aebe19fb8bb4\": container with ID starting with 2b97fd93ad52f3a83c6c5e51a3ea4b48f8890b63980c41432818aebe19fb8bb4 not found: ID does not exist" containerID="2b97fd93ad52f3a83c6c5e51a3ea4b48f8890b63980c41432818aebe19fb8bb4"
Oct 30 19:45:55.836576 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:55.836520    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2b97fd93ad52f3a83c6c5e51a3ea4b48f8890b63980c41432818aebe19fb8bb4"} err="failed to get container status \"2b97fd93ad52f3a83c6c5e51a3ea4b48f8890b63980c41432818aebe19fb8bb4\": rpc error: code = NotFound desc = could not find container \"2b97fd93ad52f3a83c6c5e51a3ea4b48f8890b63980c41432818aebe19fb8bb4\": container with ID starting with 2b97fd93ad52f3a83c6c5e51a3ea4b48f8890b63980c41432818aebe19fb8bb4 not found: ID does not exist"
Oct 30 19:45:55.848881 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:55.848854    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/abfd315d-a01a-42dd-9c06-2a80d3f8579a-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/abfd315d-a01a-42dd-9c06-2a80d3f8579a-tmp-dir\") pod \"abfd315d-a01a-42dd-9c06-2a80d3f8579a\" (UID: \"abfd315d-a01a-42dd-9c06-2a80d3f8579a\") "
Oct 30 19:45:55.849004 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:55.848904    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/abfd315d-a01a-42dd-9c06-2a80d3f8579a-tls-certs\" (UniqueName: \"kubernetes.io/secret/abfd315d-a01a-42dd-9c06-2a80d3f8579a-tls-certs\") pod \"abfd315d-a01a-42dd-9c06-2a80d3f8579a\" (UID: \"abfd315d-a01a-42dd-9c06-2a80d3f8579a\") "
Oct 30 19:45:55.849004 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:55.848991    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/abfd315d-a01a-42dd-9c06-2a80d3f8579a-dshm\" (UniqueName: \"kubernetes.io/empty-dir/abfd315d-a01a-42dd-9c06-2a80d3f8579a-dshm\") pod \"abfd315d-a01a-42dd-9c06-2a80d3f8579a\" (UID: \"abfd315d-a01a-42dd-9c06-2a80d3f8579a\") "
Oct 30 19:45:55.849137 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:55.849019    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/abfd315d-a01a-42dd-9c06-2a80d3f8579a-kube-api-access-fgk4z\" (UniqueName: \"kubernetes.io/projected/abfd315d-a01a-42dd-9c06-2a80d3f8579a-kube-api-access-fgk4z\") pod \"abfd315d-a01a-42dd-9c06-2a80d3f8579a\" (UID: \"abfd315d-a01a-42dd-9c06-2a80d3f8579a\") "
Oct 30 19:45:55.849137 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:55.849052    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/abfd315d-a01a-42dd-9c06-2a80d3f8579a-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/abfd315d-a01a-42dd-9c06-2a80d3f8579a-model-cache\") pod \"abfd315d-a01a-42dd-9c06-2a80d3f8579a\" (UID: \"abfd315d-a01a-42dd-9c06-2a80d3f8579a\") "
Oct 30 19:45:55.849137 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:55.849115    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/abfd315d-a01a-42dd-9c06-2a80d3f8579a-home\" (UniqueName: \"kubernetes.io/empty-dir/abfd315d-a01a-42dd-9c06-2a80d3f8579a-home\") pod \"abfd315d-a01a-42dd-9c06-2a80d3f8579a\" (UID: \"abfd315d-a01a-42dd-9c06-2a80d3f8579a\") "
Oct 30 19:45:55.849346 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:55.849159    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/abfd315d-a01a-42dd-9c06-2a80d3f8579a-tmp-dir" pod "abfd315d-a01a-42dd-9c06-2a80d3f8579a" (UID: "abfd315d-a01a-42dd-9c06-2a80d3f8579a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:45:55.849689 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:55.849442    4888 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/abfd315d-a01a-42dd-9c06-2a80d3f8579a-tmp-dir\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:45:55.849689 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:55.849464    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fhfmt\" (UniqueName: \"kubernetes.io/projected/f6df655d-86e8-44ff-ace8-27440f56294f-kube-api-access-fhfmt\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:45:55.849689 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:55.849478    4888 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f6df655d-86e8-44ff-ace8-27440f56294f-dshm\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:45:55.849689 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:55.849493    4888 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f6df655d-86e8-44ff-ace8-27440f56294f-tls-certs\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:45:55.849689 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:55.849505    4888 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f6df655d-86e8-44ff-ace8-27440f56294f-home\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:45:55.849689 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:55.849659    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/abfd315d-a01a-42dd-9c06-2a80d3f8579a-model-cache" pod "abfd315d-a01a-42dd-9c06-2a80d3f8579a" (UID: "abfd315d-a01a-42dd-9c06-2a80d3f8579a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:45:55.850245 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:55.849913    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/abfd315d-a01a-42dd-9c06-2a80d3f8579a-home" pod "abfd315d-a01a-42dd-9c06-2a80d3f8579a" (UID: "abfd315d-a01a-42dd-9c06-2a80d3f8579a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:45:55.851691 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:55.851671    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/abfd315d-a01a-42dd-9c06-2a80d3f8579a-dshm" pod "abfd315d-a01a-42dd-9c06-2a80d3f8579a" (UID: "abfd315d-a01a-42dd-9c06-2a80d3f8579a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:45:55.851973 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:55.851941    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/abfd315d-a01a-42dd-9c06-2a80d3f8579a-kube-api-access-fgk4z" pod "abfd315d-a01a-42dd-9c06-2a80d3f8579a" (UID: "abfd315d-a01a-42dd-9c06-2a80d3f8579a"). InnerVolumeSpecName "kube-api-access-fgk4z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:45:55.851973 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:55.851961    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/abfd315d-a01a-42dd-9c06-2a80d3f8579a-tls-certs" pod "abfd315d-a01a-42dd-9c06-2a80d3f8579a" (UID: "abfd315d-a01a-42dd-9c06-2a80d3f8579a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:45:55.950170 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:55.950145    4888 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/abfd315d-a01a-42dd-9c06-2a80d3f8579a-tls-certs\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:45:55.950170 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:55.950167    4888 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/abfd315d-a01a-42dd-9c06-2a80d3f8579a-dshm\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:45:55.950170 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:55.950176    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fgk4z\" (UniqueName: \"kubernetes.io/projected/abfd315d-a01a-42dd-9c06-2a80d3f8579a-kube-api-access-fgk4z\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:45:55.950492 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:55.950185    4888 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/abfd315d-a01a-42dd-9c06-2a80d3f8579a-model-cache\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:45:55.950492 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:55.950196    4888 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/abfd315d-a01a-42dd-9c06-2a80d3f8579a-home\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:45:56.081612 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:56.081582    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-e7929d/distinct-2-w4-e7929d-kserve-5b479cc87b-htv9p"]
Oct 30 19:45:56.086749 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:56.086716    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-e7929d/distinct-2-w4-e7929d-kserve-5b479cc87b-htv9p"]
Oct 30 19:45:56.234864 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:56.234823    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-e7929d/distinct-w4-e7929d-kserve-5bc9ff5bf5-kmrwv"]
Oct 30 19:45:56.242493 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:56.242445    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-e7929d/distinct-w4-e7929d-kserve-5bc9ff5bf5-kmrwv"]
Oct 30 19:45:57.890148 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:57.890117    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="abfd315d-a01a-42dd-9c06-2a80d3f8579a" path="/var/lib/kubelet/pods/abfd315d-a01a-42dd-9c06-2a80d3f8579a/volumes"
Oct 30 19:45:57.890576 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:45:57.890561    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f6df655d-86e8-44ff-ace8-27440f56294f" path="/var/lib/kubelet/pods/f6df655d-86e8-44ff-ace8-27440f56294f/volumes"
Oct 30 19:46:03.204591 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:46:03.204556    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-rjmdz"]
Oct 30 19:46:03.338857 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:46:03.338826    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-e7929d-t75nt"]
Oct 30 19:46:03.343891 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:46:03.343868    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-e7929d-t75nt"]
Oct 30 19:46:03.890312 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:46:03.890258    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3091e784-c5c0-401c-8b39-d5fd35db06ad" path="/var/lib/kubelet/pods/3091e784-c5c0-401c-8b39-d5fd35db06ad/volumes"
Oct 30 19:46:04.386607 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:46:04.386569    4888 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-e7929d-7cbc876b55-5mnbp" podUID="b13a4e48-bc35-4bff-9840-640c9a1d6c92" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:46:05.617831 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:46:05.617792    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-jbsw6" containerName="extractor"
Oct 30 19:46:12.294481 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:46:12.294453    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-5f5864cd46-846ft_bfa5d691-05e1-48a0-9d53-52975ea89cf4/authorino/0.log"
Oct 30 19:46:12.725832 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:46:12.725800    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-7b7bb4bc77-rvtlv_ea157aed-6416-4a50-af9f-c9ebbc6ee044/manager/0.log"
Oct 30 19:46:12.832745 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:46:12.832688    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-e7929d-7cbc876b55-5mnbp"
Oct 30 19:46:12.914990 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:46:12.914949    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b13a4e48-bc35-4bff-9840-640c9a1d6c92-kube-api-access-q6s2j\" (UniqueName: \"kubernetes.io/projected/b13a4e48-bc35-4bff-9840-640c9a1d6c92-kube-api-access-q6s2j\") pod \"b13a4e48-bc35-4bff-9840-640c9a1d6c92\" (UID: \"b13a4e48-bc35-4bff-9840-640c9a1d6c92\") "
Oct 30 19:46:12.915175 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:46:12.915084    4888 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b13a4e48-bc35-4bff-9840-640c9a1d6c92-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b13a4e48-bc35-4bff-9840-640c9a1d6c92-plugins-config-volume\") pod \"b13a4e48-bc35-4bff-9840-640c9a1d6c92\" (UID: \"b13a4e48-bc35-4bff-9840-640c9a1d6c92\") "
Oct 30 19:46:12.915524 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:46:12.915495    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b13a4e48-bc35-4bff-9840-640c9a1d6c92-plugins-config-volume" pod "b13a4e48-bc35-4bff-9840-640c9a1d6c92" (UID: "b13a4e48-bc35-4bff-9840-640c9a1d6c92"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:46:12.915820 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:46:12.915799    4888 generic.go:356] "Generic (PLEG): container finished" podID="b13a4e48-bc35-4bff-9840-640c9a1d6c92" containerID="90d3b1437af10ab33425c776de11ff80083fde4297aa09d707d7a668ad21d8c6" exitCode=0
Oct 30 19:46:12.915935 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:46:12.915894    4888 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-e7929d-7cbc876b55-5mnbp"
Oct 30 19:46:12.916001 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:46:12.915887    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-e7929d-7cbc876b55-5mnbp" event={"ID":"b13a4e48-bc35-4bff-9840-640c9a1d6c92","Type":"ContainerDied","Data":"90d3b1437af10ab33425c776de11ff80083fde4297aa09d707d7a668ad21d8c6"}
Oct 30 19:46:12.916075 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:46:12.916054    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-e7929d-7cbc876b55-5mnbp" event={"ID":"b13a4e48-bc35-4bff-9840-640c9a1d6c92","Type":"ContainerDied","Data":"066e40aad7bdb3a4aa680993bcd5bb5f1a6875ecfd4627f798997ac30490f660"}
Oct 30 19:46:12.916112 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:46:12.916103    4888 scope.go:122] "RemoveContainer" containerID="90d3b1437af10ab33425c776de11ff80083fde4297aa09d707d7a668ad21d8c6"
Oct 30 19:46:12.921705 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:46:12.921618    4888 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b13a4e48-bc35-4bff-9840-640c9a1d6c92-kube-api-access-q6s2j" pod "b13a4e48-bc35-4bff-9840-640c9a1d6c92" (UID: "b13a4e48-bc35-4bff-9840-640c9a1d6c92"). InnerVolumeSpecName "kube-api-access-q6s2j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:46:12.931678 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:46:12.931654    4888 scope.go:122] "RemoveContainer" containerID="90d3b1437af10ab33425c776de11ff80083fde4297aa09d707d7a668ad21d8c6"
Oct 30 19:46:12.932023 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:46:12.931998    4888 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"90d3b1437af10ab33425c776de11ff80083fde4297aa09d707d7a668ad21d8c6\": container with ID starting with 90d3b1437af10ab33425c776de11ff80083fde4297aa09d707d7a668ad21d8c6 not found: ID does not exist" containerID="90d3b1437af10ab33425c776de11ff80083fde4297aa09d707d7a668ad21d8c6"
Oct 30 19:46:12.932104 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:46:12.932035    4888 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"90d3b1437af10ab33425c776de11ff80083fde4297aa09d707d7a668ad21d8c6"} err="failed to get container status \"90d3b1437af10ab33425c776de11ff80083fde4297aa09d707d7a668ad21d8c6\": rpc error: code = NotFound desc = could not find container \"90d3b1437af10ab33425c776de11ff80083fde4297aa09d707d7a668ad21d8c6\": container with ID starting with 90d3b1437af10ab33425c776de11ff80083fde4297aa09d707d7a668ad21d8c6 not found: ID does not exist"
Oct 30 19:46:12.960325 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:46:12.960300    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-8557cbdb84-8dfbp_d78a842d-920b-4619-9cd7-61d8e86ae880/manager/0.log"
Oct 30 19:46:13.016139 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:46:13.016107    4888 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b13a4e48-bc35-4bff-9840-640c9a1d6c92-plugins-config-volume\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:46:13.016139 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:46:13.016133    4888 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q6s2j\" (UniqueName: \"kubernetes.io/projected/b13a4e48-bc35-4bff-9840-640c9a1d6c92-kube-api-access-q6s2j\") on node \"ip-10-0-134-46.us-west-1.compute.internal\" DevicePath \"\""
Oct 30 19:46:13.341841 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:46:13.341808    4888 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-e7929d-7cbc876b55-5mnbp"]
Oct 30 19:46:13.346173 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:46:13.346145    4888 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-e7929d-7cbc876b55-5mnbp"]
Oct 30 19:46:13.888904 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:46:13.888877    4888 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b13a4e48-bc35-4bff-9840-640c9a1d6c92" path="/var/lib/kubelet/pods/b13a4e48-bc35-4bff-9840-640c9a1d6c92/volumes"
Oct 30 19:46:14.668468 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:46:14.668436    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-678d87766f-mx24q_e31ecc7e-2afe-4777-9ae4-50954433a58e/manager/0.log"
Oct 30 19:46:14.908920 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:46:14.908893    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-7c9b87d574-rjmdz_513b0b3f-06d8-42bf-b73c-89f3bae42d98/limitador/0.log"
Oct 30 19:46:15.138124 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:46:15.138091    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-5778bb4449-f9rvt_db5d7a86-9e39-413e-8628-9a7e77270908/manager/0.log"
Oct 30 19:46:16.216562 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:46:16.216529    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-776cc66d47-rrwt2_79919fce-ca7a-47ce-bc62-490450a6ac2d/manager/0.log"
Oct 30 19:46:16.811595 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:46:16.811561    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-rjmdz"]
Oct 30 19:46:17.397582 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:46:17.397556    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-86b6bf7f47-w2sgd_149ec7c2-df24-4c6e-81ff-121bad25f86d/manager/1.log"
Oct 30 19:46:18.601263 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:46:18.601231    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-86b6bf7f47-w2sgd_149ec7c2-df24-4c6e-81ff-121bad25f86d/manager/0.log"
Oct 30 19:46:19.212430 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:46:19.212388    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-5f5864cd46-846ft_bfa5d691-05e1-48a0-9d53-52975ea89cf4/authorino/0.log"
Oct 30 19:46:29.988224 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:46:29.988190    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-776cc66d47-rrwt2_79919fce-ca7a-47ce-bc62-490450a6ac2d/manager/0.log"
Oct 30 19:46:31.093780 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:46:31.093748    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-86b6bf7f47-w2sgd_149ec7c2-df24-4c6e-81ff-121bad25f86d/manager/1.log"
Oct 30 19:46:32.294636 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:46:32.294607    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-86b6bf7f47-w2sgd_149ec7c2-df24-4c6e-81ff-121bad25f86d/manager/0.log"
Oct 30 19:46:34.750965 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:46:34.750935    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-a88dfc30-lmzpr_730325c3-8bae-453d-92b2-6a9a56027ce0/revoke-keys/0.log"
Oct 30 19:46:35.617538 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:46:35.617461    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-jbsw6" containerName="extractor"
Oct 30 19:46:36.780562 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:46:36.780533    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-5f5864cd46-846ft_bfa5d691-05e1-48a0-9d53-52975ea89cf4/authorino/0.log"
Oct 30 19:46:37.077255 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:46:37.077229    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-7b7bb4bc77-rvtlv_ea157aed-6416-4a50-af9f-c9ebbc6ee044/manager/0.log"
Oct 30 19:46:37.317653 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:46:37.317626    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-8557cbdb84-8dfbp_d78a842d-920b-4619-9cd7-61d8e86ae880/manager/0.log"
Oct 30 19:46:38.406346 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:46:38.406308    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-678d87766f-mx24q_e31ecc7e-2afe-4777-9ae4-50954433a58e/manager/0.log"
Oct 30 19:46:38.646694 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:46:38.646660    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-7c9b87d574-rjmdz_513b0b3f-06d8-42bf-b73c-89f3bae42d98/limitador/0.log"
Oct 30 19:46:38.882651 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:46:38.882616    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-5778bb4449-f9rvt_db5d7a86-9e39-413e-8628-9a7e77270908/manager/0.log"
Oct 30 19:46:39.692824 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:46:39.692776    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-8f76c657-bfbg6_e1f8438b-6784-4805-adbb-9d45026b12b6/istio-proxy/0.log"
Oct 30 19:46:40.908738 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:46:40.908713    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-ai-inference-6d5bbf59c5-wjwcj_f434df49-9016-4649-a140-82f23de7cdfc/istio-proxy/0.log"
Oct 30 19:46:41.606302 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:46:41.606259    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7dfcf9fd6c-vh769_da28351c-1c8b-40be-85a0-3aad427e3a17/router/0.log"
Oct 30 19:46:42.960882 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:46:42.960855    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-embedding-simulated-kserve-67bf594cd6-dz9tf_c7c621a4-fc1b-40e4-bb4e-1971246d0472/main/0.log"
Oct 30 19:46:43.198015 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:46:43.197985    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-69b588897d-cwzlx_e9371581-24f3-4ce8-aae8-cc1a8614f4c0/main/0.log"
Oct 30 19:46:43.417508 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:46:43.417479    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8pfh7p_01e4dc41-5ae7-4847-bf9c-59106df11788/main/0.log"
Oct 30 19:46:44.109661 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:46:44.109633    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5f5c4b459f-rkm4t_1ab20f74-2c73-4d56-a604-aad4f8289450/main/0.log"
Oct 30 19:46:54.618806 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:46:54.618776    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-fxt2k/must-gather-wjwkg"]
Oct 30 19:46:54.619231 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:46:54.619191    4888 cpu_manager.go:414] "containerMap: removing container" podUID="abfd315d-a01a-42dd-9c06-2a80d3f8579a" containerName="main"
Oct 30 19:46:54.619231 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:46:54.619203    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="abfd315d-a01a-42dd-9c06-2a80d3f8579a" containerName="main"
Oct 30 19:46:54.619231 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:46:54.619215    4888 cpu_manager.go:414] "containerMap: removing container" podUID="f6df655d-86e8-44ff-ace8-27440f56294f" containerName="main"
Oct 30 19:46:54.619231 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:46:54.619220    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f6df655d-86e8-44ff-ace8-27440f56294f" containerName="main"
Oct 30 19:46:54.619231 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:46:54.619228    4888 cpu_manager.go:414] "containerMap: removing container" podUID="b13a4e48-bc35-4bff-9840-640c9a1d6c92" containerName="payload-pre-processing"
Oct 30 19:46:54.619438 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:46:54.619233    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b13a4e48-bc35-4bff-9840-640c9a1d6c92" containerName="payload-pre-processing"
Oct 30 19:46:54.619438 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:46:54.619242    4888 cpu_manager.go:414] "containerMap: removing container" podUID="730325c3-8bae-453d-92b2-6a9a56027ce0" containerName="revoke-keys"
Oct 30 19:46:54.619438 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:46:54.619247    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="730325c3-8bae-453d-92b2-6a9a56027ce0" containerName="revoke-keys"
Oct 30 19:46:54.619438 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:46:54.619256    4888 cpu_manager.go:414] "containerMap: removing container" podUID="242ace3b-f2c6-4fd5-abe5-31ce3ae9a12c" containerName="istio-proxy"
Oct 30 19:46:54.619438 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:46:54.619260    4888 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="242ace3b-f2c6-4fd5-abe5-31ce3ae9a12c" containerName="istio-proxy"
Oct 30 19:46:54.619438 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:46:54.619330    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="f6df655d-86e8-44ff-ace8-27440f56294f" containerName="main"
Oct 30 19:46:54.619438 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:46:54.619339    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="b13a4e48-bc35-4bff-9840-640c9a1d6c92" containerName="payload-pre-processing"
Oct 30 19:46:54.619438 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:46:54.619345    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="730325c3-8bae-453d-92b2-6a9a56027ce0" containerName="revoke-keys"
Oct 30 19:46:54.619438 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:46:54.619355    4888 memory_manager.go:365] "RemoveStaleState removing state" podUID="abfd315d-a01a-42dd-9c06-2a80d3f8579a" containerName="main"
Oct 30 19:46:54.622601 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:46:54.622582    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-fxt2k/must-gather-wjwkg"
Oct 30 19:46:54.706608 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:46:54.706560    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/62c69392-7e2c-47ca-b4b2-6d67802a7689-must-gather-output\") pod \"must-gather-wjwkg\" (UID: \"62c69392-7e2c-47ca-b4b2-6d67802a7689\") " pod="openshift-must-gather-fxt2k/must-gather-wjwkg"
Oct 30 19:46:54.706608 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:46:54.706598    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zm948\" (UniqueName: \"kubernetes.io/projected/62c69392-7e2c-47ca-b4b2-6d67802a7689-kube-api-access-zm948\") pod \"must-gather-wjwkg\" (UID: \"62c69392-7e2c-47ca-b4b2-6d67802a7689\") " pod="openshift-must-gather-fxt2k/must-gather-wjwkg"
Oct 30 19:46:54.727748 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:46:54.727725    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-fxt2k\"/\"openshift-service-ca.crt\""
Oct 30 19:46:54.727969 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:46:54.727953    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-fxt2k\"/\"default-dockercfg-mqgp7\""
Oct 30 19:46:54.728067 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:46:54.728049    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-fxt2k\"/\"kube-root-ca.crt\""
Oct 30 19:46:54.730450 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:46:54.730415    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-fxt2k/must-gather-wjwkg"]
Oct 30 19:46:54.732489 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:46:54.732472    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-v8nrk/must-gather-s5wl6"]
Oct 30 19:46:54.736016 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:46:54.736002    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-v8nrk/must-gather-s5wl6"
Oct 30 19:46:54.783418 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:46:54.783396    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-v8nrk/must-gather-s5wl6"]
Oct 30 19:46:54.807681 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:46:54.807656    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4k49d\" (UniqueName: \"kubernetes.io/projected/a20e0e51-0de6-48b0-866d-ba06f223dd11-kube-api-access-4k49d\") pod \"must-gather-s5wl6\" (UID: \"a20e0e51-0de6-48b0-866d-ba06f223dd11\") " pod="openshift-must-gather-v8nrk/must-gather-s5wl6"
Oct 30 19:46:54.807798 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:46:54.807697    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/a20e0e51-0de6-48b0-866d-ba06f223dd11-must-gather-output\") pod \"must-gather-s5wl6\" (UID: \"a20e0e51-0de6-48b0-866d-ba06f223dd11\") " pod="openshift-must-gather-v8nrk/must-gather-s5wl6"
Oct 30 19:46:54.807798 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:46:54.807736    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/62c69392-7e2c-47ca-b4b2-6d67802a7689-must-gather-output\") pod \"must-gather-wjwkg\" (UID: \"62c69392-7e2c-47ca-b4b2-6d67802a7689\") " pod="openshift-must-gather-fxt2k/must-gather-wjwkg"
Oct 30 19:46:54.807798 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:46:54.807753    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zm948\" (UniqueName: \"kubernetes.io/projected/62c69392-7e2c-47ca-b4b2-6d67802a7689-kube-api-access-zm948\") pod \"must-gather-wjwkg\" (UID: \"62c69392-7e2c-47ca-b4b2-6d67802a7689\") " pod="openshift-must-gather-fxt2k/must-gather-wjwkg"
Oct 30 19:46:54.808096 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:46:54.808079    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/62c69392-7e2c-47ca-b4b2-6d67802a7689-must-gather-output\") pod \"must-gather-wjwkg\" (UID: \"62c69392-7e2c-47ca-b4b2-6d67802a7689\") " pod="openshift-must-gather-fxt2k/must-gather-wjwkg"
Oct 30 19:46:54.869624 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:46:54.869558    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zm948\" (UniqueName: \"kubernetes.io/projected/62c69392-7e2c-47ca-b4b2-6d67802a7689-kube-api-access-zm948\") pod \"must-gather-wjwkg\" (UID: \"62c69392-7e2c-47ca-b4b2-6d67802a7689\") " pod="openshift-must-gather-fxt2k/must-gather-wjwkg"
Oct 30 19:46:54.878266 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:46:54.878249    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-v8nrk\"/\"kube-root-ca.crt\""
Oct 30 19:46:54.878393 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:46:54.878274    4888 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-v8nrk\"/\"openshift-service-ca.crt\""
Oct 30 19:46:54.878393 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:46:54.878335    4888 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-v8nrk\"/\"default-dockercfg-6bfd2\""
Oct 30 19:46:54.908577 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:46:54.908557    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/a20e0e51-0de6-48b0-866d-ba06f223dd11-must-gather-output\") pod \"must-gather-s5wl6\" (UID: \"a20e0e51-0de6-48b0-866d-ba06f223dd11\") " pod="openshift-must-gather-v8nrk/must-gather-s5wl6"
Oct 30 19:46:54.908668 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:46:54.908624    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4k49d\" (UniqueName: \"kubernetes.io/projected/a20e0e51-0de6-48b0-866d-ba06f223dd11-kube-api-access-4k49d\") pod \"must-gather-s5wl6\" (UID: \"a20e0e51-0de6-48b0-866d-ba06f223dd11\") " pod="openshift-must-gather-v8nrk/must-gather-s5wl6"
Oct 30 19:46:54.908891 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:46:54.908868    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/a20e0e51-0de6-48b0-866d-ba06f223dd11-must-gather-output\") pod \"must-gather-s5wl6\" (UID: \"a20e0e51-0de6-48b0-866d-ba06f223dd11\") " pod="openshift-must-gather-v8nrk/must-gather-s5wl6"
Oct 30 19:46:54.936527 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:46:54.936509    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-fxt2k/must-gather-wjwkg"
Oct 30 19:46:54.973021 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:46:54.972993    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4k49d\" (UniqueName: \"kubernetes.io/projected/a20e0e51-0de6-48b0-866d-ba06f223dd11-kube-api-access-4k49d\") pod \"must-gather-s5wl6\" (UID: \"a20e0e51-0de6-48b0-866d-ba06f223dd11\") " pod="openshift-must-gather-v8nrk/must-gather-s5wl6"
Oct 30 19:46:55.049600 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:46:55.049571    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-v8nrk/must-gather-s5wl6"
Oct 30 19:46:55.313399 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:46:55.313371    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-fxt2k/must-gather-wjwkg"]
Oct 30 19:46:55.314837 ip-10-0-134-46 kubenswrapper[4888]: W1030 19:46:55.314812    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod62c69392_7e2c_47ca_b4b2_6d67802a7689.slice/crio-0633b535044598493d379e742cbcd66ffff124aa31f8a9499abf9fa5a6ec9b74 WatchSource:0}: Error finding container 0633b535044598493d379e742cbcd66ffff124aa31f8a9499abf9fa5a6ec9b74: Status 404 returned error can't find the container with id 0633b535044598493d379e742cbcd66ffff124aa31f8a9499abf9fa5a6ec9b74
Oct 30 19:46:55.316752 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:46:55.316737    4888 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 19:46:55.434158 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:46:55.434135    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-v8nrk/must-gather-s5wl6"]
Oct 30 19:46:55.435096 ip-10-0-134-46 kubenswrapper[4888]: W1030 19:46:55.435065    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda20e0e51_0de6_48b0_866d_ba06f223dd11.slice/crio-db1cab8b17287ed45b343098d25e12d1b13ce400f584a3c7e44180e7c2f84c7b WatchSource:0}: Error finding container db1cab8b17287ed45b343098d25e12d1b13ce400f584a3c7e44180e7c2f84c7b: Status 404 returned error can't find the container with id db1cab8b17287ed45b343098d25e12d1b13ce400f584a3c7e44180e7c2f84c7b
Oct 30 19:46:56.152471 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:46:56.152438    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-fxt2k/must-gather-wjwkg" event={"ID":"62c69392-7e2c-47ca-b4b2-6d67802a7689","Type":"ContainerStarted","Data":"0633b535044598493d379e742cbcd66ffff124aa31f8a9499abf9fa5a6ec9b74"}
Oct 30 19:46:56.153636 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:46:56.153611    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-v8nrk/must-gather-s5wl6" event={"ID":"a20e0e51-0de6-48b0-866d-ba06f223dd11","Type":"ContainerStarted","Data":"db1cab8b17287ed45b343098d25e12d1b13ce400f584a3c7e44180e7c2f84c7b"}
Oct 30 19:46:57.169237 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:46:57.169193    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-v8nrk/must-gather-s5wl6" event={"ID":"a20e0e51-0de6-48b0-866d-ba06f223dd11","Type":"ContainerStarted","Data":"9886de8617163e894e124c83b1fb3f850618deca4d3d540e5a823d85db2640e0"}
Oct 30 19:46:57.169237 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:46:57.169243    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-v8nrk/must-gather-s5wl6" event={"ID":"a20e0e51-0de6-48b0-866d-ba06f223dd11","Type":"ContainerStarted","Data":"2fa266ba3c418f8efee5d51b996404e132b13a182728a05f97e6ecb7ac3b8fde"}
Oct 30 19:46:57.288638 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:46:57.288583    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-v8nrk/must-gather-s5wl6" podStartSLOduration=2.005914715 podStartE2EDuration="3.288568814s" totalImagesPullingTime="1.282654099s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:46:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:46:57.286621606 +0000 UTC m=+4381.534027648" watchObservedRunningTime="2026-10-30 19:46:57.288568814 +0000 UTC m=+4381.535974839"
Oct 30 19:47:03.212011 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:47:03.211981    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-fxt2k/must-gather-wjwkg" event={"ID":"62c69392-7e2c-47ca-b4b2-6d67802a7689","Type":"ContainerStarted","Data":"1cc8175e919b41a0ebbfa2927ca045832a381301c824aa01c0780c2af67f27de"}
Oct 30 19:47:03.212481 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:47:03.212021    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-fxt2k/must-gather-wjwkg" event={"ID":"62c69392-7e2c-47ca-b4b2-6d67802a7689","Type":"ContainerStarted","Data":"d315a18dde2a35c61861043da50b383d9fb001e8756e7a75442ae8f2fbf0ecd2"}
Oct 30 19:47:03.332206 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:47:03.332139    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-fxt2k/must-gather-wjwkg" podStartSLOduration=1.88271582 podStartE2EDuration="9.332117843s" totalImagesPullingTime="7.449402023s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:46:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:47:03.330687834 +0000 UTC m=+4387.578093858" watchObservedRunningTime="2026-10-30 19:47:03.332117843 +0000 UTC m=+4387.579523868"
Oct 30 19:47:04.701038 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:47:04.700994    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-g2svp_46d9ac39-ee45-4e43-bda7-3931c3e599cf/global-pull-secret-syncer/0.log"
Oct 30 19:47:05.647692 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:47:05.646809    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-jbsw6" containerName="extractor"
Oct 30 19:47:05.938969 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:47:05.938172    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kas-connection-checker-685895c7c9-2grbp_e77c4479-d6e5-48a8-ac82-102a9f1a2c4a/connection-checker/0.log"
Oct 30 19:47:06.473908 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:47:06.473881    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kas-connection-checker-685895c7c9-4b95q_dedb8b54-d0ce-434c-9461-db90a189c121/connection-checker/0.log"
Oct 30 19:47:06.877305 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:47:06.876524    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-776cc66d47-rrwt2_79919fce-ca7a-47ce-bc62-490450a6ac2d/manager/0.log"
Oct 30 19:47:06.943318 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:47:06.937742    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kas-connection-checker-685895c7c9-f9ldt_df9e5074-3bb1-4817-8855-b045cee61004/connection-checker/0.log"
Oct 30 19:47:07.890094 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:47:07.890061    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-lmdpp_8ecf2b62-8053-4bba-a852-c6f6119d3288/konnectivity-agent/0.log"
Oct 30 19:47:08.198194 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:47:08.198106    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-86b6bf7f47-w2sgd_149ec7c2-df24-4c6e-81ff-121bad25f86d/manager/0.log"
Oct 30 19:47:08.394928 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:47:08.394874    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-86b6bf7f47-w2sgd_149ec7c2-df24-4c6e-81ff-121bad25f86d/manager/1.log"
Oct 30 19:47:08.472593 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:47:08.472414    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-776cc66d47-rrwt2_79919fce-ca7a-47ce-bc62-490450a6ac2d/manager/0.log"
Oct 30 19:47:09.532721 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:47:09.532686    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-86b6bf7f47-w2sgd_149ec7c2-df24-4c6e-81ff-121bad25f86d/manager/0.log"
Oct 30 19:47:09.570102 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:47:09.570043    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-134-46.us-west-1.compute.internal_6f66fae2621414751921d90f1913a614/haproxy/0.log"
Oct 30 19:47:09.776965 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:47:09.776930    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-86b6bf7f47-w2sgd_149ec7c2-df24-4c6e-81ff-121bad25f86d/manager/1.log"
Oct 30 19:47:13.344180 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:47:13.344146    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-776cc66d47-rrwt2_79919fce-ca7a-47ce-bc62-490450a6ac2d/manager/0.log"
Oct 30 19:47:14.216001 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:47:14.215964    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-5f5864cd46-846ft_bfa5d691-05e1-48a0-9d53-52975ea89cf4/authorino/0.log"
Oct 30 19:47:14.543621 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:47:14.543543    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-7b7bb4bc77-rvtlv_ea157aed-6416-4a50-af9f-c9ebbc6ee044/manager/0.log"
Oct 30 19:47:14.706469 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:47:14.706421    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-86b6bf7f47-w2sgd_149ec7c2-df24-4c6e-81ff-121bad25f86d/manager/0.log"
Oct 30 19:47:14.891063 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:47:14.891029    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-86b6bf7f47-w2sgd_149ec7c2-df24-4c6e-81ff-121bad25f86d/manager/1.log"
Oct 30 19:47:14.978136 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:47:14.978107    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-8557cbdb84-8dfbp_d78a842d-920b-4619-9cd7-61d8e86ae880/manager/0.log"
Oct 30 19:47:15.141733 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:47:15.141658    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-776cc66d47-rrwt2_79919fce-ca7a-47ce-bc62-490450a6ac2d/manager/0.log"
Oct 30 19:47:15.488820 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:47:15.488749    4888 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-v8nrk/perf-node-gather-daemonset-vkchh"]
Oct 30 19:47:15.503984 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:47:15.503932    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-v8nrk/perf-node-gather-daemonset-vkchh"
Oct 30 19:47:15.579513 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:47:15.579486    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-v8nrk/perf-node-gather-daemonset-vkchh"]
Oct 30 19:47:15.620900 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:47:15.620481    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d72wp\" (UniqueName: \"kubernetes.io/projected/5a5ee75e-33cc-4a5b-8a5c-33ba44395a4c-kube-api-access-d72wp\") pod \"perf-node-gather-daemonset-vkchh\" (UID: \"5a5ee75e-33cc-4a5b-8a5c-33ba44395a4c\") " pod="openshift-must-gather-v8nrk/perf-node-gather-daemonset-vkchh"
Oct 30 19:47:15.620900 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:47:15.620540    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5a5ee75e-33cc-4a5b-8a5c-33ba44395a4c-lib-modules\") pod \"perf-node-gather-daemonset-vkchh\" (UID: \"5a5ee75e-33cc-4a5b-8a5c-33ba44395a4c\") " pod="openshift-must-gather-v8nrk/perf-node-gather-daemonset-vkchh"
Oct 30 19:47:15.620900 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:47:15.620566    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5a5ee75e-33cc-4a5b-8a5c-33ba44395a4c-sys\") pod \"perf-node-gather-daemonset-vkchh\" (UID: \"5a5ee75e-33cc-4a5b-8a5c-33ba44395a4c\") " pod="openshift-must-gather-v8nrk/perf-node-gather-daemonset-vkchh"
Oct 30 19:47:15.620900 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:47:15.620654    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/5a5ee75e-33cc-4a5b-8a5c-33ba44395a4c-podres\") pod \"perf-node-gather-daemonset-vkchh\" (UID: \"5a5ee75e-33cc-4a5b-8a5c-33ba44395a4c\") " pod="openshift-must-gather-v8nrk/perf-node-gather-daemonset-vkchh"
Oct 30 19:47:15.620900 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:47:15.620776    4888 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/5a5ee75e-33cc-4a5b-8a5c-33ba44395a4c-proc\") pod \"perf-node-gather-daemonset-vkchh\" (UID: \"5a5ee75e-33cc-4a5b-8a5c-33ba44395a4c\") " pod="openshift-must-gather-v8nrk/perf-node-gather-daemonset-vkchh"
Oct 30 19:47:15.722225 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:47:15.722194    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d72wp\" (UniqueName: \"kubernetes.io/projected/5a5ee75e-33cc-4a5b-8a5c-33ba44395a4c-kube-api-access-d72wp\") pod \"perf-node-gather-daemonset-vkchh\" (UID: \"5a5ee75e-33cc-4a5b-8a5c-33ba44395a4c\") " pod="openshift-must-gather-v8nrk/perf-node-gather-daemonset-vkchh"
Oct 30 19:47:15.722429 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:47:15.722237    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5a5ee75e-33cc-4a5b-8a5c-33ba44395a4c-lib-modules\") pod \"perf-node-gather-daemonset-vkchh\" (UID: \"5a5ee75e-33cc-4a5b-8a5c-33ba44395a4c\") " pod="openshift-must-gather-v8nrk/perf-node-gather-daemonset-vkchh"
Oct 30 19:47:15.722429 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:47:15.722263    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5a5ee75e-33cc-4a5b-8a5c-33ba44395a4c-sys\") pod \"perf-node-gather-daemonset-vkchh\" (UID: \"5a5ee75e-33cc-4a5b-8a5c-33ba44395a4c\") " pod="openshift-must-gather-v8nrk/perf-node-gather-daemonset-vkchh"
Oct 30 19:47:15.722429 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:47:15.722334    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/5a5ee75e-33cc-4a5b-8a5c-33ba44395a4c-podres\") pod \"perf-node-gather-daemonset-vkchh\" (UID: \"5a5ee75e-33cc-4a5b-8a5c-33ba44395a4c\") " pod="openshift-must-gather-v8nrk/perf-node-gather-daemonset-vkchh"
Oct 30 19:47:15.722429 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:47:15.722377    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5a5ee75e-33cc-4a5b-8a5c-33ba44395a4c-lib-modules\") pod \"perf-node-gather-daemonset-vkchh\" (UID: \"5a5ee75e-33cc-4a5b-8a5c-33ba44395a4c\") " pod="openshift-must-gather-v8nrk/perf-node-gather-daemonset-vkchh"
Oct 30 19:47:15.722667 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:47:15.722420    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5a5ee75e-33cc-4a5b-8a5c-33ba44395a4c-sys\") pod \"perf-node-gather-daemonset-vkchh\" (UID: \"5a5ee75e-33cc-4a5b-8a5c-33ba44395a4c\") " pod="openshift-must-gather-v8nrk/perf-node-gather-daemonset-vkchh"
Oct 30 19:47:15.722667 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:47:15.722442    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/5a5ee75e-33cc-4a5b-8a5c-33ba44395a4c-podres\") pod \"perf-node-gather-daemonset-vkchh\" (UID: \"5a5ee75e-33cc-4a5b-8a5c-33ba44395a4c\") " pod="openshift-must-gather-v8nrk/perf-node-gather-daemonset-vkchh"
Oct 30 19:47:15.722667 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:47:15.722430    4888 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/5a5ee75e-33cc-4a5b-8a5c-33ba44395a4c-proc\") pod \"perf-node-gather-daemonset-vkchh\" (UID: \"5a5ee75e-33cc-4a5b-8a5c-33ba44395a4c\") " pod="openshift-must-gather-v8nrk/perf-node-gather-daemonset-vkchh"
Oct 30 19:47:15.722667 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:47:15.722474    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/5a5ee75e-33cc-4a5b-8a5c-33ba44395a4c-proc\") pod \"perf-node-gather-daemonset-vkchh\" (UID: \"5a5ee75e-33cc-4a5b-8a5c-33ba44395a4c\") " pod="openshift-must-gather-v8nrk/perf-node-gather-daemonset-vkchh"
Oct 30 19:47:15.787519 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:47:15.787442    4888 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d72wp\" (UniqueName: \"kubernetes.io/projected/5a5ee75e-33cc-4a5b-8a5c-33ba44395a4c-kube-api-access-d72wp\") pod \"perf-node-gather-daemonset-vkchh\" (UID: \"5a5ee75e-33cc-4a5b-8a5c-33ba44395a4c\") " pod="openshift-must-gather-v8nrk/perf-node-gather-daemonset-vkchh"
Oct 30 19:47:15.818871 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:47:15.818841    4888 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-v8nrk/perf-node-gather-daemonset-vkchh"
Oct 30 19:47:16.203570 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:47:16.203533    4888 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-v8nrk/perf-node-gather-daemonset-vkchh"]
Oct 30 19:47:16.209541 ip-10-0-134-46 kubenswrapper[4888]: W1030 19:47:16.209509    4888 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod5a5ee75e_33cc_4a5b_8a5c_33ba44395a4c.slice/crio-aa0974da6dc13ad377015a65581137f54599acd840264307cbb48b6fb2c7539e WatchSource:0}: Error finding container aa0974da6dc13ad377015a65581137f54599acd840264307cbb48b6fb2c7539e: Status 404 returned error can't find the container with id aa0974da6dc13ad377015a65581137f54599acd840264307cbb48b6fb2c7539e
Oct 30 19:47:16.265440 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:47:16.265403    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-86b6bf7f47-w2sgd_149ec7c2-df24-4c6e-81ff-121bad25f86d/manager/0.log"
Oct 30 19:47:16.400666 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:47:16.400622    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-v8nrk/perf-node-gather-daemonset-vkchh" event={"ID":"5a5ee75e-33cc-4a5b-8a5c-33ba44395a4c","Type":"ContainerStarted","Data":"8ac424f4b7cdadef6ea6a14801f41e46c3010dc2201b9d4b52bcdd46b4f90c0a"}
Oct 30 19:47:16.400877 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:47:16.400717    4888 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-v8nrk/perf-node-gather-daemonset-vkchh" event={"ID":"5a5ee75e-33cc-4a5b-8a5c-33ba44395a4c","Type":"ContainerStarted","Data":"aa0974da6dc13ad377015a65581137f54599acd840264307cbb48b6fb2c7539e"}
Oct 30 19:47:16.400877 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:47:16.400778    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-v8nrk/perf-node-gather-daemonset-vkchh"
Oct 30 19:47:16.521048 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:47:16.520933    4888 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-v8nrk/perf-node-gather-daemonset-vkchh" podStartSLOduration=1.52091292 podStartE2EDuration="1.52091292s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 19:47:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:47:16.519268489 +0000 UTC m=+4400.766674513" watchObservedRunningTime="2026-10-30 19:47:16.52091292 +0000 UTC m=+4400.768318947"
Oct 30 19:47:16.530133 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:47:16.530084    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-86b6bf7f47-w2sgd_149ec7c2-df24-4c6e-81ff-121bad25f86d/manager/1.log"
Oct 30 19:47:16.706330 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:47:16.706272    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-678d87766f-mx24q_e31ecc7e-2afe-4777-9ae4-50954433a58e/manager/0.log"
Oct 30 19:47:17.019185 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:47:17.019139    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-7c9b87d574-rjmdz_513b0b3f-06d8-42bf-b73c-89f3bae42d98/limitador/0.log"
Oct 30 19:47:17.491561 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:47:17.491530    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-5778bb4449-f9rvt_db5d7a86-9e39-413e-8628-9a7e77270908/manager/0.log"
Oct 30 19:47:19.817504 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:47:19.817480    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-776cc66d47-rrwt2_79919fce-ca7a-47ce-bc62-490450a6ac2d/manager/0.log"
Oct 30 19:47:21.032010 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:47:21.031981    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-86b6bf7f47-w2sgd_149ec7c2-df24-4c6e-81ff-121bad25f86d/manager/0.log"
Oct 30 19:47:21.224274 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:47:21.224236    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-86b6bf7f47-w2sgd_149ec7c2-df24-4c6e-81ff-121bad25f86d/manager/1.log"
Oct 30 19:47:21.375971 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:47:21.375947    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-776cc66d47-rrwt2_79919fce-ca7a-47ce-bc62-490450a6ac2d/manager/0.log"
Oct 30 19:47:22.418896 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:47:22.418868    4888 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-v8nrk/perf-node-gather-daemonset-vkchh"
Oct 30 19:47:22.444524 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:47:22.444493    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-86b6bf7f47-w2sgd_149ec7c2-df24-4c6e-81ff-121bad25f86d/manager/0.log"
Oct 30 19:47:22.619649 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:47:22.619610    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-86b6bf7f47-w2sgd_149ec7c2-df24-4c6e-81ff-121bad25f86d/manager/1.log"
Oct 30 19:47:23.211099 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:47:23.211064    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-884576c57-gmswj_68195e19-b1ff-4148-9fae-3545f2997ff5/cluster-monitoring-operator/0.log"
Oct 30 19:47:25.699258 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:47:25.699222    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-c759s_b1345e7d-caa8-4faf-ac76-e79a57a99e39/node-exporter/0.log"
Oct 30 19:47:26.142247 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:47:26.142201    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-c759s_b1345e7d-caa8-4faf-ac76-e79a57a99e39/kube-rbac-proxy/0.log"
Oct 30 19:47:26.506463 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:47:26.506369    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-8f76c657-bfbg6_e1f8438b-6784-4805-adbb-9d45026b12b6/istio-proxy/0.log"
Oct 30 19:47:26.585264 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:47:26.585236    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-c759s_b1345e7d-caa8-4faf-ac76-e79a57a99e39/init-textfile/0.log"
Oct 30 19:47:28.014040 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:47:28.014008    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-8f76c657-bfbg6_e1f8438b-6784-4805-adbb-9d45026b12b6/istio-proxy/0.log"
Oct 30 19:47:28.698107 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:47:28.698081    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-ai-inference-6d5bbf59c5-wjwcj_f434df49-9016-4649-a140-82f23de7cdfc/istio-proxy/0.log"
Oct 30 19:47:29.356959 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:47:29.356931    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7dfcf9fd6c-vh769_da28351c-1c8b-40be-85a0-3aad427e3a17/router/0.log"
Oct 30 19:47:30.107296 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:47:30.107257    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-ai-inference-6d5bbf59c5-wjwcj_f434df49-9016-4649-a140-82f23de7cdfc/istio-proxy/0.log"
Oct 30 19:47:30.759027 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:47:30.758996    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7dfcf9fd6c-vh769_da28351c-1c8b-40be-85a0-3aad427e3a17/router/0.log"
Oct 30 19:47:34.733894 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:47:34.733851    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-5f5864cd46-846ft_bfa5d691-05e1-48a0-9d53-52975ea89cf4/authorino/0.log"
Oct 30 19:47:34.847174 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:47:34.847135    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-7b7bb4bc77-rvtlv_ea157aed-6416-4a50-af9f-c9ebbc6ee044/manager/0.log"
Oct 30 19:47:35.061933 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:47:35.061855    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-8557cbdb84-8dfbp_d78a842d-920b-4619-9cd7-61d8e86ae880/manager/0.log"
Oct 30 19:47:35.625643 ip-10-0-134-46 kubenswrapper[4888]: E1030 19:47:35.625594    4888 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-jbsw6" containerName="extractor"
Oct 30 19:47:36.276951 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:47:36.276914    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-678d87766f-mx24q_e31ecc7e-2afe-4777-9ae4-50954433a58e/manager/0.log"
Oct 30 19:47:36.436263 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:47:36.436225    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-7c9b87d574-rjmdz_513b0b3f-06d8-42bf-b73c-89f3bae42d98/limitador/0.log"
Oct 30 19:47:36.471724 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:47:36.469309    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-5f5864cd46-846ft_bfa5d691-05e1-48a0-9d53-52975ea89cf4/authorino/0.log"
Oct 30 19:47:36.581153 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:47:36.581121    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-7b7bb4bc77-rvtlv_ea157aed-6416-4a50-af9f-c9ebbc6ee044/manager/0.log"
Oct 30 19:47:36.687338 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:47:36.687305    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-5778bb4449-f9rvt_db5d7a86-9e39-413e-8628-9a7e77270908/manager/0.log"
Oct 30 19:47:36.792799 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:47:36.792766    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-8557cbdb84-8dfbp_d78a842d-920b-4619-9cd7-61d8e86ae880/manager/0.log"
Oct 30 19:47:37.778132 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:47:37.778097    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-678d87766f-mx24q_e31ecc7e-2afe-4777-9ae4-50954433a58e/manager/0.log"
Oct 30 19:47:37.905404 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:47:37.905376    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-7c9b87d574-rjmdz_513b0b3f-06d8-42bf-b73c-89f3bae42d98/limitador/0.log"
Oct 30 19:47:38.152367 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:47:38.152339    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-5778bb4449-f9rvt_db5d7a86-9e39-413e-8628-9a7e77270908/manager/0.log"
Oct 30 19:47:38.938632 ip-10-0-134-46 kubenswrapper[4888]: I1030 19:47:38.938599    4888 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-8f76c657-bfbg6_e1f8438b-6784-4805-adbb-9d45026b12b6/istio-proxy/0.log"